Loss function (or cost function) is widely used for measuring the differences between predicted goal and real value. In different scenario, user should choose different loss function. This blog will introduce some markable loss function and their usage conditions and disadvantages.
KL Divergence / Relative Entropy
Mean Square Error
$$ \lambda(x) = C (t-x)^2 $$
Huber Loss / Smooth Mean Absolute Error
Log Cosh Loss
Mean Squared Error (MSE), also known as mean squared deviation (MSD) in statics.
Mean Absolute Error (MAE)