| |
|
Kullback-Leibler EntropyAlso known as the relative entropy, the Kullback-Leibler entropy is a measure of difference
between two random variables. The definitions are:
 for the continuous case and:  for the discrete case, where f(·) and g(·) are the probability densities of the variable that we are comparing. This measure is also called the Kullback-Leibler distance since it exhibits some distance characteristics, that of being positive and that of being equal to zero if f (x) = g (x). It is not a real distance though since K (f,g)≠K (g,f).
|
