# Kullback–Leibler divergence

#### Dear Wikiwand AI, let's keep it short by simply answering these key questions:

Can you list the top facts and stats about Kullback–Leibler divergence?

In mathematical statistics, the Kullback–Leibler divergence (also called relative entropy and I-divergence[1]), denoted ${\displaystyle D_{\text{KL}}(P\parallel Q)}$, is a type of statistical distance: a measure of how one probability distribution P is different from a second, reference probability distribution Q.[2][3] A simple interpretation of the KL divergence of P from Q is the expected excess surprise from using Q as a model when the actual distribution is P. While it is a measure of how different two distributions are, and in some sense is thus a "distance", it is not actually a metric, which is the most familiar and formal type of distance. In particular, it is not symmetric in the two distributions (in contrast to variation of information), and does not satisfy the triangle inequality. Instead, in terms of information geometry, it is a type of divergence,[4] a generalization of squared distance, and for certain classes of distributions (notably an exponential family), it satisfies a generalized Pythagorean theorem (which applies to squared distances).[5]