Top Qs
Timeline
Chat
Perspective
Data processing inequality
Concept in information processing From Wikipedia, the free encyclopedia
Remove ads
The data processing inequality is an information theoretic concept that states that the information content of a signal cannot be increased via a local physical operation. This can be expressed concisely as 'post-processing cannot increase information'.[1]
Statement
Let three random variables form the Markov chain , implying that the conditional distribution of depends only on and is conditionally independent of . Specifically, we have such a Markov chain if the joint probability mass function can be written as
In this setting, no processing of , deterministic or random, can increase the information that contains about . Using the mutual information, this can be written as :
with the equality if and only if . That is, and contain the same information about , and also forms a Markov chain.[2]
Remove ads
Proof
One can apply the chain rule for mutual information to obtain two different decompositions of :
By the relationship , we know that and are conditionally independent, given , which means the conditional mutual information, . The data processing inequality then follows from the non-negativity of .
Remove ads
See also
References
External links
Wikiwand - on
Seamless Wikipedia browsing. On steroids.
Remove ads