Category of Markov kernels

In mathematics, the category of Markov kernels, often denoted Stoch, is the category whose objects are measurable spaces and whose morphisms are Markov kernels. ^[1][2][3][4] It is analogous to the category of sets and functions, but where the arrows can be interpreted as being stochastic.

Several variants of this category are used in the literature. For example, one can use subprobability kernels^[5] instead of probability kernels, or more general s-finite kernels.^[6] Also, one can take as morphisms equivalence classes of Markov kernels under almost sure equality;^[7] see below.

Definition

Recall that a Markov kernel between measurable spaces ${\displaystyle (X,{\mathcal {F}})}$ and ${\displaystyle (Y,{\mathcal {G}})}$ is an assignment ${\displaystyle k:X\times {\mathcal {G}}\to \mathbb {R} }$ which is measurable as a function on ${\displaystyle X}$ and which is a probability measure on ${\displaystyle {\mathcal {G}}}$.^[4] We denote its values by ${\displaystyle k(B|x)}$ for ${\displaystyle x\in X}$ and ${\displaystyle B\in {\mathcal {G}}}$, which suggests an interpretation as conditional probability.

The category Stoch has:^[4]

• As objects, measurable spaces;
• As morphisms, Markov kernels between them;
• For each measurable space ${\displaystyle (X,{\mathcal {F}})}$, the identity morphism is given by the kernel

${\displaystyle \delta (A|x)=1_{A}(x)={\begin{cases}1&x\in A;\\0&x\notin A\end{cases}}}$

for all ${\displaystyle x\in X}$ and ${\displaystyle A\in {\mathcal {F}}}$;

• Given kernels ${\displaystyle k:(X,{\mathcal {F}})\to (Y,{\mathcal {G}})}$ and ${\displaystyle h:(Y,{\mathcal {G}})\to (Z,{\mathcal {H}})}$, the composite morphism ${\displaystyle h\circ k:(X,{\mathcal {F}})\to (Z,{\mathcal {H}})}$ is given by

${\displaystyle (h\circ k)(C|x)=\int _{Y}h(C|y)\,k(dy|x)}$

for all ${\displaystyle x\in X}$ and ${\displaystyle C\in {\mathcal {H}}}$.

This composition formula is sometimes called the Chapman-Kolmogorov equation.^[4]

This composition is unital, and associative by the monotone convergence theorem, so that one indeed has a category.

Basic properties

Probability measures

The terminal object of Stoch is the one-point space ${\displaystyle 1}$.^[4] Morphisms in the form ${\displaystyle 1\to X}$ can be equivalently seen as probability measures on ${\displaystyle X}$, since they correspond to functions ${\displaystyle 1\to PX}$, i.e. elements of ${\displaystyle PX}$.

Given kernels ${\displaystyle p:1\to X}$ and ${\displaystyle k:X\to Y}$, the composite kernel ${\displaystyle k\circ p:1\to Y}$ gives the probability measure on ${\displaystyle Y}$ with values

${\displaystyle (k\circ p)(B)=\int _{X}k(B|x)\,p(dx),}$

for every measurable subset ${\displaystyle B}$ of ${\displaystyle Y}$.^[7]

Given probability spaces ${\displaystyle (X,{\mathcal {F}},p)}$ and ${\displaystyle (Y,{\mathcal {G}},q)}$, a measure-preserving Markov kernel ${\displaystyle (X,{\mathcal {F}},p)\to (Y,{\mathcal {G}},q)}$ is a Markov kernel ${\displaystyle k:(X,{\mathcal {F}})\to (Y,{\mathcal {G}})}$ such that for every measurable subset ${\displaystyle B\in {\mathcal {G}}}$,^[7]

${\displaystyle q(B)=\int _{X}k(B|x)\,p(dx).}$

Probability spaces and measure-preserving Markov kernels form a category, which can be seen as the slice category ${\displaystyle (\mathrm {Hom} _{\mathrm {Stoch} }(1,-),\mathrm {Stoch} )}$.

Measurable functions

Every measurable function ${\displaystyle f:(X,{\mathcal {F}})\to (Y,{\mathcal {G}})}$ defines canonically a Markov kernel ${\displaystyle \delta _{f}:(X,{\mathcal {F}})\to (Y,{\mathcal {G}})}$ as follows,

${\displaystyle \delta _{f}(B|x)=1_{B}(f(x))={\begin{cases}1&f(x)\in B;\\0&f(x)\notin B\end{cases}}}$

for every ${\displaystyle x\in X}$ and every ${\displaystyle B\in {\mathcal {G}}}$. This construction preserves identities and compositions, and is therefore a functor from Meas to Stoch.

Isomorphisms

By functoriality, every isomorphism of measurable spaces (in the category Meas) induces an isomorphism in Stoch. However, in Stoch there are more isomorphisms, and in particular, measurable spaces can be isomorphic in Stoch even when the underlying sets are not in bijection.

Relationship with other categories

• Stoch is the Kleisli category of the Giry monad.^[3][4] This in particular implies that there is an adjunction

${\displaystyle \mathrm {Hom} _{\mathrm {Stoch} }(X,Y)\cong \mathrm {Hom} _{\mathrm {Meas} }(X,PY)}$

between Stoch and the category of measurable spaces.

• The left adjoint ${\displaystyle L:\mathrm {Meas} \to \mathrm {Stoch} }$ of the adjunction above is the identity on objects, and on morphisms it gives the canonical Markov kernel induced by a measurable function described above.

• As mentioned above, one can construct a category of probability spaces and measure-preserving Markov kernels as the slice category ${\displaystyle (\mathrm {Hom} _{\mathrm {Stoch} }(1,-),\mathrm {Stoch} )}$.

• Similarly, the category of probability spaces can be seen as the comma category ${\displaystyle (\mathrm {Hom} _{\mathrm {Stoch} }(1,-),L)}$.

Particular limits and colimits

Since the functor ${\displaystyle L:\mathrm {Meas} \to \mathrm {Stoch} }$ is left adjoint, it preserves colimits.^[8] Because of this, all colimits in the category of measurable spaces are also colimits in Stoch. For example,

• The initial object is the empty set, with its trivial measurable structure;
• The coproduct is given by the disjoint union of measurable spaces, with its canonical sigma-algebra.
• The sequential colimit of a decreasing filtration is given by the intersection of sigma-algebras.

In general, the functor ${\displaystyle L}$ does not preserve limits. This in particular implies that the product of measurable spaces is not a product in Stoch in general. Since the Giry monad is monoidal, however, the product of measurable spaces still makes Stoch a monoidal category.^[4]

A limit of particular significance for probability theory is de Finetti's theorem, which can be interpreted as the fact that the space of probability measures (Giry monad) is the limit in Stoch of the diagram formed by finite permutations of sequences.

Almost sure version

Sometimes it is useful to consider Markov kernels only up to almost sure equality, for example when talking about disintegrations or about regular conditional probability.

Given probability spaces ${\displaystyle (X,{\mathcal {F}},p)}$ and ${\displaystyle (Y,{\mathcal {G}},q)}$, we say that two measure-preserving kernels ${\displaystyle k,h:(X,{\mathcal {F}},p)\to (Y,{\mathcal {G}},q)}$ are almost surely equal if and only if for every measurable subset ${\displaystyle B\in {\mathcal {G}}}$,

${\displaystyle k(B|x)=h(B|x)}$

for ${\displaystyle p}$-almost all ${\displaystyle x\in X}$.^[7] This defines an equivalence relation on the set of measure-preserving Markov kernels ${\displaystyle k,h:(X,{\mathcal {F}},p)\to (Y,{\mathcal {G}},q)}$.

Probability spaces and equivalence classes of Markov kernels under the relation defined above form a category. When restricted to standard Borel probability spaces, the category is often denoted by Krn.^[7]

See also

• Measurable space
• Markov kernel
• Categorical probability