Finite-alphabet conditional entropy given an observation

Let $X$ take values in a finite nonempty set $S$ and let $Y$ take values in a standard-Borel space $(T,\mathcal{T})$. Let $K:T⇝S$ be a conditional law of $X$ given $Y$, so that $K(Y,\cdot )$ is a regular conditional distribution given $\sigma (Y)$. Define $H(X\mid Y):={\int }_T(-{\sum }_{x\in S}K(y,\{x\})\ln K(y,\{x\}))P_Y(dy),$ with $0\ln 0:=0$ and natural logarithms. The finite entropy convention is that of Finite distributions, entropy, divergence, and stochastic matrices. Each evaluation $y\mapsto K(y,\{x\})$ is measurable. The explicit argument in Kullback–Leibler divergence proves $-t\ln t$ extends continuously to zero and is bounded by $1\mathrm{/}e$ on $[0,1]$; hence the finite entropy integrand is measurable and bounded by $\mathrm{\mid }S\mathrm{\mid }\mathrm{/}e$. The result is independent of the supplied version without a choice axiom: the finite supplied singleton family $\{\{x\}:x\in S\}$ determines probability measures on $S$ by finite additivity. Apply the supplied-determining-family argument in Conditional law given a random element to two supplied kernels $K,L$. Outside the finite union of their singleton-evaluation discrepancy sets, which is a measurable $P_Y$-null set, all probabilities $K(y,\{x\})$ and $L(y,\{x\})$ agree. Their entropy integrands therefore agree almost everywhere, proving equality of the integrals. This uses supplied kernels, not a theorem asserting their existence.

This definition permits a non-finite observation Y; only the target alphabet S is finite. If Y is finite, then for each positive-mass y the conditional-law identity gives $K(y,\{x\})P_Y(\{y\})=P(X=x,Y=y)$, so $K(y,\{x\})=p_{X\mathrm{\mid }y}(x)$. Evaluating the integral as a finite weighted sum therefore gives exactly the convention of Finite conditional entropy, by Expectation agrees with the published finite weighted sum.