Conditional Distributions and Regular Conditional Probability — Examples

For a positive-mass cell, preimages under X preserve disjoint unions, so $A\mapsto P(H_j\cap X^{-1}(A))$ is countably additive, vanishes at the empty set and has total mass $P(H_j)$. Division by this positive finite mass gives a probability. On zero-mass cells the supplied $\rho$ is a probability. For each A the evaluation is constant on every cell and is therefore measurable for the finite partition sigma-algebra. Every event H in that sigma-algebra is a union of cells: the set of such unions is itself a sigma-algebra containing the cells. Thus ${\int }_{H}K(\omega ,A)dP={\sum }_{j:H_j\subseteq H}P(H_j)K{\mathrm{\mid }}_{H_j}(A)={\sum }_{j:H_j\subseteq H}P(H_j\cap \{X\in A\})=P(H\cap \{X\in A\})\mathrm{.}$ Each zero cell contributes zero on both sides, and an empty cell can be ignored. This proves [F1]–[F2].

In the displayed finite model the cell masses are $1\mathrm{/}2,1\mathrm{/}2,0$. On the first cell, $P(X=0,H_1)=1\mathrm{/}4$ and $P(X=1,H_1)=1\mathrm{/}4$, so the conditional probabilities are $1\mathrm{/}2$ and $1\mathrm{/}2$. On the second cell, $P(X=1,H_2)=1\mathrm{/}2$ gives probability one at 1. The third uses the specified filler despite $X(4)=2$, since its entire cell has mass zero. For instance testing A={1} and H=Omega gives $(1\mathrm{/}2)(1\mathrm{/}2)+(1\mathrm{/}2)(1)+0(0)=3\mathrm{/}4=P(X=1)$; testing H=$H_1$ gives $1\mathrm{/}4$ on both sides. Hence the calculated kernels have exactly the claimed values.

Write $\varphi (t)=e^{-t^2\mathrm{/}2}\mathrm{/}\sqrt{2\pi }$ and $g_{a,s}(x)=s^{-1}\varphi ((x-a)\mathrm{/}s)$ for s>0. For any compact interval [b,d], use [F7] with affine map $(x-a)\mathrm{/}s$ and continuous outer function phi; its derivative is the integrable constant 1/s. By [F5] this proves the same substitution for Lebesgue integrals. Let b decrease to minus infinity and d increase to infinity, using [F9], to get $\int g_{a,s}=1$ from [F2]. For a fixed upper endpoint z, the identical limiting argument gives ${\int }_{-\mathrm{\infty }}^z{g}_{a,s}(x)dx={\int }_{-\mathrm{\infty }}^{(z-a)\mathrm{/}s}\varphi (t)dt$. This is the CDF of the affine law [F3], since s>0. Thus [F10] identifies the density law with $N(a,s^2)$. All countable-choice hypotheses are supplied by [F6].

Completing the square gives $u^2-2ruv+v^2=(u-rv{)}^2+(1-r^2)v^2$. Put $a(y)=m_X+r{\sigma }_{X}v$ and $s={\sigma }_X\sqrt{1-r^2}>0$. Direct substitution into the displayed p yields $p(x,y)=g_{m_Y,{\sigma }_Y}(y)g_{a(y),s}(x)\mathrm{.}$ It is positive and product-measurable: it is obtained from measurable coordinate projections by continuous arithmetic and exponential operations with fixed nonzero denominators. Step 1.1 and [F4] give marginal $m(y)=g_{m_Y,{\sigma }_Y}(y)$ and total mass $\int m=1$. Hence [F8] constructs the joint probability. Its marginal is finite positive at every y. Apply [F1]: the normalized section is exactly $g_{a(y),s}$, which step 1.1 identifies as the asserted normal law. No exceptional filler is needed here. When r=0 this conditional density is independent of y and has the original X parameters.

For completeness the parameters have their claimed moment meanings. By [F12]–[F13], $\varphi \mathrm{\prime }(t)=-t\varphi (t)$. On [-R,R], [F11] with factors t and phi gives ${\int }_{-R}^R{t}^2\varphi (t)dt={\int }_{-R}^R\varphi (t)dt-2R\varphi (R)$. The derivatives are continuous, hence satisfy its compact integrability hypotheses, and [F5] converts to Lebesgue integrals. Since $\varphi \mathrm{\prime }(t)=-t\varphi (t)\le 0$ for $t\ge 0$, $R\varphi (R)\le 2{\int }_{R\mathrm{/}2}^R\varphi (t)dt\to 0$, using the finite integral in [F2]. By [F9] the second moment is one. The bound $\mathrm{\mid }t\mathrm{\mid }\le 1+t^2$ gives finite first absolute moment; symmetry and substitution t to -t give mean zero. Affine substitution now gives $\int x{g}_{a,s}(x)dx=a$ and $\int (x-a{)}^2{g}_{a,s}(x)dx=s^2$, using [F14] for the density interpretation.

Using the factorization of step 2.1 and step 2.2, nonnegative Tonelli gives $E(Y-m_Y{)}^2={\sigma }_Y^2$ and $E(X-m_X{)}^2=\int (s^2+(a(y)-m_X{)}^2)m(y)dy={\sigma }_X^2(1-r^2)+r^2{\sigma }_X^2={\sigma }_X^2\mathrm{.}$ This also gives finite absolute first moments, so [F15] permits signed integration and yields $EX=m_X$, $EY=m_Y$. The product $(X-m_X)(Y-m_Y)$ is absolutely integrable because $2\mathrm{\mid }ab\mathrm{\mid }\le a^2+b^2$. Thus [F15] again gives $E[(X-m_X)(Y-m_Y)]=\int r\frac{{\sigma }_X}{{\sigma }_Y}(y-m_Y{)}^{2}m(y)dy=r{\sigma }_X{\sigma }_Y\mathrm{.}$ Division by the positive standard deviations gives correlation r. At $\mathrm{\mid }r\mathrm{\mid }=1$ the displayed density denominator and conditional scale cease to be positive, so neither the normalized density nor this density argument asserts that singular case.

For each y the section $K(y,\cdot )=P_X$ is a probability measure; for each A the evaluation is constant and therefore measurable. For $B\in \mathcal{T}$, independence through [F3] gives ${\int }_{\{Y\in B\}}K(Y,A)dP=P_X(A)P_Y(B)=P(X\in A,Y\in B)\mathrm{.}$ Every event in $\sigma (Y)$ is of this form, because the inverse images of all measurable B already form a sigma-algebra. This proves [F1], while the first two observations prove [F2].

For a concrete calculation, put probability 1/6 at each point of $\{0,1\}\times \{0,1,2\}$ and let X,Y be the two coordinates. Each pair has probability $(1\mathrm{/}2)(1\mathrm{/}3)$, so their marginal rectangle probabilities factor and the coordinates are independent. The kernel gives $K(y,\{1\})=1\mathrm{/}2$ for every y. For $B=\{0,2\}$ its event integral is $(1\mathrm{/}2)(2\mathrm{/}3)=1\mathrm{/}3$, equal to the mass of the two points $(1,0),(1,2)$. If either variable is deterministic the same formula applies, with a Dirac marginal where appropriate.

For j=0 the integral of $g_0$ is one by [F2]. For j=1, the translation $t=y-1$ has derivative one; [F6] with continuous outer phi, then [F8] and [F7] on increasing compact intervals, gives $\int g_1=\int \varphi =1$. The same calculation on $(-\mathrm{\infty },z]$ gives ${\int }_{-\mathrm{\infty }}^z{g}_j={\int }_{-\mathrm{\infty }}^{z-j}\varphi$, the CDF of the translated standard law [F3]. Therefore [F9] identifies it as $N(j,1)$. Both functions are positive measurable and finite. By [F5] they define probability likelihoods; measurability in the discrete parameter j is automatic because the parameter set is finite. The joint density $(j,y)\mapsto g_j(y)$ is measurable since each of its two sections is, and the two slices are measurable. The declared AC [F4] covers all analytic choice assumptions.

Take prior $\pi =p{\delta }_0+(1-p){\delta }_1$ on the two-point space and observation Lebesgue measure, which is sigma-finite. In [F1] the marginal density is $m(y)=p{g}_0(y)+(1-p)g_1(y)>0$ and finite for every real y. Its total mass is $p+(1-p)=1$ by step 1.1. The posterior mass of {1} is therefore the first displayed ratio, and that of {0} is $1-w(y)$; these sum to one. Dividing numerator and denominator by $g_0(y)>0$ and computing $g_1(y)\mathrm{/}{g}_0(y)=\exp ((y^2-(y-1{)}^2)\mathrm{/}2)=e^{y-1\mathrm{/}2}$ gives the second ratio. At $y=1\mathrm{/}2$ the exponential is one, so $w(1\mathrm{/}2)=1-p$. For p=1/2, for example, $w(0)=1\mathrm{/}(1+e^{1\mathrm{/}2})$ and $w(1)=e^{1\mathrm{/}2}\mathrm{/}(1+e^{1\mathrm{/}2})$. Finally for every Borel B, ${\int }_{B}w(y)m(y)dy=(1-p){\int }_B{g}_1(y)dy=P(J=1,Y\in B)$, directly verifying the posterior event calculation.

The triangle is product-measurable, since its defining strict inequalities between coordinates are open conditions (or countable rational rectangle unions). For $0<y<1$ the section integral is $m(y)={\int }_0^{y}2dx=2y$, and it is zero for all other y. The constant primitive 2x computes this integral by [F2] and [F3]; finite endpoints are Lebesgue-null, as follows from containment in intervals of arbitrarily small length. Likewise ${\int }_0^{1}2ydy=[y^2{]}_0^1=1$. Thus [F5] shows the nonnegative joint density has total mass one, and [F6] constructs the probability. The bridge uses the countable choice supplied by [F4].

On $0<y<1$, $0<m(y)=2y<\mathrm{\infty }$ and ${\int }_{A}p(x,y)dx=2{\lambda }_1(A\cap (0,y))$. Dividing gives the displayed uniform kernel. Off this interval the marginal is zero, so [F1] allows the supplied point-mass probability ${\delta }_0$. Its event value is ${\mathbf{1}}_A(0)$ and it is countably additive because at most one member of a disjoint event sequence contains 0. For any Borel A,B, the conditional rectangle calculation is ${\int }_{B}K(y,A)P_Y(dy)={\int }_{B\cap (0,1)}\frac{{\lambda }_1(A\cap (0,y))}{y}2ydy={\int }_{B\cap (0,1)}2{\lambda }_1(A\cap (0,y))dy,$ the joint probability by [F5]. In particular $K(1\mathrm{/}2,(0,1\mathrm{/}4))=(1\mathrm{/}4)\mathrm{/}(1\mathrm{/}2)=1\mathrm{/}2$, while $K(0,(0,1\mathrm{/}4))=0$ under the specified filler. The latter is not a value of a density ratio.

For fixed s, $A\mapsto {\mathbf{1}}_A(g(s))$ vanishes on the empty set and equals one on T. If $A_j$ are disjoint, at most one contains g(s), so ${\mathbf{1}}_{{\bigcup }_j{A}_j}(g(s))={\sum }_j{\mathbf{1}}_{A_j}(g(s))$. This is countable additivity, so the section is the Dirac probability at g(s). For a measurable A, the evaluation is ${\mathbf{1}}_{g^{-1}(A)}(s)$; its set $g^{-1}(A)$ is measurable by hypothesis. These are exactly [F1].

By [F2] and [F3], for a measurable $C\subseteq U$, $(K_g{K}_h)(s,C)={\int }_T{\mathbf{1}}_C(h(t))K_g(s,dt)=K_g(s,h^{-1}(C))={\mathbf{1}}_C(h(g(s)))\mathrm{.}$ The middle equality is the integral of an indicator of the measurable set $h^{-1}(C)$. The composite h after g is measurable because $(h\circ g{)}^{-1}(C)=g^{-1}(h^{-1}(C))$. This proves the formula at every s, with no reference to a null set. For example on real Borel spaces take $g(s)=s^2$ and $h(t)=t+1$. Then the composite section is ${\delta }_{s^2+1}$; at s=2 its mass on (4,6) is one and on (0,4] is zero. If S is empty the assertions are vacuous; an empty T with nonempty S cannot support the assumed map g.

The density p is the indicator of a Borel rectangle. By [F3]–[F5], ${\int }_0^{1}1dx=1$; endpoints have measure zero by containment in intervals with arbitrarily small lengths. Tonelli [F6] gives total joint mass $1\cdot 1=1$ and marginal $m(y)={\mathbf{1}}_{(0,1)}(y)$. The measure construction is [F7]. At y=2, $p(x,2)=0$ for every x and $m(2)=0$. The asserted quotient is therefore 0/0, which is undefined, at every x on that fibre. Thus the claimed everywhere formula fails for a fully normalized bounded joint density.

Put $\rho (A)={\lambda }_1(A\cap (0,1))$ and $K(y,A)=\rho (A)$ for all real y. By [F7] and the mass calculation, rho is a probability, and constant evaluations are measurable, proving [F2]. On $0<y<1$ this agrees with the density ratio. On its complement it is the supplied filler allowed by [F1]. Directly, for Borel A,B, ${\int }_{B}K(y,A)P_Y(dy)={\lambda }_1(A\cap (0,1)){\lambda }_1(B\cap (0,1))=P(X\in A,Y\in B),$ with the last equality from [F6]. So the extension is a conditional law, including an everywhere probability section at y=2. For example $K(2,(0,1\mathrm{/}2))=1\mathrm{/}2$, a chosen valid extension value, not a value of 0/0.

Take $\mathrm{\Omega }=(0,1)$ with Borel sigma-algebra and Lebesgue probability, Y(omega)=omega and X(omega)=0. The mass is one by [F3]–[F6], integrating the constant derivative of x on [0,1] and ignoring its null endpoints. The singleton $N=\{1\mathrm{/}2\}$ is Borel and has measure zero: for every positive integer n it is contained in an interval of length $2\mathrm{/}n$, so its measure is at most $2\mathrm{/}n$ and hence zero. Each Dirac section is a probability because for disjoint sets at most one contains its point. For Borel A, $L(y,A)={\mathbf{1}}_A(0)+{\mathbf{1}}_{\{1\mathrm{/}2\}}(y)({\mathbf{1}}_A(1)-{\mathbf{1}}_A(0)),$ a measurable function; K has constant measurable evaluations. Thus both satisfy [F2].

For every $H\in \sigma (Y)$ and Borel A, ${\int }_{H}K(Y,A)dP={\mathbf{1}}_A(0)P(H)=P(H\cap \{X\in A\})$. The difference $L(Y,A)-K(Y,A)$ is bounded in absolute value by ${\mathbf{1}}_N(Y)$, whose integral over H is zero. Therefore L satisfies the same identity, proving [F1] for both kernels. They nevertheless disagree at y=1/2: for the event A={1}, $K(1\mathrm{/}2,A)=0$ whereas $L(1\mathrm{/}2,A)=1$. This is a difference of probability measures at an actual conditioning value in (0,1), not merely outside the range of Y. Their equality outside N is consistent with almost-everywhere uniqueness.