Modes of Convergence for Random Variables

By [L1], the convergence set has the following countable description. [L1] ${\bigcap }_{r\in {\mathbb{Q}}_{>0}}{\bigcup }_{N=0}^{\mathrm{\infty }}{\bigcap }_{n\ge N}\{\mathrm{\mid }{X}_n-X\mathrm{\mid }<r\}\mathrm{.}$

Each set $\{\mathrm{\mid }{X}_n-X\mathrm{\mid }<r\}$ in the display is measurable because $X_n-X$ is a real random variable and the interval $(-r,r)$ is Borel. Countable unions and intersections preserve measurability. Thus the displayed set, and hence the convergence event, is measurable.

For $a>0$, the triangle inequality gives the containment [given] $\{\mathrm{\mid }X-Y\mathrm{\mid }>a\}\subseteq \{\mathrm{\mid }X-X_n\mathrm{\mid }>a\mathrm{/}2\}\cup \{\mathrm{\mid }{X}_n-Y\mathrm{\mid }>a\mathrm{/}2\}\mathrm{.}$

Taking probabilities in step 1.1 and then limits gives the following. [step 1.1, L1] $\mathbb{P}(\mathrm{\mid }X-Y\mathrm{\mid }>a)=0$ for every $a>0$. The union over $a=1\mathrm{/}m$ is $\{X\mathrm{\ne }Y\}$, so it is null.

Fix $\epsilon >0$ and set $I_n={\mathbf{1}}_{\{\mathrm{\mid }{X}_n-X\mathrm{\mid }>\epsilon \}}$. [given, L1] By [L1] these are measurable; the hypothesis gives $I_n\to 0$ almost surely, and $0\le I_n\le 1$.

Apply [L2] to the indicators from step 1.1 with majorant $1$. [step 1.1, L2] $\mathbb{P}(\mathrm{\mid }{X}_n-X\mathrm{\mid }>\epsilon )=\mathbb{E}{I}_n\to 0$, which is the required probability convergence.

For $\epsilon >0$, apply [L1] to $Z=\mathrm{\mid }{X}_n-X{\mathrm{\mid }}^p$ with $a={\epsilon }^p$ to obtain [L1] $\mathbb{P}(\mathrm{\mid }{X}_n-X\mathrm{\mid }>\epsilon )\le {\epsilon }^{-p}\mathbb{E}\mathrm{\mid }{X}_n-X{\mathrm{\mid }}^p\mathrm{.}$

The bound in step 1.1 tends to $0$ by [L2], so the definition of convergence in probability applies.

Apply [L2] to $X_n-X$ and use [L1] to obtain the following bound. [L1, L2] $\mathrm{\parallel }{X}_n-X{\mathrm{\parallel }}_p\le \mathrm{\parallel }{X}_n-X{\mathrm{\parallel }}_q$.

The right-hand side in step 1.1 tends to zero by the given $L^q$ convergence, so the left-hand side does too. This is $L^p$ convergence.

Fix a continuity point $x$ of $F_X$ and $\delta >0$. The following inclusions give a CDF squeeze. [given] $\{X\le x-\delta \}\setminus \{\mathrm{\mid }{X}_n-X\mathrm{\mid }>\delta \}\subseteq \{X_n\le x\}\subseteq \{X\le x+\delta \}\cup \{\mathrm{\mid }{X}_n-X\mathrm{\mid }>\delta \}$ give $F_X(x-\delta )-p_n\le F_{X_n}(x)\le F_X(x+\delta )+p_n,$ where $p_n=\mathbb{P}(\mathrm{\mid }{X}_n-X\mathrm{\mid }>\delta )$.

By [L2], $p_n\to 0$; taking liminf and limsup in step 1.1 gives the required limiting bounds. [step 1.1, L1, L2] $\delta \downarrow 0$ uses continuity at $x$ to give $F_{X_n}(x)\to F_X(x)$. By [L1], this is $X_n\Rightarrow X$.

The constant-law CDF is continuous at $c-\epsilon$ and $c+\epsilon$, so [L1] gives [L1] $F_{X_n}(c-\epsilon )\to 0$ and $F_{X_n}(c+\epsilon )\to 1$.

The error event is contained in the following union. [step 1.1] $\{X_n\le c-\epsilon \}\cup \{X_n>c+\epsilon \}$, its probability is at most $F_{X_n}(c-\epsilon )+1-F_{X_n}(c+\epsilon )$, which tends to zero.

Recursively choose $n_k>n_{k-1}$ least subject to the following bound. [L1, construct] $\mathbb{P}(\mathrm{\mid }{X}_{n_k}-X\mathrm{\mid }>2^{-k})<2^{-k}$; [L1] makes every choice possible. Put $A_k=\{\mathrm{\mid }{X}_{n_k}-X\mathrm{\mid }>2^{-k}\}$.

The bounds in step 1.1 are summable, so [L2] applies. [step 1.1, L2, discharge-construct] It says only finitely many $A_k$ occur almost surely. Hence $\mathrm{\mid }{X}_{n_k}-X\mathrm{\mid }\le 2^{-k}$ eventually almost surely, so $X_{n_k}\to X$ almost surely.

If $X_n\to X$ in probability, every subsequence has the same property. Apply [L2] to that subsequence to obtain the asserted further subsequence.

Conversely, if probability convergence failed, some $\epsilon >0$ and a subsequence would satisfy $\mathbb{P}(\mathrm{\mid }{X}_{n_k}-X\mathrm{\mid }>\epsilon )\ge \epsilon$ for every $k$. Any almost-surely convergent further subsequence would converge in probability by [L1], a contradiction.

The displayed integrand is measurable, nonnegative and bounded by $1$, so its expectation is finite. Almost-sure replacement of either representative leaves the integrand unchanged almost surely and hence leaves its expectation unchanged by [L1]. Symmetry is immediate. The real triangle inequality and $\min (1,a+b)\le \min (1,a)+\min (1,b)$ for $a,b\ge 0$ give $\min (1,\mathrm{\mid }X-Z\mathrm{\mid })\le \min (1,\mathrm{\mid }X-Y\mathrm{\mid })+\min (1,\mathrm{\mid }Y-Z\mathrm{\mid })$ pointwise. Taking expectations gives the triangle inequality.

For $0<\epsilon \le 1$, on $\{\mathrm{\mid }{X}_n-X\mathrm{\mid }>\epsilon \}$ the integrand is at least $\epsilon$, while on its complement it is at most $\epsilon$ and everywhere it is at most $1$. Splitting the expectation over those events yields $\epsilon \mathbb{P}(\mathrm{\mid }{X}_n-X\mathrm{\mid }>\epsilon )\le d_0([X_n],[X])\le \epsilon +\mathbb{P}(\mathrm{\mid }{X}_n-X\mathrm{\mid }>\epsilon )$.

If $d_0([X],[Y])=0$, [L2] makes $\min (1,\mathrm{\mid }X-Y\mathrm{\mid })=0$ almost surely, hence $X=Y$ almost surely. Conversely, almost-sure equality makes the expectation zero. Thus $d_0$ is a metric on almost-sure classes.

If $d_0([X_n],[X])\to 0$, the first inequality gives convergence in probability for each $0<\epsilon \le 1$; for a larger threshold, bound its tail event by the tail event at threshold $1$. Conversely, convergence in probability and the second inequality give ${\mathrm{lim\hspace{0.17em}sup}}_n{d}_0([X_n],[X])\le \epsilon$ for every $0<\epsilon \le 1$, hence $d_0([X_n],[X])\to 0$.

Intersect the countably many full-measure events on which $\mathrm{\mid }{X}_n\mathrm{\mid }\le Y$ with the full-measure convergence event. Outside the resulting null set, $X_n\to X$ and $\mathrm{\mid }{X}_n\mathrm{\mid }\le Y$ for every $n$, so $\mathrm{\mid }X\mathrm{\mid }\le Y$ there. Since $Y\in L^p$, these bounds also show that every $X_n$ and $X$ belongs to $L^p$. Moreover, $\mathrm{\mid }{X}_n-X{\mathrm{\mid }}^p\to 0$ almost everywhere and $\mathrm{\mid }{X}_n-X{\mathrm{\mid }}^p\le (2\mathrm{\mid }Y\mathrm{\mid }{)}^p$.

Since $(2\mathrm{\mid }Y\mathrm{\mid }{)}^p$ is integrable, [L1] applied to step 1.1 yields $\mathbb{E}\mathrm{\mid }{X}_n-X{\mathrm{\mid }}^p\to 0$. By [L2] and the established $L^p$ membership, this is $X_n\to X$ in $L^p$.

The underlying measure has total mass one, hence is finite; [L1] converts the hypothesis to convergence in measure.

Apply the finite-measure reverse implication of [L2] to $(X_n)$. [step 1.1, L2] It supplies $X\in L^1$ and $\mathbb{E}\mathrm{\mid }{X}_n-X\mathrm{\mid }\to 0$, namely $L^1$ convergence.

Given $\epsilon >0$, choose $N$ from [L2] so that the following tail estimate holds for $n\ge N$. [L2, algebra] $\mathbb{E}\mathrm{\mid }{X}_n-X\mathrm{\mid }<\epsilon \mathrm{/}4$ for $n\ge N$. For $M>0$, $\mathbb{E}[\mathrm{\mid }{X}_n\mathrm{\mid }{\mathbf{1}}_{\mathrm{\mid }{X}_n\mathrm{\mid }>M}]\le 2\mathbb{E}\mathrm{\mid }{X}_n-X\mathrm{\mid }+2\mathbb{E}[\mathrm{\mid }X\mathrm{\mid }{\mathbf{1}}_{\mathrm{\mid }X\mathrm{\mid }>M\mathrm{/}2}]$ for $n\ge N$, by splitting at $\mathrm{\mid }X\mathrm{\mid }>M\mathrm{/}2$.

Choose $M$ so the tail of $X$ in step 1.1 is below $\epsilon \mathrm{/}4$ and control the finite initial family separately. [step 1.1, L1, choose] The finitely many functions $X,X_0,\dots ,X_{N-1}$ each have tail below $\epsilon$. Then step 1.1 gives the same bound for all later $X_n$. By [L1] the whole family is uniformly integrable.

If $X_n\to X$ in $L^1$, [L1] makes the larger family uniformly integrable. [L1] Thus its subfamily $\{X_n:n\in \mathbb{N}\}$ is uniformly integrable.

Conversely, if $(X_n)$ is uniformly integrable, [L2] applies to the given probability convergence. [L2] It yields $X\in L^1$ and $X_n\to X$ in $L^1$.

For real $U_n,V_n$ on a common probability space, if $U_n\Rightarrow U$ and $\mathrm{\mid }{V}_n-U_n\mathrm{\mid }\to 0$ in probability, then $V_n\Rightarrow U$. Indeed, for every $\delta >0$, with $p_n=\mathbb{P}(\mathrm{\mid }{V}_n-U_n\mathrm{\mid }>\delta )$, $F_{U_n}(t-\delta )-p_n\le F_{V_n}(t)\le F_{U_n}(t+\delta )+p_n\mathrm{.}$ At a continuity point $t$ of $F_U$, take $\delta \downarrow 0$ through values for which both $t-\delta$ and $t+\delta$ are continuity points. These values exist because a CDF has at most countably many jumps (for each positive integer $k$, there are at most $k$ jumps larger than $1\mathrm{/}k$). First let $n\to \mathrm{\infty }$ for each such $\delta$, then let $\delta \downarrow 0$; [L1] and [L2] give the assertion.

The CDF definition [L1] gives both affine operations needed below. First, $X_n+a\Rightarrow X+a$ because $F_{X_n+a}(t)=F_{X_n}(t-a)$. It also gives $a{X}_n\Rightarrow aX$ for every constant $a$: for $a>0$ use $F_{a{X}_n}(t)=F_{X_n}(t\mathrm{/}a)$; for $a<0$, use $F_{a{X}_n}(t)=1-F_{X_n}((t\mathrm{/}a)-)$ and squeeze the left limit between $F_{X_n}(t\mathrm{/}a-\delta )$ and $F_{X_n}(t\mathrm{/}a)$, taking $\delta \downarrow 0$ through continuity points $t\mathrm{/}a-\delta$; and for $a=0$ the claim is immediate. At continuity points of the transformed limit CDF, the corresponding point of $F_X$ is a continuity point.

The sequence $(X_n)$ is bounded in probability: CDF convergence [L1] at two continuity points outside a sufficiently large interval makes ${\mathrm{lim\hspace{0.17em}sup}}_n\mathbb{P}(\mathrm{\mid }{X}_n\mathrm{\mid }>M)$ arbitrarily small. Therefore $\mathbb{P}(\mathrm{\mid }{X}_n(Y_n-c)\mathrm{\mid }>\epsilon )\le \mathbb{P}(\mathrm{\mid }{X}_n\mathrm{\mid }>M)+\mathbb{P}(\mathrm{\mid }{Y}_n-c\mathrm{\mid }>\epsilon \mathrm{/}M)$ shows $X_n(Y_n-c)\to 0$ in probability. If $c\mathrm{\ne }0$, on $\mathrm{\mid }{Y}_n-c\mathrm{\mid }<\mathrm{\mid }c\mathrm{\mid }\mathrm{/}2$, $\mid Q_n-X_n\mathrm{/}c\mid \le \frac{2}{\mathrm{\mid }c{\mathrm{\mid }}^2}\mathrm{\mid }{X}_n\mathrm{\mid }\mathrm{\mid }{Y}_n-c\mathrm{\mid },$ and the exceptional event $\{\mathrm{\mid }{Y}_n-c\mathrm{\mid }\ge \mathrm{\mid }c\mathrm{\mid }\mathrm{/}2\}$ contains $\{Y_n=0\}$ and has probability at most $\mathbb{P}(\mathrm{\mid }{Y}_n-c\mathrm{\mid }>\mathrm{\mid }c\mathrm{\mid }\mathrm{/}4)\to 0$; the same boundedness argument gives $Q_n-X_n\mathrm{/}c\to 0$ in probability.

Addition follows from step 1.1 with $U_n=X_n+c$ and $V_n=X_n+Y_n$: $U_n\Rightarrow X+c$ by step 1.2, while $\mathrm{\mid }{V}_n-U_n\mathrm{\mid }=\mathrm{\mid }{Y}_n-c\mathrm{\mid }\to 0$ in probability by [L2].

Apply step 1.1 to $U_n=c{X}_n$, $V_n=X_n{Y}_n$, using step 1.2 and step 1.3, to obtain $X_n{Y}_n\Rightarrow cX$. When $c\mathrm{\ne }0$, apply it again to $U_n=X_n\mathrm{/}c$, $V_n=Q_n$, to obtain $Q_n\Rightarrow X\mathrm{/}c$.

For $\epsilon >0$, the max-metric bad event equals $\{\mathrm{\mid }{X}_n-X\mathrm{\mid }>\epsilon \}\cup \{\mathrm{\mid }{Y}_n-Y\mathrm{\mid }>\epsilon \}\mathrm{.}$

The union bound and [L1] make the probability in step 1.1 tend to zero, which proves the claim.

Fix $\epsilon ,\eta >0$. Choose a compact cube $K$ with $\mathbb{P}(Z\mathrm{\notin }K)<\eta$ and a compact cube $K^{\mathrm{\prime }}$ containing every point within distance $1$ of $K$. Uniform continuity of $f$ on $K^{\mathrm{\prime }}$ gives $\delta \in (0,1)$ such that points of $K^{\mathrm{\prime }}$ within $\delta$ have $f$-images within $\epsilon$.

If $Z\in K$ and $\mathrm{\parallel }{Z}_n-Z\mathrm{\parallel }<\delta$, then $Z_n\in K^{\mathrm{\prime }}$ and $\mathrm{\parallel }f(Z_n)-f(Z)\mathrm{\parallel }<\epsilon$. Thus the image bad-event probability is at most $\eta +\mathbb{P}(\mathrm{\parallel }{Z}_n-Z\mathrm{\parallel }\ge \delta )$, which is at most $\eta +\mathbb{P}(\mathrm{\parallel }{Z}_n-Z\mathrm{\parallel }>\delta \mathrm{/}2)$. Its limsup is at most $\eta$ by [L1]. Letting $\eta \downarrow 0$ proves the claim.

Apply [L2] and the claim to $(x,y)\mapsto x+y$ and $(x,y)\mapsto xy$. For division, first restrict to $\mathrm{\mid }y\mathrm{\mid }\ge r$ and then let $r\downarrow 0$; the limiting denominator is nonzero almost surely, and the zero-denominator convention for the approximating pair is contained in the remaining event.