Kolmogorov convergence criterion

Kolmogorov maximal inequality: Let $X_1,\dots ,X_n$ be independent centered square-integrable real random variables, $n\ge 1$, and $S_k={\sum }_{j=1}^k{X}_j$. For every $\lambda >0$, $\mathbb{P}({\max }_{1\le k\le n}\mathrm{\mid }{S}_k\mathrm{\mid }\ge \lambda )\le \frac{\mathrm{Var}(S_n)}{{\lambda }^2}=\frac{{\sum }_{j=1}^n\mathrm{Var}(X_j)}{{\lambda }^2}\mathrm{.}$ Thus controlling the whole finite maximum costs no larger bound than controlling the final sum by Chebyshev.

Almost-sure convergence of a random series: For real random variables $(X_n{)}_{n\ge 1}$, the series ${\sum }_{n\ge 1}{X}_n$ converges almost surely if its partial sums $S_n$ converge to a finite real limit on an event of probability one, as in def-almost-sure-convergence-of-random-variables. With $S_0=0$ from def-partial-sums-and-sample-means, its convergence event is $C={\bigcap }_{r\ge 1}{\bigcup }_{N\ge 1}{\bigcap }_{j\ge i\ge N}\{\mathrm{\mid }{S}_j-S_i\mathrm{\mid }<1\mathrm{/}r\}\mathrm{.}$ This is exactly the real Cauchy condition, with the indexing of thm-series-cauchy-criterion shifted by one. Measurable arithmetic makes every event in this countable expression measurable. For any fixed $m$, the union over $N$ may be restricted to $N\ge m$; then each difference uses only $X_{m+1},X_{m+2},\dots$. Thus $C$ is in the tail sigma-algebra, without assuming independence. Under independence, cor-almost-sure-convergence-of-an-independent-series-is-a-zero-one-event gives $\mathbb{P}(C)\in \{0,1\}$. Set $S={\lim }_n{S}_n$ on $C$ and $S=0$ off $C$. The functions ${\mathbf{1}}_C{S}_n$ converge everywhere to $S$, so thm-sequential-suprema-infima-limsup-liminf-and-pointwise-limits-are-measurable and thm-arithmetic-and-lattice-operations-preserve-measurability make $S$ measurable. For Borel sets $B_n$, the event $\{X_n\in B_n\text{ infinitely often}\}={\bigcap }_m{\bigcup }_{n\ge m}\{X_n\in B_n\}$ is likewise tail measurable. Changing finitely many summands adds an eventually constant finite difference to $S_n$; divided by deterministic $c_n>0$ tending to infinity that difference tends to zero, so the normalized limsup is unchanged. The sign of the unnormalized limsup need not be unchanged: the all-zero sequence has limsup zero, while changing its first term to $1$ makes the limsup of partial sums equal to $1$.

A series converges iff for every $\epsilon >0$ there is $N$ with $\mathrm{\mid }{a}_{m+1}+\cdots +a_n\mathrm{\mid }<\epsilon$ for all $n>m\ge N$: Let $(a_k)$ be a sequence of reals, with partial sums $s_n={\sum }_{k<n}{a}_k$ (def-series). Then $\sum a_k$ converges if and only if $\text{for every real }\epsilon >0\text{ there is }N\in \mathbb{N}\text{ such that }\mid {\sum }_{k=m+1}^n{a}_k\mid <\epsilon \text{ for all }n>m\ge N\mathrm{.}$ The block ${\sum }_{k=m+1}^n{a}_k$ is the finite sum $a_{m+1}+\cdots +a_n$ of def-finite-sum, and it equals $s_{n+1}-s_{m+1}$. This is the Cauchy criterion transported from sequences to series. Its value is that it decides convergence without producing, or even naming, the sum.

Continuity from below for measures: Let $(E_n{)}_{n\in \mathbb{N}}$ be an increasing sequence of measurable sets for a measure $\mu$, so $E_n\subseteq E_{n+1}$. Then $\mu \left({\bigcup }_{n\in \mathbb{N}}{E}_n\right)={\sup }_{n\in \mathbb{N}}\mu (E_n)\mathrm{.}$ No finiteness hypothesis is required.

Continuity from above when one set has finite measure: Let $(E_n{)}_{n\in \mathbb{N}}$ be a decreasing sequence of measurable sets for a measure $\mu$. If $\mu (E_{n_0})<+\mathrm{\infty }$ for some $n_0$, then $\mu \left({\bigcap }_{n\in \mathbb{N}}{E}_n\right)={\inf }_{n\in \mathbb{N}}\mu (E_n)\mathrm{.}$

Variance and covariance identities for random variables: Let $X,Y$ be square-integrable real random variables on one probability space. Then $\mathrm{Var}(X)=\mathbb{E}[X^2]-\mathbb{E}[X{]}^2,$ $\mathrm{Cov}(X,Y)=\mathbb{E}[XY]-\mathbb{E}[X]\mathbb{E}[Y]\mathrm{.}$ Moreover, covariance is symmetric and bilinear on finite linear combinations. On finite full-power-set probability spaces these formulas reduce to the published finite identities.

Riesz-Fischer completeness of $L^p$ for $1\le p\le \mathrm{\infty }$: Let $(X,\mathcal{A},\mu )$ be a measure space and let $1\le p\le \mathrm{\infty }$. Then $L^p(\mu )$, with the norm of thm-the-l-p-norm-descends-to-the-quotient-and-makes-l-p-a-normed-space, is complete. Equivalently, the metric induced by that norm is a complete metric in the sense of def-complete-metric-space. Moreover, if a sequence in $L^p(\mu )$ converges in norm, then some subsequence admits measurable representatives converging almost everywhere in the sense of def-convergence-almost-everywhere-relative-to-a-measure.

$L^p$ convergence implies convergence in probability: Let $1\le p<\mathrm{\infty }$. If $X_n\to X$ in $L^p$, then $X_n\to X$ in probability.

Almost-sure convergence implies convergence in probability: If $X_n\to X$ almost surely, then $X_n\to X$ in probability.

Limits in probability are unique almost surely: If $X_n\to X$ and $X_n\to Y$ in probability, then $X=Y$ almost surely.