Weak Laws and Series of Independent Random Variables — Examples

IID finite-variance weak law: Let $(X_k{)}_{k\ge 1}$ be IID square-integrable real random variables, with $\mu =\mathbb{E}{X}_1$ and ${\sigma }^2=\mathrm{Var}(X_1)$. For $S_n={\sum }_{k=1}^n{X}_k$, $\mathbb{E}\mathrm{\mid }{S}_n\mathrm{/}n-\mu {\mathrm{\mid }}^2={\sigma }^2\mathrm{/}n,$ and $S_n\mathrm{/}n\to \mu$ in $L^2$ and in probability. Also $\mathbb{P}(\mathrm{\mid }{S}_n\mathrm{/}n-\mu \mathrm{\mid }\ge \epsilon )\le {\sigma }^2\mathrm{/}(n{\epsilon }^2)$ for $\epsilon >0$.

Bernoulli random variables and binomial random variables as sums of independent Bernoulli trials: For $p\in [0,1]$, a Bernoulli$(p)$ random variable takes the value $1$ with probability $p$ and $0$ with probability $1-p$. For $n\in \mathbb{N}$, a binomial$(n,p)$ random variable is a sum $X={\sum }_{i<n}{X}_i$ of $n$ mutually independent Bernoulli$(p)$ variables. When $n=0$, this is the constant zero random variable.

A Bernoulli$(p)$ variable has mean $p$ and variance $p(1-p)$; a binomial$(n,p)$ variable has mean $np$ and variance $np(1-p)$: If $X$ is Bernoulli$(p)$, then $\mathbb{E}[X]=p$ and $\mathrm{Var}(X)=p(1-p)$. If $S$ is binomial$(n,p)$, then $\mathbb{E}[S]=np,\mathrm{Var}(S)=np(1-p)\mathrm{.}$ These formulas include $p=0$, $p=1$, and $n=0$.

Countably many independent copies of a prescribed law exist: Assume countable choice and dependent choice. Every probability measure $\nu$ on $(S,\mathrm{\Sigma })$ is the common law of a countable independent family of $S$-valued random elements.

Chebyshev weak law for uncorrelated arrays: For each $n\ge 1$, let $X_{n,1},\dots ,X_{n,r_n}$ be square-integrable real random variables on one probability space, pairwise uncorrelated within the row, where $r_n\ge 0$ is finite. Set $S_n={\sum }_{k=1}^{r_n}{X}_{n,k}$ and let $b_n>0$ be deterministic. If $v_n:=b_n^{-2}{\sum }_{k=1}^{r_n}\mathrm{Var}(X_{n,k})⟶0,$ then $(S_n-\mathbb{E}{S}_n)\mathrm{/}{b}_n\to 0$ in $L^2$ and in probability. More precisely, its second moment is $v_n$, and its probability of absolute value at least $\epsilon >0$ is at most $v_n\mathrm{/}{\epsilon }^2$. No independence between rows is required.

Coordinate random elements of a countable product are independent: Under the measure of thm-countable-product-of-probability-spaces, the coordinate maps $X_n(x)=x_n$ have laws ${\mu }_n$ and are independent.

Assuming countable and dependent choice, countable products of arbitrary probability spaces: Assume countable choice and dependent choice. For probability spaces $(E_n,{\mathcal{E}}_n,{\mu }_n{)}_{n\in \mathbb{N}}$ there is a unique probability measure $\mu$ on ${\mathcal{C}}_{\mathbb{N}}$ such that, for every finite $F$, its $F$-coordinate marginal is ${⨂}_{n\in F}{\mu }_n$.

A Bernoulli$(p)$ variable has mean $p$ and variance $p(1-p)$; a binomial$(n,p)$ variable has mean $np$ and variance $np(1-p)$: If $X$ is Bernoulli$(p)$, then $\mathbb{E}[X]=p$ and $\mathrm{Var}(X)=p(1-p)$. If $S$ is binomial$(n,p)$, then $\mathbb{E}[S]=np,\mathrm{Var}(S)=np(1-p)\mathrm{.}$ These formulas include $p=0$, $p=1$, and $n=0$.

For $n\ge 1$, if $X_0,\dots ,X_{n-1}$ are independent real random variables and $g_i:\mathbb{R}\to \mathbb{R}$ are Borel measurable with each $g_i(X_i)$ integrable, then ${\prod }_{i<n}{g}_i(X_i)$ is integrable and $\mathbb{E}[{\prod }_{i<n}{g}_i(X_i)]={\prod }_{i<n}\mathbb{E}[g_i(X_i)]$ in $\mathbb{R}$ (Expectations factor over finite products of independent random variables).

Kolmogorov three-series theorem: Let $(X_n{)}_{n\ge 1}$ be independent real random variables and fix $A>0$. Put $Y_n=X_n{\mathbf{1}}_{\{\mathrm{\mid }{X}_n\mathrm{\mid }\le A\}}$. Then ${\sum }_n{X}_n$ converges almost surely if and only if all three conditions hold: ${\sum }_n\mathbb{P}(\mathrm{\mid }{X}_n\mathrm{\mid }>A)<\mathrm{\infty },{\sum }_n\mathbb{E}{Y}_n\text{ converges in }\mathbb{R},{\sum }_n\mathrm{Var}(Y_n)<\mathrm{\infty }\mathrm{.}$ The conditions hold for some $A>0$ if and only if they hold for every $A>0$. No moment assumption is imposed on the untruncated variables.

Almost-sure convergence of an independent series is a zero-one event: Let $(X_n{)}_{n\in \mathbb{N}}$ be an independent sequence of real random variables. Then the event $\left\{{\sum }_{n=0}^{\mathrm{\infty }}{X}_n\text{ converges}\right\}$ has probability $0$ or $1$.

Countably many independent copies of a prescribed law exist: Assume countable choice and dependent choice. Every probability measure $\nu$ on $(S,\mathrm{\Sigma })$ is the common law of a countable independent family of $S$-valued random elements.

The p-series for a real exponent p converges exactly when p is greater than one: For every real $p$, ${\sum }_{k\ge 1}\frac{1}{k^p}\text{ converges}⟺p>1.$

Kolmogorov three-series theorem: Let $(X_n{)}_{n\ge 1}$ be independent real random variables and fix $A>0$. Put $Y_n=X_n{\mathbf{1}}_{\{\mathrm{\mid }{X}_n\mathrm{\mid }\le A\}}$. Then ${\sum }_n{X}_n$ converges almost surely if and only if all three conditions hold: ${\sum }_n\mathbb{P}(\mathrm{\mid }{X}_n\mathrm{\mid }>A)<\mathrm{\infty },{\sum }_n\mathbb{E}{Y}_n\text{ converges in }\mathbb{R},{\sum }_n\mathrm{Var}(Y_n)<\mathrm{\infty }\mathrm{.}$ The conditions hold for some $A>0$ if and only if they hold for every $A>0$. No moment assumption is imposed on the untruncated variables.

Countably many independent copies of a prescribed law exist: Assume countable choice and dependent choice. Every probability measure $\nu$ on $(S,\mathrm{\Sigma })$ is the common law of a countable independent family of $S$-valued random elements.

The p-series for a real exponent p converges exactly when p is greater than one: For every real $p$, ${\sum }_{k\ge 1}\frac{1}{k^p}\text{ converges}⟺p>1.$

The alternating series test: if $(b_k)$ is nonincreasing with $b_k\to 0$ then ${\sum }_k(-1{)}^k{b}_k$ converges, the sum lies between any two consecutive partial sums, and the error after $n$ terms is at most $b_n$: Let $({\epsilon }_k)$ be the alternating sequence of lem-alternating-sequence, that is the unique sequence of reals with ${\epsilon }_0=1$ and ${\epsilon }_{k+1}=-{\epsilon }_k$, which is what is usually written ${\epsilon }_k=(-1{)}^k$; let $e$ and $o$ be its even and odd index maps, so that ${\epsilon }_{e_j}=1$, ${\epsilon }_{o_j}=-1$, and every natural number is $e_j$ for exactly one $j$ or $o_j$ for exactly one $j$. Let $(b_k)$ be a sequence of reals that is nonincreasing (def-monotone-sequence) and converges to $0$ (def-real-limit); then $b_k\ge 0$ for every $k$. Write $t_n:={\sum }_{k<n}{\epsilon }_k{b}_k$ for the partial sums (def-series). Then: 1. the series $\sum {\epsilon }_k{b}_k$ converges; write $L$ for its sum; 2. $t_{e_j}\le L\le t_{o_j}$ for every $j\in \mathbb{N}$, and for every $n\in \mathbb{N}$ the sum $L$ lies between the two consecutive partial sums $t_n$ and $t_{n+1}$; 3. $\mathrm{\mid }L-t_n\mathrm{\mid }\le b_n$ for every $n\in \mathbb{N}$. Claim 3 is the error bound: the partial sum $t_n$, which uses the $n$ terms ${\epsilon }_0{b}_0,\dots ,{\epsilon }_{n-1}{b}_{n-1}$, differs from the sum by at most the first term omitted. Only claim 1 is a corollary of thm-dirichlet-test. Claims 2 and 3 are not: they come from the interlacing of the even-index and odd-index partial sums, and that argument is carried out below rather than smuggled into the Dirichlet estimate, which produces no bracketing at all.

Kolmogorov three-series theorem: Let $(X_n{)}_{n\ge 1}$ be independent real random variables and fix $A>0$. Put $Y_n=X_n{\mathbf{1}}_{\{\mathrm{\mid }{X}_n\mathrm{\mid }\le A\}}$. Then ${\sum }_n{X}_n$ converges almost surely if and only if all three conditions hold: ${\sum }_n\mathbb{P}(\mathrm{\mid }{X}_n\mathrm{\mid }>A)<\mathrm{\infty },{\sum }_n\mathbb{E}{Y}_n\text{ converges in }\mathbb{R},{\sum }_n\mathrm{Var}(Y_n)<\mathrm{\infty }\mathrm{.}$ The conditions hold for some $A>0$ if and only if they hold for every $A>0$. No moment assumption is imposed on the untruncated variables.

Second Borel-Cantelli lemma under pairwise independence: Let $(A_n{)}_{n\in \mathbb{N}}$ be pairwise independent events with ${\sum }_{n=0}^{\mathrm{\infty }}\mathbb{P}(A_n)=+\mathrm{\infty }\mathrm{.}$ Then $\mathbb{P}(A_n\mathrm{i}\mathrm{.}\mathrm{o}\mathrm{.})=1.$

The p-series for a real exponent p converges exactly when p is greater than one: For every real $p$, ${\sum }_{k\ge 1}\frac{1}{k^p}\text{ converges}⟺p>1.$

Coordinate random elements of a countable product are independent: Under the measure of thm-countable-product-of-probability-spaces, the coordinate maps $X_n(x)=x_n$ have laws ${\mu }_n$ and are independent.

Assuming countable and dependent choice, countable products of arbitrary probability spaces: Assume countable choice and dependent choice. For probability spaces $(E_n,{\mathcal{E}}_n,{\mu }_n{)}_{n\in \mathbb{N}}$ there is a unique probability measure $\mu$ on ${\mathcal{C}}_{\mathbb{N}}$ such that, for every finite $F$, its $F$-coordinate marginal is ${⨂}_{n\in F}{\mu }_n$.

Chebyshev weak law for uncorrelated arrays: For each $n\ge 1$, let $X_{n,1},\dots ,X_{n,r_n}$ be square-integrable real random variables on one probability space, pairwise uncorrelated within the row, where $r_n\ge 0$ is finite. Set $S_n={\sum }_{k=1}^{r_n}{X}_{n,k}$ and let $b_n>0$ be deterministic. If $v_n:=b_n^{-2}{\sum }_{k=1}^{r_n}\mathrm{Var}(X_{n,k})⟶0,$ then $(S_n-\mathbb{E}{S}_n)\mathrm{/}{b}_n\to 0$ in $L^2$ and in probability. More precisely, its second moment is $v_n$, and its probability of absolute value at least $\epsilon >0$ is at most $v_n\mathrm{/}{\epsilon }^2$. No independence between rows is required.

Convergence in probability: For real random variables $(X_n)$ and $X$ on one probability space, write $X_n\to X$ in probability when, for every $\epsilon >0$, $\mathbb{P}(\mathrm{\mid }{X}_n-X\mathrm{\mid }>\epsilon )⟶0.$ This is precisely def-convergence-in-measure for the probability measure.

Countably many independent copies of a prescribed law exist: Assume countable choice and dependent choice. Every probability measure $\nu$ on $(S,\mathrm{\Sigma })$ is the common law of a countable independent family of $S$-valued random elements.

First Borel-Cantelli lemma for events: Let $(A_n{)}_{n\in \mathbb{N}}$ be events in a probability space. If ${\sum }_{n=0}^{\mathrm{\infty }}\mathbb{P}(A_n)<+\mathrm{\infty },$ then $\mathbb{P}(A_n\mathrm{i}\mathrm{.}\mathrm{o}\mathrm{.})=0.$ No independence hypothesis is needed.

Coordinate random elements of a countable product are independent: Under the measure of thm-countable-product-of-probability-spaces, the coordinate maps $X_n(x)=x_n$ have laws ${\mu }_n$ and are independent.

Assuming countable and dependent choice, countable products of arbitrary probability spaces: Assume countable choice and dependent choice. For probability spaces $(E_n,{\mathcal{E}}_n,{\mu }_n{)}_{n\in \mathbb{N}}$ there is a unique probability measure $\mu$ on ${\mathcal{C}}_{\mathbb{N}}$ such that, for every finite $F$, its $F$-coordinate marginal is ${⨂}_{n\in F}{\mu }_n$.

The p-series for a real exponent p converges exactly when p is greater than one: For every real $p$, ${\sum }_{k\ge 1}\frac{1}{k^p}\text{ converges}⟺p>1.$

Almost-sure convergence of a random series: For real random variables $(X_n{)}_{n\ge 1}$, the series ${\sum }_{n\ge 1}{X}_n$ converges almost surely if its partial sums $S_n$ converge to a finite real limit on an event of probability one, as in def-almost-sure-convergence-of-random-variables. With $S_0=0$ from def-partial-sums-and-sample-means, its convergence event is $C={\bigcap }_{r\ge 1}{\bigcup }_{N\ge 1}{\bigcap }_{j\ge i\ge N}\{\mathrm{\mid }{S}_j-S_i\mathrm{\mid }<1\mathrm{/}r\}\mathrm{.}$ This is exactly the real Cauchy condition, with the indexing of thm-series-cauchy-criterion shifted by one. Measurable arithmetic makes every event in this countable expression measurable. For any fixed $m$, the union over $N$ may be restricted to $N\ge m$; then each difference uses only $X_{m+1},X_{m+2},\dots$. Thus $C$ is in the tail sigma-algebra, without assuming independence. Under independence, cor-almost-sure-convergence-of-an-independent-series-is-a-zero-one-event gives $\mathbb{P}(C)\in \{0,1\}$. Set $S={\lim }_n{S}_n$ on $C$ and $S=0$ off $C$. The functions ${\mathbf{1}}_C{S}_n$ converge everywhere to $S$, so thm-sequential-suprema-infima-limsup-liminf-and-pointwise-limits-are-measurable and thm-arithmetic-and-lattice-operations-preserve-measurability make $S$ measurable. For Borel sets $B_n$, the event $\{X_n\in B_n\text{ infinitely often}\}={\bigcap }_m{\bigcup }_{n\ge m}\{X_n\in B_n\}$ is likewise tail measurable. Changing finitely many summands adds an eventually constant finite difference to $S_n$; divided by deterministic $c_n>0$ tending to infinity that difference tends to zero, so the normalized limsup is unchanged. The sign of the unnormalized limsup need not be unchanged: the all-zero sequence has limsup zero, while changing its first term to $1$ makes the limsup of partial sums equal to $1$.

Exact tail criterion for a truncated-centered IID weak law: For IID real random variables $(X_n{)}_{n\ge 1}$ and $S_n={\sum }_{k=1}^n{X}_k$, there exist deterministic real constants $({\mu }_n)$ with $S_n\mathrm{/}n-{\mu }_n\to 0$ in probability if and only if $n\mathbb{P}(\mathrm{\mid }{X}_1\mathrm{\mid }>n)⟶0.$ When this condition holds, ${\mu }_n=\mathbb{E}[X_1{\mathbf{1}}_{\{\mathrm{\mid }{X}_1\mathrm{\mid }\le n\}}]$ works. Neither existence of an untruncated mean nor convergence of $({\mu }_n)$ is asserted.

Countably many independent copies of a prescribed law exist: Assume countable choice and dependent choice. Every probability measure $\nu$ on $(S,\mathrm{\Sigma })$ is the common law of a countable independent family of $S$-valued random elements.

Law or distribution of a random element: Let $X:(\mathrm{\Omega },\mathcal{F},\mathbb{P})\to (S,\mathrm{\Sigma })$ be a random element. Its law or distribution is the set function ${\mathbb{P}}_X:\mathrm{\Sigma }\to [0,+\mathrm{\infty }],{\mathbb{P}}_X(B):=\mathbb{P}(X^{-1}(B))\mathrm{.}$ Thus the law of $X$ records the probability of each measurable target set by pulling it back to an event in the original probability space.

Probability laws correspond to distribution functions: Assume the Axiom of Countable Choice. 1. Let $X$ be a real random variable, let ${\mathbb{P}}_X$ be its law, and let $F_X(x)=\mathbb{P}(X\le x)$. Then $F_X$ is nondecreasing and right-continuous, satisfies ${\lim }_{x\to -\mathrm{\infty }}{F}_X(x)=0,{\lim }_{x\to +\mathrm{\infty }}{F}_X(x)=1,$ and obeys ${\mathbb{P}}_X((a,b])=F_X(b)-F_X(a)(a<b)\mathrm{.}$ 2. Conversely, if $F:\mathbb{R}\to \mathbb{R}$ is nondecreasing and right-continuous with ${\lim }_{x\to -\mathrm{\infty }}F(x)=0,{\lim }_{x\to +\mathrm{\infty }}F(x)=1,$ then there is a unique Borel probability measure $\mu$ on $\mathbb{R}$ such that $\mu ((a,b])=F(b)-F(a)(a<b),$ equivalently $F(x)=\mu ((-\mathrm{\infty },x])(x\in \mathbb{R})\mathrm{.}$

Exact tail criterion for a truncated-centered IID weak law: For IID real random variables $(X_n{)}_{n\ge 1}$ and $S_n={\sum }_{k=1}^n{X}_k$, there exist deterministic real constants $({\mu }_n)$ with $S_n\mathrm{/}n-{\mu }_n\to 0$ in probability if and only if $n\mathbb{P}(\mathrm{\mid }{X}_1\mathrm{\mid }>n)⟶0.$ When this condition holds, ${\mu }_n=\mathbb{E}[X_1{\mathbf{1}}_{\{\mathrm{\mid }{X}_1\mathrm{\mid }\le n\}}]$ works. Neither existence of an untruncated mean nor convergence of $({\mu }_n)$ is asserted.

Countably many independent copies of a prescribed law exist: Assume countable choice and dependent choice. Every probability measure $\nu$ on $(S,\mathrm{\Sigma })$ is the common law of a countable independent family of $S$-valued random elements.

Layer-cake formulas for random variables: Let $(\mathrm{\Omega },\mathcal{F},\mathbb{P})$ be a probability space. 1. If $X:\mathrm{\Omega }\to [0,+\mathrm{\infty }]$ is measurable, then $\mathbb{E}[X]={\int }_0^{\mathrm{\infty }}\mathbb{P}(X>t)dt,$ where the right-hand side may be $+\mathrm{\infty }$. 2. If $X$ is an integrable real random variable, then $\mathbb{E}[X]={\int }_0^{\mathrm{\infty }}\mathbb{P}(X>t)dt-{\int }_0^{\mathrm{\infty }}\mathbb{P}(X<-t)dt\mathrm{.}$

Probability laws correspond to distribution functions: Assume the Axiom of Countable Choice. 1. Let $X$ be a real random variable, let ${\mathbb{P}}_X$ be its law, and let $F_X(x)=\mathbb{P}(X\le x)$. Then $F_X$ is nondecreasing and right-continuous, satisfies ${\lim }_{x\to -\mathrm{\infty }}{F}_X(x)=0,{\lim }_{x\to +\mathrm{\infty }}{F}_X(x)=1,$ and obeys ${\mathbb{P}}_X((a,b])=F_X(b)-F_X(a)(a<b)\mathrm{.}$ 2. Conversely, if $F:\mathbb{R}\to \mathbb{R}$ is nondecreasing and right-continuous with ${\lim }_{x\to -\mathrm{\infty }}F(x)=0,{\lim }_{x\to +\mathrm{\infty }}F(x)=1,$ then there is a unique Borel probability measure $\mu$ on $\mathbb{R}$ such that $\mu ((a,b])=F(b)-F(a)(a<b),$ equivalently $F(x)=\mu ((-\mathrm{\infty },x])(x\in \mathbb{R})\mathrm{.}$