Weak Laws and Series of Independent Random Variables

Variance and covariance identities for random variables: Let $X,Y$ be square-integrable real random variables on one probability space. Then $\mathrm{Var}(X)=\mathbb{E}[X^2]-\mathbb{E}[X{]}^2,$ $\mathrm{Cov}(X,Y)=\mathbb{E}[XY]-\mathbb{E}[X]\mathbb{E}[Y]\mathrm{.}$ Moreover, covariance is symmetric and bilinear on finite linear combinations. On finite full-power-set probability spaces these formulas reduce to the published finite identities.

Chebyshev's inequality for random variables: If $X$ is a square-integrable real random variable and $a>0$, then $\mathbb{P}(\mathrm{\mid }X-\mathbb{E}[X]\mathrm{\mid }\ge a)\le \frac{\mathrm{Var}(X)}{a^2}\mathrm{.}$

Convergence in probability: For real random variables $(X_n)$ and $X$ on one probability space, write $X_n\to X$ in probability when, for every $\epsilon >0$, $\mathbb{P}(\mathrm{\mid }{X}_n-X\mathrm{\mid }>\epsilon )⟶0.$ This is precisely def-convergence-in-measure for the probability measure.

$L^p$ convergence for random variables: Let $1\le p\le \mathrm{\infty }$. For real random variables whose classes lie in $L^p(\mathbb{P})$ as defined by def-l-p-space-as-a-quotient-by-null-functions, write $X_n\to X$ in $L^p$ when $\mathrm{\parallel }[X_n]-[X]{\mathrm{\parallel }}_{L^p(\mathbb{P})}⟶0.$ For $p<\mathrm{\infty }$, this norm is $\mathrm{\parallel }[X_n]-[X]{\mathrm{\parallel }}_{L^p(\mathbb{P})}=(\mathbb{E}\mathrm{\mid }{X}_n-X{\mathrm{\mid }}^p{)}^{1\mathrm{/}p};$ for $p=\mathrm{\infty }$, it is the essential-supremum norm. Thus the assertion concerns almost-everywhere equivalence classes, not chosen representatives.

Linearity, monotonicity, and the modulus bound for expectation: Let $X,Y$ be integrable real or complex random variables on one probability space. 1. For scalars $a,b$, $\mathbb{E}[aX+bY]=a\mathbb{E}[X]+b\mathbb{E}[Y]\mathrm{.}$ 2. If $X$ and $Y$ are real-valued and $X\le Y$ almost surely, then $\mathbb{E}[X]\le \mathbb{E}[Y]\mathrm{.}$ 3. $\mathrm{\mid }\mathbb{E}[X]\mathrm{\mid }\le \mathbb{E}[\mathrm{\mid }X\mathrm{\mid }]\mathrm{.}$

Identical distribution and IID families: Let $(X_i{)}_{i\in I}$ be random elements with the same measurable target $(E,\mathcal{E})$. They are identically distributed if $\mathbb{P}(X_i\in B)=\mathbb{P}(X_j\in B)$ for all $i,j\in I$ and $B\in \mathcal{E}$, that is, their laws in def-law-or-distribution-of-a-random-element agree. They are independent and identically distributed (IID) if, in addition, the whole family is independent in def-independent-random-elements. Independence means mutual independence, not merely pairwise independence. No moment assumption is part of either definition. The empty family satisfies these universal conditions vacuously.

Chebyshev weak law for uncorrelated arrays: For each $n\ge 1$, let $X_{n,1},\dots ,X_{n,r_n}$ be square-integrable real random variables on one probability space, pairwise uncorrelated within the row, where $r_n\ge 0$ is finite. Set $S_n={\sum }_{k=1}^{r_n}{X}_{n,k}$ and let $b_n>0$ be deterministic. If $v_n:=b_n^{-2}{\sum }_{k=1}^{r_n}\mathrm{Var}(X_{n,k})⟶0,$ then $(S_n-\mathbb{E}{S}_n)\mathrm{/}{b}_n\to 0$ in $L^2$ and in probability. More precisely, its second moment is $v_n$, and its probability of absolute value at least $\epsilon >0$ is at most $v_n\mathrm{/}{\epsilon }^2$. No independence between rows is required.

Expectations factor over finite products of independent random variables: Let $n\ge 1$, let $X_0,\dots ,X_{n-1}$ be independent real random variables on a common probability space, and let $g_i:\mathbb{R}\to \mathbb{R}$ be Borel measurable for each $i<n$. 1. If every $g_i$ is nonnegative, then $\mathbb{E}[{\prod }_{i<n}{g}_i(X_i)]={\prod }_{i<n}\mathbb{E}[g_i(X_i)]$ in $[0,+\mathrm{\infty }]$. 2. If every $g_i(X_i)$ is integrable, then ${\prod }_{i<n}{g}_i(X_i)$ is integrable and the same factorization holds in $\mathbb{R}$.

Zero truncation at a positive level: For a real random variable $X$ and a deterministic level $A>0$, its zero truncation is $X^{(A)}=X{\mathbf{1}}_{\{\mathrm{\mid }X\mathrm{\mid }\le A\}}\mathrm{.}$ The threshold event is measurable because $X$ is measurable and $[-A,A]$ is Borel; its indicator and the product are measurable by thm-arithmetic-and-lattice-operations-preserve-measurability. Thus $X^{(A)}$ is a real random variable as in def-random-element-and-real-random-variable. It equals $X$ at both cutoff endpoints and is zero outside the interval. Since $\mathrm{\mid }{X}^{(A)}\mathrm{\mid }\le A$, for every $0<p<\mathrm{\infty }$ its absolute $p$th moment is at most $A^p\mathbb{P}(\mathrm{\Omega })=A^p$. This is not clipping to the endpoints.

IID finite-variance weak law: Let $(X_k{)}_{k\ge 1}$ be IID square-integrable real random variables, with $\mu =\mathbb{E}{X}_1$ and ${\sigma }^2=\mathrm{Var}(X_1)$. For $S_n={\sum }_{k=1}^n{X}_k$, $\mathbb{E}\mathrm{\mid }{S}_n\mathrm{/}n-\mu {\mathrm{\mid }}^2={\sigma }^2\mathrm{/}n,$ and $S_n\mathrm{/}n\to \mu$ in $L^2$ and in probability. Also $\mathbb{P}(\mathrm{\mid }{S}_n\mathrm{/}n-\mu \mathrm{\mid }\ge \epsilon )\le {\sigma }^2\mathrm{/}(n{\epsilon }^2)$ for $\epsilon >0$.

Measurable coordinatewise functions preserve independence: Let $(X_i{)}_{i\in I}$ be an independent family of random elements $X_i:(\mathrm{\Omega },\mathcal{F},\mathbb{P})\to (S_i,{\mathrm{\Sigma }}_i)$. For each $i$, let $g_i:(S_i,{\mathrm{\Sigma }}_i)\to (T_i,{\mathcal{T}}_i)$ be measurable. Then the family $(g_i\circ X_i{)}_{i\in I}$ is independent.

Dominated convergence: Let $f$ and $(f_n)$ be measurable complex-valued functions such that $f_n\to f$ almost everywhere and $\mathrm{\mid }{f}_n\mathrm{\mid }\le g$ almost everywhere for a single nonnegative measurable function $g$ with $\int gd\mu <+\mathrm{\infty }$. Then $f\in L^1(\mu )$, $\int \mathrm{\mid }{f}_n-f\mathrm{\mid }d\mu ⟶0,$ and hence $\int f_{n}d\mu ⟶\int fd\mu \mathrm{.}$

Markov's inequality for random variables: If $X:\mathrm{\Omega }\to [0,+\mathrm{\infty }]$ is a nonnegative random variable on a probability space and $a>0$, then $\mathbb{P}(X\ge a)\le \frac{\mathbb{E}[X]}{a}\mathrm{.}$

Finite-measure $L^r$ includes into $L^p$ for $p<r$: Let $(X,\mathcal{A},\mu )$ be a measure space with $\mu (X)<\mathrm{\infty }$. 1. If $1\le p<r<\mathrm{\infty }$ and $f\in {\mathcal{L}}^r(\mu )$, then $f\in {\mathcal{L}}^p(\mu )$ and $\mathrm{\parallel }f{\mathrm{\parallel }}_p\le \mu (X{)}^{1\mathrm{/}p-1\mathrm{/}r}\mathrm{\parallel }f{\mathrm{\parallel }}_r\mathrm{.}$ 2. If $1\le p<\mathrm{\infty }$ and $f\in L^{\mathrm{\infty }}(\mu )$, then $f\in {\mathcal{L}}^p(\mu )$ and $\mathrm{\parallel }f{\mathrm{\parallel }}_p\le \mu (X{)}^{1\mathrm{/}p}\mathrm{\parallel }f{\mathrm{\parallel }}_{\mathrm{\infty }}\mathrm{.}$

Linearity, monotonicity, and the modulus bound for expectation: Let $X,Y$ be integrable real or complex random variables on one probability space. 1. For scalars $a,b$, $\mathbb{E}[aX+bY]=a\mathbb{E}[X]+b\mathbb{E}[Y]\mathrm{.}$ 2. If $X$ and $Y$ are real-valued and $X\le Y$ almost surely, then $\mathbb{E}[X]\le \mathbb{E}[Y]\mathrm{.}$ 3. $\mathrm{\mid }\mathbb{E}[X]\mathrm{\mid }\le \mathbb{E}[\mathrm{\mid }X\mathrm{\mid }]\mathrm{.}$

Disjoint groups of an independent sigma-algebra family remain independent: Let $({\mathcal{F}}_i{)}_{i\in I}$ be an independent family of sigma-algebras on a probability space, and let $J_0,\dots ,J_{m-1}\subseteq I$ be pairwise disjoint index sets. For each $r<m$, define ${\mathcal{G}}_r:=\sigma \left({\bigcup }_{i\in J_r}{\mathcal{F}}_i\right)\mathrm{.}$ Then the sigma-algebras ${\mathcal{G}}_0,\dots ,{\mathcal{G}}_{m-1}$ are independent.

Expectations factor over finite products of independent random variables: Let $n\ge 1$, let $X_0,\dots ,X_{n-1}$ be independent real random variables on a common probability space, and let $g_i:\mathbb{R}\to \mathbb{R}$ be Borel measurable for each $i<n$. 1. If every $g_i$ is nonnegative, then $\mathbb{E}[{\prod }_{i<n}{g}_i(X_i)]={\prod }_{i<n}\mathbb{E}[g_i(X_i)]$ in $[0,+\mathrm{\infty }]$. 2. If every $g_i(X_i)$ is integrable, then ${\prod }_{i<n}{g}_i(X_i)$ is integrable and the same factorization holds in $\mathbb{R}$.

Variance and covariance identities for random variables: Let $X,Y$ be square-integrable real random variables on one probability space. Then $\mathrm{Var}(X)=\mathbb{E}[X^2]-\mathbb{E}[X{]}^2,$ $\mathrm{Cov}(X,Y)=\mathbb{E}[XY]-\mathbb{E}[X]\mathbb{E}[Y]\mathrm{.}$ Moreover, covariance is symmetric and bilinear on finite linear combinations. On finite full-power-set probability spaces these formulas reduce to the published finite identities.

Arithmetic and lattice operations preserve measurability whenever they are defined: Let $(X,\mathcal{A})$ be a measurable space and let $f,g:X\to \bar{\mathbb{R}}$ be measurable. Then: 1. $cf$ is measurable for every real scalar $c$; 2. $\max (f,g)$, $\min (f,g)$, $\mathrm{\mid }f\mathrm{\mid }$, $f^{+}$, and $f^{-}$ are measurable; 3. if $f+g$ is pointwise defined, then $f+g$ is measurable; 4. with the convention of rem-zero-times-infinity-convention-for-pointwise-products, the pointwise product $fg$ is measurable.

Kolmogorov maximal inequality: Let $X_1,\dots ,X_n$ be independent centered square-integrable real random variables, $n\ge 1$, and $S_k={\sum }_{j=1}^k{X}_j$. For every $\lambda >0$, $\mathbb{P}({\max }_{1\le k\le n}\mathrm{\mid }{S}_k\mathrm{\mid }\ge \lambda )\le \frac{\mathrm{Var}(S_n)}{{\lambda }^2}=\frac{{\sum }_{j=1}^n\mathrm{Var}(X_j)}{{\lambda }^2}\mathrm{.}$ Thus controlling the whole finite maximum costs no larger bound than controlling the final sum by Chebyshev.

Almost-sure convergence of a random series: For real random variables $(X_n{)}_{n\ge 1}$, the series ${\sum }_{n\ge 1}{X}_n$ converges almost surely if its partial sums $S_n$ converge to a finite real limit on an event of probability one, as in def-almost-sure-convergence-of-random-variables. With $S_0=0$ from def-partial-sums-and-sample-means, its convergence event is $C={\bigcap }_{r\ge 1}{\bigcup }_{N\ge 1}{\bigcap }_{j\ge i\ge N}\{\mathrm{\mid }{S}_j-S_i\mathrm{\mid }<1\mathrm{/}r\}\mathrm{.}$ This is exactly the real Cauchy condition, with the indexing of thm-series-cauchy-criterion shifted by one. Measurable arithmetic makes every event in this countable expression measurable. For any fixed $m$, the union over $N$ may be restricted to $N\ge m$; then each difference uses only $X_{m+1},X_{m+2},\dots$. Thus $C$ is in the tail sigma-algebra, without assuming independence. Under independence, cor-almost-sure-convergence-of-an-independent-series-is-a-zero-one-event gives $\mathbb{P}(C)\in \{0,1\}$. Set $S={\lim }_n{S}_n$ on $C$ and $S=0$ off $C$. The functions ${\mathbf{1}}_C{S}_n$ converge everywhere to $S$, so thm-sequential-suprema-infima-limsup-liminf-and-pointwise-limits-are-measurable and thm-arithmetic-and-lattice-operations-preserve-measurability make $S$ measurable. For Borel sets $B_n$, the event $\{X_n\in B_n\text{ infinitely often}\}={\bigcap }_m{\bigcup }_{n\ge m}\{X_n\in B_n\}$ is likewise tail measurable. Changing finitely many summands adds an eventually constant finite difference to $S_n$; divided by deterministic $c_n>0$ tending to infinity that difference tends to zero, so the normalized limsup is unchanged. The sign of the unnormalized limsup need not be unchanged: the all-zero sequence has limsup zero, while changing its first term to $1$ makes the limsup of partial sums equal to $1$.

A series converges iff for every $\epsilon >0$ there is $N$ with $\mathrm{\mid }{a}_{m+1}+\cdots +a_n\mathrm{\mid }<\epsilon$ for all $n>m\ge N$: Let $(a_k)$ be a sequence of reals, with partial sums $s_n={\sum }_{k<n}{a}_k$ (def-series). Then $\sum a_k$ converges if and only if $\text{for every real }\epsilon >0\text{ there is }N\in \mathbb{N}\text{ such that }\mid {\sum }_{k=m+1}^n{a}_k\mid <\epsilon \text{ for all }n>m\ge N\mathrm{.}$ The block ${\sum }_{k=m+1}^n{a}_k$ is the finite sum $a_{m+1}+\cdots +a_n$ of def-finite-sum, and it equals $s_{n+1}-s_{m+1}$. This is the Cauchy criterion transported from sequences to series. Its value is that it decides convergence without producing, or even naming, the sum.

Continuity from below for measures: Let $(E_n{)}_{n\in \mathbb{N}}$ be an increasing sequence of measurable sets for a measure $\mu$, so $E_n\subseteq E_{n+1}$. Then $\mu \left({\bigcup }_{n\in \mathbb{N}}{E}_n\right)={\sup }_{n\in \mathbb{N}}\mu (E_n)\mathrm{.}$ No finiteness hypothesis is required.

Continuity from above when one set has finite measure: Let $(E_n{)}_{n\in \mathbb{N}}$ be a decreasing sequence of measurable sets for a measure $\mu$. If $\mu (E_{n_0})<+\mathrm{\infty }$ for some $n_0$, then $\mu \left({\bigcap }_{n\in \mathbb{N}}{E}_n\right)={\inf }_{n\in \mathbb{N}}\mu (E_n)\mathrm{.}$

Variance and covariance identities for random variables: Let $X,Y$ be square-integrable real random variables on one probability space. Then $\mathrm{Var}(X)=\mathbb{E}[X^2]-\mathbb{E}[X{]}^2,$ $\mathrm{Cov}(X,Y)=\mathbb{E}[XY]-\mathbb{E}[X]\mathbb{E}[Y]\mathrm{.}$ Moreover, covariance is symmetric and bilinear on finite linear combinations. On finite full-power-set probability spaces these formulas reduce to the published finite identities.

Riesz-Fischer completeness of $L^p$ for $1\le p\le \mathrm{\infty }$: Let $(X,\mathcal{A},\mu )$ be a measure space and let $1\le p\le \mathrm{\infty }$. Then $L^p(\mu )$, with the norm of thm-the-l-p-norm-descends-to-the-quotient-and-makes-l-p-a-normed-space, is complete. Equivalently, the metric induced by that norm is a complete metric in the sense of def-complete-metric-space. Moreover, if a sequence in $L^p(\mu )$ converges in norm, then some subsequence admits measurable representatives converging almost everywhere in the sense of def-convergence-almost-everywhere-relative-to-a-measure.

$L^p$ convergence implies convergence in probability: Let $1\le p<\mathrm{\infty }$. If $X_n\to X$ in $L^p$, then $X_n\to X$ in probability.

Almost-sure convergence implies convergence in probability: If $X_n\to X$ almost surely, then $X_n\to X$ in probability.

Limits in probability are unique almost surely: If $X_n\to X$ and $X_n\to Y$ in probability, then $X=Y$ almost surely.

Kolmogorov convergence criterion: For independent centered square-integrable real random variables $(X_n{)}_{n\ge 1}$, if ${\sum }_{n\ge 1}\mathrm{Var}(X_n)<\mathrm{\infty }$, then ${\sum }_{n\ge 1}{X}_n$ converges almost surely and in $L^2$ to the same finite real random variable.

Measurable coordinatewise functions preserve independence: Let $(X_i{)}_{i\in I}$ be an independent family of random elements $X_i:(\mathrm{\Omega },\mathcal{F},\mathbb{P})\to (S_i,{\mathrm{\Sigma }}_i)$. For each $i$, let $g_i:(S_i,{\mathrm{\Sigma }}_i)\to (T_i,{\mathcal{T}}_i)$ be measurable. Then the family $(g_i\circ X_i{)}_{i\in I}$ is independent.

Variance and covariance identities for random variables: Let $X,Y$ be square-integrable real random variables on one probability space. Then $\mathrm{Var}(X)=\mathbb{E}[X^2]-\mathbb{E}[X{]}^2,$ $\mathrm{Cov}(X,Y)=\mathbb{E}[XY]-\mathbb{E}[X]\mathbb{E}[Y]\mathrm{.}$ Moreover, covariance is symmetric and bilinear on finite linear combinations. On finite full-power-set probability spaces these formulas reduce to the published finite identities.

Independent random elements: Let $X_i:(\mathrm{\Omega },\mathcal{F},\mathbb{P})\to (S_i,{\mathrm{\Sigma }}_i)$ be random elements on a common probability space. For each $i$, write $\sigma (X_i):=\{X_i^{-1}(B):B\in {\mathrm{\Sigma }}_i\}\subseteq \mathcal{F}\mathrm{.}$ The family $(X_i{)}_{i\in I}$ is independent when the sigma-algebras $(\sigma (X_i){)}_{i\in I}$ are independent in the sense of def-independent-sigma-algebras-and-events. When ${\mathrm{\Sigma }}_i$ is a Borel sigma-algebra, this agrees with the notation $\sigma (X_i)$ from def-sigma-algebra-generated-by-a-function.

Symmetric real random variables: A real random variable $X$ is symmetric if its law as defined in def-law-or-distribution-of-a-random-element equals the law of $-X$. Equivalently, $\mathbb{P}(X\in B)=\mathbb{P}(X\in -B)$ for every Borel $B\subseteq \mathbb{R}$, where $-B=\{-b:b\in B\}$. No existence of an expectation is assumed in this definition. In particular atoms, including an atom at zero, are allowed.

Almost-sure convergence of a random series: For real random variables $(X_n{)}_{n\ge 1}$, the series ${\sum }_{n\ge 1}{X}_n$ converges almost surely if its partial sums $S_n$ converge to a finite real limit on an event of probability one, as in def-almost-sure-convergence-of-random-variables. With $S_0=0$ from def-partial-sums-and-sample-means, its convergence event is $C={\bigcap }_{r\ge 1}{\bigcup }_{N\ge 1}{\bigcap }_{j\ge i\ge N}\{\mathrm{\mid }{S}_j-S_i\mathrm{\mid }<1\mathrm{/}r\}\mathrm{.}$ This is exactly the real Cauchy condition, with the indexing of thm-series-cauchy-criterion shifted by one. Measurable arithmetic makes every event in this countable expression measurable. For any fixed $m$, the union over $N$ may be restricted to $N\ge m$; then each difference uses only $X_{m+1},X_{m+2},\dots$. Thus $C$ is in the tail sigma-algebra, without assuming independence. Under independence, cor-almost-sure-convergence-of-an-independent-series-is-a-zero-one-event gives $\mathbb{P}(C)\in \{0,1\}$. Set $S={\lim }_n{S}_n$ on $C$ and $S=0$ off $C$. The functions ${\mathbf{1}}_C{S}_n$ converge everywhere to $S$, so thm-sequential-suprema-infima-limsup-liminf-and-pointwise-limits-are-measurable and thm-arithmetic-and-lattice-operations-preserve-measurability make $S$ measurable. For Borel sets $B_n$, the event $\{X_n\in B_n\text{ infinitely often}\}={\bigcap }_m{\bigcup }_{n\ge m}\{X_n\in B_n\}$ is likewise tail measurable. Changing finitely many summands adds an eventually constant finite difference to $S_n$; divided by deterministic $c_n>0$ tending to infinity that difference tends to zero, so the normalized limsup is unchanged. The sign of the unnormalized limsup need not be unchanged: the all-zero sequence has limsup zero, while changing its first term to $1$ makes the limsup of partial sums equal to $1$.

For sigma-finite factors, the product measure exists, has the rectangle formula, is sigma-finite, and is unique: Let $(X,\mathcal{A},\mu )$ and $(Y,\mathcal{B},\nu )$ be sigma-finite measure spaces. Then: 1. the set function $E\mapsto (\mu \times \nu )(E)$ of def-product-measure-on-sigma-finite-spaces is a measure on $\mathcal{A}\otimes \mathcal{B}$; 2. for measurable rectangles, $(\mu \times \nu )(A\times B)=\mu (A)\nu (B);$ 3. the measure $\mu \times \nu$ is sigma-finite; and 4. it is the unique measure on $\mathcal{A}\otimes \mathcal{B}$ with the rectangle formula.

Disjoint groups of an independent sigma-algebra family remain independent: Let $({\mathcal{F}}_i{)}_{i\in I}$ be an independent family of sigma-algebras on a probability space, and let $J_0,\dots ,J_{m-1}\subseteq I$ be pairwise disjoint index sets. For each $r<m$, define ${\mathcal{G}}_r:=\sigma \left({\bigcup }_{i\in J_r}{\mathcal{F}}_i\right)\mathrm{.}$ Then the sigma-algebras ${\mathcal{G}}_0,\dots ,{\mathcal{G}}_{m-1}$ are independent.

Measurable coordinatewise functions preserve independence: Let $(X_i{)}_{i\in I}$ be an independent family of random elements $X_i:(\mathrm{\Omega },\mathcal{F},\mathbb{P})\to (S_i,{\mathrm{\Sigma }}_i)$. For each $i$, let $g_i:(S_i,{\mathrm{\Sigma }}_i)\to (T_i,{\mathcal{T}}_i)$ be measurable. Then the family $(g_i\circ X_i{)}_{i\in I}$ is independent.

Expectations factor over finite products of independent random variables: Let $n\ge 1$, let $X_0,\dots ,X_{n-1}$ be independent real random variables on a common probability space, and let $g_i:\mathbb{R}\to \mathbb{R}$ be Borel measurable for each $i<n$. 1. If every $g_i$ is nonnegative, then $\mathbb{E}[{\prod }_{i<n}{g}_i(X_i)]={\prod }_{i<n}\mathbb{E}[g_i(X_i)]$ in $[0,+\mathrm{\infty }]$. 2. If every $g_i(X_i)$ is integrable, then ${\prod }_{i<n}{g}_i(X_i)$ is integrable and the same factorization holds in $\mathbb{R}$.

Variance and covariance identities for random variables: Let $X,Y$ be square-integrable real random variables on one probability space. Then $\mathrm{Var}(X)=\mathbb{E}[X^2]-\mathbb{E}[X{]}^2,$ $\mathrm{Cov}(X,Y)=\mathbb{E}[XY]-\mathbb{E}[X]\mathbb{E}[Y]\mathrm{.}$ Moreover, covariance is symmetric and bilinear on finite linear combinations. On finite full-power-set probability spaces these formulas reduce to the published finite identities.

Almost-sure convergence of a random series: For real random variables $(X_n{)}_{n\ge 1}$, the series ${\sum }_{n\ge 1}{X}_n$ converges almost surely if its partial sums $S_n$ converge to a finite real limit on an event of probability one, as in def-almost-sure-convergence-of-random-variables. With $S_0=0$ from def-partial-sums-and-sample-means, its convergence event is $C={\bigcap }_{r\ge 1}{\bigcup }_{N\ge 1}{\bigcap }_{j\ge i\ge N}\{\mathrm{\mid }{S}_j-S_i\mathrm{\mid }<1\mathrm{/}r\}\mathrm{.}$ This is exactly the real Cauchy condition, with the indexing of thm-series-cauchy-criterion shifted by one. Measurable arithmetic makes every event in this countable expression measurable. For any fixed $m$, the union over $N$ may be restricted to $N\ge m$; then each difference uses only $X_{m+1},X_{m+2},\dots$. Thus $C$ is in the tail sigma-algebra, without assuming independence. Under independence, cor-almost-sure-convergence-of-an-independent-series-is-a-zero-one-event gives $\mathbb{P}(C)\in \{0,1\}$. Set $S={\lim }_n{S}_n$ on $C$ and $S=0$ off $C$. The functions ${\mathbf{1}}_C{S}_n$ converge everywhere to $S$, so thm-sequential-suprema-infima-limsup-liminf-and-pointwise-limits-are-measurable and thm-arithmetic-and-lattice-operations-preserve-measurability make $S$ measurable. For Borel sets $B_n$, the event $\{X_n\in B_n\text{ infinitely often}\}={\bigcap }_m{\bigcup }_{n\ge m}\{X_n\in B_n\}$ is likewise tail measurable. Changing finitely many summands adds an eventually constant finite difference to $S_n$; divided by deterministic $c_n>0$ tending to infinity that difference tends to zero, so the normalized limsup is unchanged. The sign of the unnormalized limsup need not be unchanged: the all-zero sequence has limsup zero, while changing its first term to $1$ makes the limsup of partial sums equal to $1$.