Strong law estimator of an integrable mean

Continuity and derivatives of positive-base real powers: For $a>0$, the function $x\mapsto a^x$ is continuous on $\mathbb{R}$ and $(a^x{)}^{\mathrm{\prime }}=a^x\log a\mathrm{.}$ For $\alpha \in \mathbb{R}$, the function $x\mapsto x^{\alpha }$ is continuous and differentiable on $(0,\mathrm{\infty })$, with $(x^{\alpha }{)}^{\mathrm{\prime }}=\alpha x^{\alpha -1}\mathrm{.}$

Probability laws correspond to distribution functions: Assume the Axiom of Countable Choice.

1. Let $X$ be a real random variable, let ${\mathbb{P}}_X$ be its law, and let $F_X(x)=\mathbb{P}(X\le x)$. Then $F_X$ is nondecreasing and right-continuous, satisfies ${\lim }_{x\to -\mathrm{\infty }}{F}_X(x)=0,{\lim }_{x\to +\mathrm{\infty }}{F}_X(x)=1,$ and obeys ${\mathbb{P}}_X((a,b])=F_X(b)-F_X(a)(a<b)\mathrm{.}$
2. Conversely, if $F:\mathbb{R}\to \mathbb{R}$ is nondecreasing and right-continuous with ${\lim }_{x\to -\mathrm{\infty }}F(x)=0,{\lim }_{x\to +\mathrm{\infty }}F(x)=1,$ then there is a unique Borel probability measure $\mu$ on $\mathbb{R}$ such that $\mu ((a,b])=F(b)-F(a)(a<b),$ equivalently $F(x)=\mu ((-\mathrm{\infty },x])(x\in \mathbb{R})\mathrm{.}$

The indefinite integral of a nonnegative measurable function is a measure: Let $f:X\to [0,+\mathrm{\infty }]$ be measurable and define ${\nu }_f(A):={\int }_{A}fd\mu (A\in \mathcal{A})\mathrm{.}$ Then ${\nu }_f$ is a measure on $(X,\mathcal{A})$.

A continuous function on $[a,b]$ is Riemann integrable, by Heine-Cantor and Riemann's criterion: Let $a<b$ be reals and let $f:[a,b]\to \mathbb{R}$ be continuous on $[a,b]$ (def-continuity-real). Then $f$ is bounded (def-bounded-set) and Riemann integrable on $[a,b]$ (def-darboux-integral).

The proof gives more than integrability: it gives a partition that works. For every real $\epsilon >0$ the uniform partition into $N$ parts already satisfies $U(f,P)-L(f,P)<\epsilon$, as soon as $N$ is large enough that $(b-a)\mathrm{/}N$ is below the $\delta$ that uniform continuity supplies for $\epsilon \mathrm{/}(2(b-a))$. Uniform continuity is exactly what makes one $\delta$ serve all $N$ subintervals at once, and it is the only place where the compactness of $[a,b]$ is used.

The second fundamental theorem: if $G$ is differentiable on $[a,b]$ with $G^{\mathrm{\prime }}=f$ and $f$ is integrable, then ${\int }_a^{b}f=G(b)-G(a)$: Let $a<b$ be reals, let $G:[a,b]\to \mathbb{R}$ be differentiable at every point of $[a,b]$ as a function on $[a,b]$ (def-derivative; at $a$ and $b$ this is the one-sided derivative), let $f:=G^{\mathrm{\prime }}$, and suppose $f$ is integrable on $[a,b]$ (def-darboux-integral). Then

${\int }_a^{b}f=G(b)-G(a)\mathrm{.}$

Both hypotheses are needed and neither is removable. A function may be differentiable everywhere with $G^{\mathrm{\prime }}$ not integrable — then the left-hand side does not exist (an everywhere differentiable function with unbounded derivative) — and an integrable $f$ need not be the derivative of anything (the sign function); both witnesses are on the companion page.

No continuity of $f$ is assumed, which is what makes this the working form: the theorem evaluates ${\int }_a^{b}f$ for every integrable derivative, not only for continuous integrands.

A bounded Riemann integrable function on a closed bounded interval is Lebesgue measurable and has the same integral: Assume the Axiom of Countable Choice. Let $a<b$ and let $f:[a,b]\to \mathbb{R}$ be bounded and Riemann integrable. Then $f$ is Lebesgue measurable on $[a,b]$ and is integrable there, and its Lebesgue integral equals its Riemann integral: ${\int }_{[a,b]}fd{\lambda }_1={\int }_a^{b}f(x)dx\mathrm{.}$

This is the point at which the completeness of Lebesgue measure is used essentially: the proof obtains a Borel function equal to $f$ almost everywhere, and measurability of $f$ itself is then a completeness statement.

Monotone convergence for the integral: Let $0\le f_1\le f_2\le \cdots$ be measurable and suppose $f_n(x)\uparrow f(x)$ for every $x$. Then $\int f_{n}d\mu \uparrow \int fd\mu \mathrm{.}$

Measures agreeing on a generating pi-system are equal under an increasing finite-measure exhaustion from that pi-system: Let $\mathcal{P}$ be a $\pi$-system on $X$ generating $\mathcal{A}$, and let $\mu ,\nu$ be measures on $(X,\mathcal{A})$ that agree on $\mathcal{P}$. Suppose there is an increasing sequence $(P_n)$ in $\mathcal{P}$ with

$X={\bigcup }_n{P}_n,\mu (P_n)=\nu (P_n)<+\mathrm{\infty }(n\in \mathbb{N})\mathrm{.}$

Then $\mu =\nu$ on $\mathcal{A}$.

Layer-cake formulas for random variables: Let $(\mathrm{\Omega },\mathcal{F},\mathbb{P})$ be a probability space.

1. If $X:\mathrm{\Omega }\to [0,+\mathrm{\infty }]$ is measurable, then $\mathbb{E}[X]={\int }_0^{\mathrm{\infty }}\mathbb{P}(X>t)dt,$ where the right-hand side may be $+\mathrm{\infty }$.
2. If $X$ is an integrable real random variable, then $\mathbb{E}[X]={\int }_0^{\mathrm{\infty }}\mathbb{P}(X>t)dt-{\int }_0^{\mathrm{\infty }}\mathbb{P}(X<-t)dt\mathrm{.}$

For 0 < p < infinity, the layer-cake formula computes the integral of |f|^p from the distribution function: Let $(X,\mathcal{A},\mu )$ be a measure space, let $f:X\to \mathbb{C}$ be measurable, and let $0<p<\mathrm{\infty }$. Then ${\int }_X\mathrm{\mid }f{\mathrm{\mid }}^{p}d\mu =p{\int }_0^{\mathrm{\infty }}{t}^{p-1}\mu (\{\mathrm{\mid }f\mathrm{\mid }>t\})dt=p{\int }_0^{\mathrm{\infty }}{t}^{p-1}{A}_f(t)dt,$ where either side may be $+\mathrm{\infty }$.

The recursion theorem: Let $(N,0,\sigma )$ be a Peano system (def-peano-system), in particular the natural numbers $\mathbb{N}$ (def-natural-numbers). For any set $A$, any element $a\in A$, and any function $f:A\to A$, there is a unique function $g:N\to A$ such that $g(0)=a$ and $g(\sigma (n))=f(g(n))$ for all $n\in N$.

The axiom of dependent choice: a relation in which every element is related to something admits an $\mathbb{N}$-indexed chain: Let $X$ be a set and let $R\subseteq X\times X$ be a binary relation on $X$. Call $R$ entire on $X$ when

$\text{for every }x\in X\text{ there is }y\in X\text{ with }xRy\mathrm{.}$

The Axiom of Dependent Choice, written $\mathrm{D}\mathrm{C}$, is the following statement.

For every nonempty set $X$, every relation $R$ entire on $X$, and every $a\in X$, there is a function $x:\mathbb{N}\to X$ (def-function, def-natural-numbers) with $x_0=a\text{and}{x}_{n}R{x}_{n+1}\text{ for every }n\in \mathbb{N}\mathrm{.}$

Here a sequence in $X$ means a function from $\mathbb{N}$ to $X$, not necessarily a real-valued sequence. As everywhere in this library $\mathbb{N}$ contains $0$, and the sequence is indexed from $0$; the term $x_0$ is the prescribed starting point $a$ and every later term is related to its predecessor.

What DC adds to what came before. def-choice-function and def-axiom-of-choice select one element from each member of a family that is fixed in advance, and def-countable-choice does the same for a family indexed by $\mathbb{N}$. In both, the family is given before any selection is made. DC is the principle needed when the $n$-th set to select from is not known until the first $n$ selections have been made: here the admissible values of $x_{n+1}$ are exactly the $R$-successors of $x_n$, so the family being chosen from is built along the choosing. That is precisely the situation ${\mathrm{A}\mathrm{C}}_{\omega }$ does not cover, and it is why a construction "pick $x_{n+1}$ depending on $x_n$, for every $n$ at once" is not licensed by countable choice.

The starting point may be dropped. The formally weaker statement obtained by deleting the clause $x_0=a$ — for every nonempty $X$ and every entire $R$ there is a sequence with $x_{n}R{x}_{n+1}$ for all $n$ — is an immediate consequence of the form above, since $X$ is nonempty and any of its elements may be taken as $a$. The reverse derivation is standard and is not needed anywhere in this library, so it is not carried out; every use below prescribes $x_0$.

$R$ need not be an order and the terms need not be distinct. What DC delivers is a sequence, that is a function $\mathbb{N}\to X$, not a chain in the order-theoretic sense (def-chain). The relation may be symmetric, and the sequence may repeat a value or be constant; all that is asserted is $x_{n}R{x}_{n+1}$ at every index.

Countably many independent copies of a prescribed law exist: Assume countable choice and dependent choice. Every probability measure $\nu$ on $(S,\mathrm{\Sigma })$ is the common law of a countable independent family of $S$-valued random elements.

Kolmogorov iid l1 strong law: For IID real $(X_n{)}_{n\ge 1}$ with $\mathbb{E}\mathrm{\mid }{X}_1\mathrm{\mid }<\mathrm{\infty }$, $S_n\mathrm{/}n\to \mu =\mathbb{E}{X}_1$ almost surely.