Probability Spaces Random Variables and Expectation

Because $\mathbb{P}(\mathrm{\Omega })=1$ by [L1] and $A\subseteq \mathrm{\Omega }$, [L2] gives $1=\mathbb{P}(\mathrm{\Omega })=\mathbb{P}(A)+\mathbb{P}(A^c),$ so $\mathbb{P}(A^c)=1-\mathbb{P}(A)$. Monotonicity in [L2] also gives $A\subseteq B\Rightarrow \mathbb{P}(A)\le \mathbb{P}(B)$.

Since every probability is at most $1$, the finite-measure hypotheses in [L2] apply to $A$, $B$, and $A\cup B$. Thus if $A\subseteq B$, then $\mathbb{P}(B\setminus A)=\mathbb{P}(B)-\mathbb{P}(A),$ subadditivity gives the countable and finite union bounds, finite inclusion-exclusion gives $\mathbb{P}(A\cup B)=\mathbb{P}(A)+\mathbb{P}(B)-\mathbb{P}(A\cap B),$ and continuity from below and from above give the two monotone-limit formulas, because a decreasing probability sequence always has finite first term.

Steps 1.1 and 1.2 are exactly the stated probability identities.

If $(\mathrm{\Omega },w)$ is a finite probability space, then [L1] already states that every subset of $\mathrm{\Omega }$ is an event and that $A\mapsto {\sum }_{\omega \in A}w(\omega )$ is its probability. Therefore ${\mathbb{P}}_w$ is a probability measure on $(\mathrm{\Omega },\mathcal{P}(\mathrm{\Omega }))$ by [L2].

Conversely, let $\mathbb{P}$ be a probability measure on $(\mathrm{\Omega },\mathcal{P}(\mathrm{\Omega }))$ and put $w(\omega )=\mathbb{P}(\{\omega \})$. Each singleton is an atom of the full power-set sigma-algebra, and every $A\subseteq \mathrm{\Omega }$ is the union of the singletons it contains. Thus [L3] gives $\mathbb{P}(A)={\sum }_{\omega \in A}\mathbb{P}(\{\omega \})={\sum }_{\omega \in A}w(\omega )\mathrm{.}$ Taking $A=\mathrm{\Omega }$ yields ${\sum }_{\omega \in \mathrm{\Omega }}w(\omega )=\mathbb{P}(\mathrm{\Omega })=1$, and nonnegativity of $w$ comes from the measure axioms inside [L2]. So $(\mathrm{\Omega },w)$ is a finite probability space.

Step 1.1 constructs a full-power-set probability measure from any finite weight model, and step 1.2 recovers exactly those singleton weights from any full-power-set probability measure. Hence the two descriptions are equivalent, including the boundary case of outcomes with weight $0$.

By [L1], every subset of $\mathrm{\Omega }$ is measurable. Hence for every Borel set $B\subseteq \mathbb{R}$, the preimage $X^{-1}(B)$ is a subset of $\mathrm{\Omega }$, so it lies in $\mathcal{P}(\mathrm{\Omega })$. Therefore $X$ is measurable.

Step 1.1 proves that every finite random variable in the sense of [L3] is a real random variable in the sense of [L2], so the two notions agree exactly on finite full-power-set probability spaces.

By [L2], every $B\in \mathrm{\Sigma }$ has $X^{-1}(B)\in \mathcal{F}$, so [L1] is well defined. Also $X^{-1}(\mathrm{\varnothing })=\mathrm{\varnothing }$ and $X^{-1}(S)=\mathrm{\Omega }$, so ${\mathbb{P}}_X(\mathrm{\varnothing })=0,{\mathbb{P}}_X(S)=\mathbb{P}(\mathrm{\Omega })=1.$

If $(B_n)$ is a pairwise disjoint sequence in $\mathrm{\Sigma }$, then the preimages $X^{-1}(B_n)$ are pairwise disjoint and $X^{-1}\left({\bigcup }_n{B}_n\right)={\bigcup }_n{X}^{-1}(B_n)\mathrm{.}$ Therefore ${\mathbb{P}}_X\left({\bigcup }_n{B}_n\right)=\mathbb{P}({\bigcup }_n{X}^{-1}(B_n))={\sum }_{n=0}^{\mathrm{\infty }}\mathbb{P}(X^{-1}(B_n))={\sum }_{n=0}^{\mathrm{\infty }}{\mathbb{P}}_X(B_n)\mathrm{.}$

Steps 1.1 and 1.2 show that ${\mathbb{P}}_X$ is a measure of total mass $1$, hence a probability measure by [L3].

By [L1], the composite $g\circ X$ is measurable, hence a random element.

For every $B\in \mathrm{T}$, $(g\circ X{)}^{-1}(B)=X^{-1}(g^{-1}(B))\mathrm{.}$ Therefore [L2] gives ${\mathbb{P}}_{g\circ X}(B)=\mathbb{P}((g\circ X{)}^{-1}(B))=\mathbb{P}(X^{-1}(g^{-1}(B)))={\mathbb{P}}_X(g^{-1}(B))\mathrm{.}$ This right-hand side is defined because [L3] makes ${\mathbb{P}}_X$ a probability measure on $(S,\mathrm{\Sigma })$.

Steps 1.1 and 1.2 prove the measurable-map compatibility of laws.

If $a<b$, then $(-\mathrm{\infty },a]\subseteq (-\mathrm{\infty },b]$, so [L1] and [L2] give $F_X(a)\le F_X(b)$. Also $(-\mathrm{\infty },b]\setminus (-\mathrm{\infty },a]=(a,b],$ so the finite-measure difference formula from [L2] yields ${\mathbb{P}}_X((a,b])=F_X(b)-F_X(a)\mathrm{.}$

For fixed $x$, the sets $(-\mathrm{\infty },x+1\mathrm{/}n]$ decrease to $(-\mathrm{\infty },x]$, and ${\mathbb{P}}_X((-\mathrm{\infty },x+1])\le {\mathbb{P}}_X(\mathbb{R})=1$. Hence [L2] gives right continuity of $F_X$. Likewise $(-\mathrm{\infty },n]\uparrow \mathbb{R}$ and $(-\mathrm{\infty },-n]\downarrow \mathrm{\varnothing }$, so continuity from below and from above give ${\lim }_{n\to \mathrm{\infty }}{F}_X(n)=1,{\lim }_{n\to \mathrm{\infty }}{F}_X(-n)=0.$

Put $G(x):=F(x)-F(0)$. Then $G$ is still nondecreasing and right-continuous, so [L3] gives a unique Borel measure $\mu$ finite on compact sets with $\mu ((a,b])=G(b)-G(a)=F(b)-F(a)(a<b)\mathrm{.}$

For fixed $x$, the sets $(-n,x]$ increase to $(-\mathrm{\infty },x]$. Hence [L2] and step 1.3 give $\mu ((-\mathrm{\infty },x])={\lim }_{n\to \mathrm{\infty }}\mu ((-n,x])={\lim }_{n\to \mathrm{\infty }}(F(x)-F(-n))=F(x)\mathrm{.}$ Applying continuity from below once more to $(-\mathrm{\infty },n]\uparrow \mathbb{R}$ shows $\mu (\mathbb{R})={\lim }_{n}F(n)=1$, so $\mu$ is a probability measure. If $\nu$ is another Borel probability measure with $\nu ((-\mathrm{\infty },x])=F(x)$ for all $x$, then $\nu ((a,b])=F(b)-F(a)=\mu ((a,b])$ for every $a<b$, and [L3] gives $\nu =\mu$.

Steps 1.1 and 1.2 prove part 1, and steps 1.3 and 2.1 prove part 2.

Since $X=Y$ almost surely and both are integrable, [L2] applied to the measurable set $\mathrm{\Omega }$ gives ${\int }_{\mathrm{\Omega }}Xd\mathbb{P}={\int }_{\mathrm{\Omega }}Yd\mathbb{P}\mathrm{.}$

Rewriting the two integrals as expectations by [L1] yields $\mathbb{E}[X]=\mathbb{E}[Y]$.

By [L2], the composite $g\circ X$ is measurable. If $s={\sum }_{j=1}^m{c}_j{\chi }_{B_j}$ is a nonnegative simple function on $S$ with the $B_j$ pairwise disjoint, then $s\circ X={\sum }_{j=1}^m{c}_j{\chi }_{X^{-1}(B_j)}$ is a nonnegative simple function on $\mathrm{\Omega }$. Using [L1], [L3], and [L4], $\mathbb{E}[s(X)]={\sum }_{j=1}^m{c}_j\mathbb{P}(X^{-1}(B_j))={\sum }_{j=1}^m{c}_j{\mathbb{P}}_X(B_j)={\int }_{S}sd{\mathbb{P}}_X\mathrm{.}$

Assume now that $g\ge 0$. By [L3], choose nonnegative simple functions $s_n\uparrow g$ on $S$. Then $s_n\circ X\uparrow g\circ X$ by step 1.1, so monotone convergence on both spaces and step 1.1 give $\mathbb{E}[g(X)]={\lim }_{n\to \mathrm{\infty }}\mathbb{E}[s_n(X)]={\lim }_{n\to \mathrm{\infty }}{\int }_S{s}_{n}d{\mathbb{P}}_X={\int }_{S}gd{\mathbb{P}}_X\mathrm{.}$

Assume $g(X)$ is integrable. Then $\mathbb{E}[\mathrm{\mid }g(X)\mathrm{\mid }]<\mathrm{\infty }$, so step 2.1 applied to $\mathrm{\mid }g\mathrm{\mid }$ gives ${\int }_S\mathrm{\mid }g\mathrm{\mid }d{\mathbb{P}}_X=\mathbb{E}[\mathrm{\mid }g(X)\mathrm{\mid }]<\mathrm{\infty }\mathrm{.}$ Hence $g$ is ${\mathbb{P}}_X$-integrable. For real-valued $g$, [L4] and [L5] give $g=g^{+}-g^{-}$ with both parts nonnegative, so step 2.1 and linearity yield $\mathbb{E}[g(X)]=\mathbb{E}[g^{+}(X)]-\mathbb{E}[g^{-}(X)]={\int }_S{g}^{+}d{\mathbb{P}}_X-{\int }_S{g}^{-}d{\mathbb{P}}_X={\int }_{S}gd{\mathbb{P}}_X\mathrm{.}$

If $g$ is complex-valued, write $g=u+iv$ with real measurable parts $u,v$. The inequality $\mathrm{\mid }u\mathrm{\mid },\mathrm{\mid }v\mathrm{\mid }\le \mathrm{\mid }g\mathrm{\mid }$ and step 3.1 show that $u(X)$ and $v(X)$ are integrable, so the real-valued case applied to $u$ and $v$, followed by complex-linearity from [L5], gives $\mathbb{E}[g(X)]=\mathbb{E}[u(X)]+i\mathbb{E}[v(X)]={\int }_{S}ud{\mathbb{P}}_X+i{\int }_{S}vd{\mathbb{P}}_X={\int }_{S}gd{\mathbb{P}}_X\mathrm{.}$

Step 2.1 proves the nonnegative case, while steps 3.1 and 4.1 prove the integrable real and complex cases.

By [L1], the general expectation and the law of $X$ are defined on the same full-power-set probability space attached to $(\mathrm{\Omega },w)$.

Applying [L2] to the identity map on $\mathbb{R}$ gives $\mathbb{E}[X]={\int }_{\mathbb{R}}xd{\mathbb{P}}_X\mathrm{.}$ For a finite random variable, [L3] identifies this quantity with both ${\sum }_{\omega \in \mathrm{\Omega }}X(\omega )w(\omega )\text{and}{\sum }_{x\in X(\mathrm{\Omega })}x\mathbb{P}(X=x)\mathrm{.}$

Thus the general expectation is exactly the published finite weighted-sum expectation and its finite-distribution reformulation.

The function ${\mathbf{1}}_A$ is the simple function ${\mathbf{1}}_A=1\cdot {\chi }_A+0\cdot {\chi }_{A^c}\mathrm{.}$ Therefore [L1] and [L3] give $\mathbb{E}[{\mathbf{1}}_A]=1\cdot \mathbb{P}(A)+0\cdot \mathbb{P}(A^c)\mathrm{.}$

Using [L2] only to note that $A^c$ is the complementary event in the same probability space, step 1.1 simplifies to $\mathbb{E}[{\mathbf{1}}_A]=\mathbb{P}(A)$.

Apply [L2] with $f=X$ and $p=1$. Because $X\ge 0$, one has $\mathrm{\mid }X\mathrm{\mid }=X$ and $\{\mathrm{\mid }X\mathrm{\mid }>t\}=\{X>t\}$, so [L1] gives $\mathbb{E}[X]={\int }_0^{\mathrm{\infty }}\mathbb{P}(X>t)dt\mathrm{.}$

If $X$ is integrable and real, then $X^{+},X^{-}\in L^1$ and [L1] gives $X=X^{+}-X^{-}$. By step 1.1 applied to $X^{+}$ and $X^{-}$, $\mathbb{E}[X^{+}]={\int }_0^{\mathrm{\infty }}\mathbb{P}(X>t)dt,\mathbb{E}[X^{-}]={\int }_0^{\mathrm{\infty }}\mathbb{P}(X<-t)dt\mathrm{.}$ Subtracting these identities and using [L3] proves the second formula.

Rewriting expectation as the integral by [L1], linearity in [L2] gives $\mathbb{E}[aX+bY]=a\mathbb{E}[X]+b\mathbb{E}[Y]\mathrm{.}$

Applying the integral triangle inequality from [L2] after [L1] gives $\mathrm{\mid }\mathbb{E}[X]\mathrm{\mid }=\mid \int Xd\mathbb{P}\mid \le \int \mathrm{\mid }X\mathrm{\mid }d\mathbb{P}=\mathbb{E}[\mathrm{\mid }X\mathrm{\mid }]\mathrm{.}$

If $X\le Y$ almost surely, then $Y-X\ge 0$ almost surely. Hence [L1] and [L2] give $0\le \mathbb{E}[Y-X]=\mathbb{E}[Y]-\mathbb{E}[X],$ so $\mathbb{E}[X]\le \mathbb{E}[Y]$.

Steps 1.1, 1.2, and 2.1 prove the three assertions.

Expanding $(X-\mathbb{E}[X]{)}^2$ and applying [L2] gives $\mathrm{Var}(X)=\mathbb{E}[X^2]-2\mathbb{E}[X]\mathbb{E}[X]+\mathbb{E}[X{]}^2=\mathbb{E}[X^2]-\mathbb{E}[X{]}^2\mathrm{.}$ Likewise, $\mathrm{Cov}(X,Y)=\mathbb{E}[XY]-\mathbb{E}[X]\mathbb{E}[Y]\mathrm{.}$

The covariance formula in step 1.1 is symmetric in $X$ and $Y$, so $\mathrm{Cov}(X,Y)=\mathrm{Cov}(Y,X)$. If $U={\sum }_{i<m}{a}_i{X}_i$ and $V={\sum }_{j<n}{b}_j{Y}_j$ are finite linear combinations of square-integrable real random variables, expanding $\mathbb{E}[UV]-\mathbb{E}[U]\mathbb{E}[V]$ and using [L2] gives $\mathrm{Cov}(U,V)={\sum }_{i<m}{\sum }_{j<n}{a}_i{b}_j\mathrm{Cov}(X_i,Y_j)\mathrm{.}$

On a finite probability space, [L3] identifies the general formulas above with the already-published finite ones, so the finite and general identities agree exactly.

If $\phi (X)$ is nonnegative and $\mathbb{E}[\phi (X)]=+\mathrm{\infty }$, then the claimed inequality is automatic, because $\phi (\mathbb{E}[X])$ is a real number while the right-hand side is $+\mathrm{\infty }$.

In every remaining case, $\phi (X)$ is integrable: this is assumed directly, or follows from nonnegativity and finite expectation. Thus [L1] and [L2] applied to the probability measure $\mathbb{P}$ give $\phi (\mathbb{E}[X])=\phi (\int Xd\mathbb{P})\le \int \phi (X)d\mathbb{P}=\mathbb{E}[\phi (X)]\mathrm{.}$

Steps 1.1 and 1.2 cover the infinite and finite expectation cases.

Apply [L2] to the probability measure $\mathbb{P}$, the function $X$, and the threshold $a>0$. Rewriting the integral by [L1] gives $\mathbb{P}(X\ge a)\le \frac{\mathbb{E}[X]}{a}\mathrm{.}$

Step 1.1 is exactly Markov's inequality for random variables.

The random variable $Y:=(X-\mathbb{E}[X]{)}^2$ is nonnegative, and $\{Y\ge a^2\}=\{\mathrm{\mid }X-\mathbb{E}[X]\mathrm{\mid }\ge a\}\mathrm{.}$

Applying [L2] to $Y$ and using [L1] gives $\mathbb{P}(\mathrm{\mid }X-\mathbb{E}[X]\mathrm{\mid }\ge a)=\mathbb{P}(Y\ge a^2)\le \frac{\mathbb{E}[Y]}{a^2}=\frac{\mathrm{Var}(X)}{a^2}\mathrm{.}$

Apply [L2] to the measure space $(\mathrm{\Omega },\mathcal{F},\mathbb{P})$. Rewriting the left-hand side with [L1] gives $\mathbb{E}[\mathrm{\mid }XY\mathrm{\mid }]=\int \mathrm{\mid }XY\mathrm{\mid }d\mathbb{P}\le \mathrm{\parallel }X{\mathrm{\parallel }}_p\mathrm{\parallel }Y{\mathrm{\parallel }}_q\mathrm{.}$

The same theorem [L2] already states that the right-hand side is finite in every allowed case, so $XY$ is integrable.

Step [L1] at $p=q=2$ gives $\mathbb{E}[\mathrm{\mid }XY\mathrm{\mid }]\le \mathrm{\parallel }X{\mathrm{\parallel }}_2\mathrm{\parallel }Y{\mathrm{\parallel }}_2=(\mathbb{E}[X^2]{)}^{1\mathrm{/}2}(\mathbb{E}[Y^2]{)}^{1\mathrm{/}2}\mathrm{.}$

The equality clause is exactly the probability-measure specialization of [L2].

If $p=q$, the inequality is equality. If $p<q<\mathrm{\infty }$, apply [L2] with $\mu =\mathbb{P}$ and use [L1] to collapse the factor $\mathbb{P}(\mathrm{\Omega }{)}^{1\mathrm{/}p-1\mathrm{/}q}$ to $1$.

If $q=\mathrm{\infty }$, the same specialization of [L2] and [L1] gives $\mathrm{\parallel }X{\mathrm{\parallel }}_p\le \mathbb{P}(\mathrm{\Omega }{)}^{1\mathrm{/}p}\mathrm{\parallel }X{\mathrm{\parallel }}_{\mathrm{\infty }}=\mathrm{\parallel }X{\mathrm{\parallel }}_{\mathrm{\infty }}\mathrm{.}$

Steps 1.1 and 1.2 are exactly the finite-$q$ and $q=\mathrm{\infty }$ cases of Lyapunov's moment inequality on a probability space.