Probability Spaces Random Variables and Expectation

Because $\mathbb{P}(\mathrm{\Omega })=1$ by [L1] and $A\subseteq \mathrm{\Omega }$, [L2] gives $1=\mathbb{P}(\mathrm{\Omega })=\mathbb{P}(A)+\mathbb{P}(A^c),$ so $\mathbb{P}(A^c)=1-\mathbb{P}(A)$. Monotonicity in [L2] also gives $A\subseteq B\Rightarrow \mathbb{P}(A)\le \mathbb{P}(B)$.

Since every probability is at most $1$, the finite-measure hypotheses in [L2] apply to $A$, $B$, and $A\cup B$. Thus if $A\subseteq B$, then $\mathbb{P}(B\setminus A)=\mathbb{P}(B)-\mathbb{P}(A),$ subadditivity gives the countable and finite union bounds, finite inclusion-exclusion gives $\mathbb{P}(A\cup B)=\mathbb{P}(A)+\mathbb{P}(B)-\mathbb{P}(A\cap B),$ and continuity from below and from above give the two monotone-limit formulas, because a decreasing probability sequence always has finite first term.

Steps 1.1 and 1.2 are exactly the stated probability identities.

If $(\mathrm{\Omega },w)$ is a finite probability space, then [L1] makes every subset an event and gives the displayed formula. It is nonnegative, has ${\mathbb{P}}_w(\mathrm{\varnothing })=0$, and has ${\mathbb{P}}_w(\mathrm{\Omega })=1$. For a sequence of pairwise disjoint subsets $(A_j{)}_{j\ge 0}$, at most $\mathrm{\mid }\mathrm{\Omega }\mathrm{\mid }$ of them are nonempty. Each $\omega$ in their union occurs in exactly one $A_j$, so rearranging this finite sum gives ${\mathbb{P}}_w({\bigcup }_j{A}_j)={\sum }_j{\mathbb{P}}_w(A_j)$. Thus ${\mathbb{P}}_w$ is a probability measure on $(\mathrm{\Omega },\mathcal{P}(\mathrm{\Omega }))$.

Conversely, let $\mathbb{P}$ be a probability measure on $(\mathrm{\Omega },\mathcal{P}(\mathrm{\Omega }))$ and put $w(\omega )=\mathbb{P}(\{\omega \})$. Each singleton is an atom of the full power-set sigma-algebra, and every $A\subseteq \mathrm{\Omega }$ is the union of the singletons it contains. Thus [L3] gives $\mathbb{P}(A)={\sum }_{\omega \in A}\mathbb{P}(\{\omega \})={\sum }_{\omega \in A}w(\omega )\mathrm{.}$ Taking $A=\mathrm{\Omega }$ yields ${\sum }_{\omega \in \mathrm{\Omega }}w(\omega )=\mathbb{P}(\mathrm{\Omega })=1$, and nonnegativity of $w$ comes from the measure axioms inside [L2]. So $(\mathrm{\Omega },w)$ is a finite probability space.

Step 1.1 constructs a full-power-set probability measure from any finite weight model, and step 1.2 recovers exactly those singleton weights from any full-power-set probability measure. Hence the two descriptions are equivalent, including the boundary case of outcomes with weight $0$.

By [L1], every subset of $\mathrm{\Omega }$ is measurable. Hence for every Borel set $B\subseteq \mathbb{R}$, the preimage $X^{-1}(B)$ is a subset of $\mathrm{\Omega }$, so it lies in $\mathcal{P}(\mathrm{\Omega })$. Therefore $X$ is measurable.

Step 1.1 proves that every finite random variable in the sense of [L3] is a real random variable in the sense of [L2], so the two notions agree exactly on finite full-power-set probability spaces.

By [L2], every $B\in \mathrm{\Sigma }$ has $X^{-1}(B)\in \mathcal{F}$, so [L1] is well defined. Also $X^{-1}(\mathrm{\varnothing })=\mathrm{\varnothing }$ and $X^{-1}(S)=\mathrm{\Omega }$, so ${\mathbb{P}}_X(\mathrm{\varnothing })=0,{\mathbb{P}}_X(S)=\mathbb{P}(\mathrm{\Omega })=1.$

If $(B_n)$ is a pairwise disjoint sequence in $\mathrm{\Sigma }$, then the preimages $X^{-1}(B_n)$ are pairwise disjoint and $X^{-1}\left({\bigcup }_n{B}_n\right)={\bigcup }_n{X}^{-1}(B_n)\mathrm{.}$ Therefore ${\mathbb{P}}_X\left({\bigcup }_n{B}_n\right)=\mathbb{P}({\bigcup }_n{X}^{-1}(B_n))={\sum }_{n=0}^{\mathrm{\infty }}\mathbb{P}(X^{-1}(B_n))={\sum }_{n=0}^{\mathrm{\infty }}{\mathbb{P}}_X(B_n)\mathrm{.}$

Steps 1.1 and 1.2 show that ${\mathbb{P}}_X$ is a measure of total mass $1$, hence a probability measure by [L3].

By [L1], the composite $g\circ X$ is measurable, hence a random element.

For every $B\in \mathrm{T}$, $(g\circ X{)}^{-1}(B)=X^{-1}(g^{-1}(B))\mathrm{.}$ Therefore [L2] gives ${\mathbb{P}}_{g\circ X}(B)=\mathbb{P}((g\circ X{)}^{-1}(B))=\mathbb{P}(X^{-1}(g^{-1}(B)))={\mathbb{P}}_X(g^{-1}(B))\mathrm{.}$ This right-hand side is defined because [L3] makes ${\mathbb{P}}_X$ a probability measure on $(S,\mathrm{\Sigma })$.

Steps 1.1 and 1.2 prove the measurable-map compatibility of laws.

If $a<b$, then $(-\mathrm{\infty },a]\subseteq (-\mathrm{\infty },b]$, so [L1] and [L2] give $F_X(a)\le F_X(b)$. Also $(-\mathrm{\infty },b]\setminus (-\mathrm{\infty },a]=(a,b],$ so the finite-measure difference formula from [L2] yields ${\mathbb{P}}_X((a,b])=F_X(b)-F_X(a)\mathrm{.}$

For fixed $x$, the sets $(-\mathrm{\infty },x+1\mathrm{/}n]$ decrease to $(-\mathrm{\infty },x]$, and ${\mathbb{P}}_X((-\mathrm{\infty },x+1])\le {\mathbb{P}}_X(\mathbb{R})=1$. Hence [L2] gives right continuity of $F_X$. Likewise $(-\mathrm{\infty },n]\uparrow \mathbb{R}$ and $(-\mathrm{\infty },-n]\downarrow \mathrm{\varnothing }$, so continuity from below and from above give ${\lim }_{n\to \mathrm{\infty }}{F}_X(n)=1,{\lim }_{n\to \mathrm{\infty }}{F}_X(-n)=0.$

Put $G(x):=F(x)-F(0)$. Then $G$ is still nondecreasing and right-continuous, so [L3] gives a unique Borel measure $\mu$ finite on compact sets with $\mu ((a,b])=G(b)-G(a)=F(b)-F(a)(a<b)\mathrm{.}$

For fixed $x$, the sets $(-n,x]$ increase to $(-\mathrm{\infty },x]$. Hence [L2] and step 1.3 give $\mu ((-\mathrm{\infty },x])={\lim }_{n\to \mathrm{\infty }}\mu ((-n,x])={\lim }_{n\to \mathrm{\infty }}(F(x)-F(-n))=F(x)\mathrm{.}$ Applying continuity from below once more to $(-\mathrm{\infty },n]\uparrow \mathbb{R}$ shows $\mu (\mathbb{R})={\lim }_{n}F(n)=1$, so $\mu$ is a probability measure. If $\nu$ is another Borel probability measure with $\nu ((-\mathrm{\infty },x])=F(x)$ for all $x$, then $\nu ((a,b])=F(b)-F(a)=\mu ((a,b])$ for every $a<b$, and [L3] gives $\nu =\mu$.

Steps 1.1 and 1.2 prove part 1, and steps 1.3 and 2.1 prove part 2.

Since $X=Y$ almost surely and both are integrable, [L2] applied to the measurable set $\mathrm{\Omega }$ gives ${\int }_{\mathrm{\Omega }}Xd\mathbb{P}={\int }_{\mathrm{\Omega }}Yd\mathbb{P}\mathrm{.}$

Rewriting the two integrals as expectations by [L1] yields $\mathbb{E}[X]=\mathbb{E}[Y]$.

By [L2], the composite $g\circ X$ is measurable. If $s={\sum }_{j=1}^m{c}_j{\chi }_{B_j}$ is a nonnegative simple function on $S$ with the $B_j$ pairwise disjoint, then $s\circ X={\sum }_{j=1}^m{c}_j{\chi }_{X^{-1}(B_j)}$ is a nonnegative simple function on $\mathrm{\Omega }$. Using [L1], [L3], and [L4], $\mathbb{E}[s(X)]={\sum }_{j=1}^m{c}_j\mathbb{P}(X^{-1}(B_j))={\sum }_{j=1}^m{c}_j{\mathbb{P}}_X(B_j)={\int }_{S}sd{\mathbb{P}}_X\mathrm{.}$

Assume now that $g\ge 0$. By [L3], choose nonnegative simple functions $s_n\uparrow g$ on $S$. Then $s_n\circ X\uparrow g\circ X$ by step 1.1, so monotone convergence on both spaces and step 1.1 give $\mathbb{E}[g(X)]={\lim }_{n\to \mathrm{\infty }}\mathbb{E}[s_n(X)]={\lim }_{n\to \mathrm{\infty }}{\int }_S{s}_{n}d{\mathbb{P}}_X={\int }_{S}gd{\mathbb{P}}_X\mathrm{.}$

Assume $g(X)$ is integrable. Then $\mathbb{E}[\mathrm{\mid }g(X)\mathrm{\mid }]<\mathrm{\infty }$, so step 2.1 applied to $\mathrm{\mid }g\mathrm{\mid }$ gives ${\int }_S\mathrm{\mid }g\mathrm{\mid }d{\mathbb{P}}_X=\mathbb{E}[\mathrm{\mid }g(X)\mathrm{\mid }]<\mathrm{\infty }\mathrm{.}$ Hence $g$ is ${\mathbb{P}}_X$-integrable. For real-valued $g$, [L4] and [L5] give $g=g^{+}-g^{-}$ with both parts nonnegative, so step 2.1 and linearity yield $\mathbb{E}[g(X)]=\mathbb{E}[g^{+}(X)]-\mathbb{E}[g^{-}(X)]={\int }_S{g}^{+}d{\mathbb{P}}_X-{\int }_S{g}^{-}d{\mathbb{P}}_X={\int }_{S}gd{\mathbb{P}}_X\mathrm{.}$

If $g$ is complex-valued, write $g=u+iv$ with real measurable parts $u,v$. The inequality $\mathrm{\mid }u\mathrm{\mid },\mathrm{\mid }v\mathrm{\mid }\le \mathrm{\mid }g\mathrm{\mid }$ and step 3.1 show that $u(X)$ and $v(X)$ are integrable, so the real-valued case applied to $u$ and $v$, followed by complex-linearity from [L5], gives $\mathbb{E}[g(X)]=\mathbb{E}[u(X)]+i\mathbb{E}[v(X)]={\int }_{S}ud{\mathbb{P}}_X+i{\int }_{S}vd{\mathbb{P}}_X={\int }_{S}gd{\mathbb{P}}_X\mathrm{.}$

Step 2.1 proves the nonnegative case, while steps 3.1 and 4.1 prove the integrable real and complex cases.

By [L1], the general expectation and the law of $X$ are defined on the same full-power-set probability space attached to $(\mathrm{\Omega },w)$.

Applying [L2] to the identity map on $\mathbb{R}$ gives $\mathbb{E}[X]={\int }_{\mathbb{R}}xd{\mathbb{P}}_X\mathrm{.}$ For a finite random variable, [L3] identifies this quantity with both ${\sum }_{\omega \in \mathrm{\Omega }}X(\omega )w(\omega )\text{and}{\sum }_{x\in X(\mathrm{\Omega })}x\mathbb{P}(X=x)\mathrm{.}$

Thus the general expectation is exactly the published finite weighted-sum expectation and its finite-distribution reformulation.

The function ${\mathbf{1}}_A$ is the simple function ${\mathbf{1}}_A=1\cdot {\chi }_A+0\cdot {\chi }_{A^c}\mathrm{.}$ Therefore [L1] and [L3] give $\mathbb{E}[{\mathbf{1}}_A]=1\cdot \mathbb{P}(A)+0\cdot \mathbb{P}(A^c)\mathrm{.}$

Using [L2] only to note that $A^c$ is the complementary event in the same probability space, step 1.1 simplifies to $\mathbb{E}[{\mathbf{1}}_A]=\mathbb{P}(A)$.

For each integer $n\ge 1$, set $X_n=\min \{X,n\}$. This is a finite-valued nonnegative measurable real function: for a real $a$, $\{X_n>a\}$ is $\mathrm{\Omega }$ if $a<0$, is $\{X>a\}$ if $0\le a<n$, and is empty if $a\ge n$. Also $X_n\uparrow X$, including where $X=+\mathrm{\infty }$. Apply [L2] to $X_n$ to obtain $\mathbb{E}[X_n]={\int }_0^{\mathrm{\infty }}\mathbb{P}(X_n>t)dt$. For $t\ge 0$, the tail $\mathbb{P}(X_n>t)$ equals $\mathbb{P}(X>t)$ when $t<n$ and zero when $t\ge n$, so these tails increase pointwise to $\mathbb{P}(X>t)$. Every tail is a nonincreasing Borel function of $t$: each strict upper level set is an initial interval, possibly empty or the whole half-line. Apply [L4] first on $\mathrm{\Omega }$ and then on $(0,\mathrm{\infty })$ to pass to the limits in the identities for $X_n$. Using [L1] gives $\mathbb{E}[X]={\int }_0^{\mathrm{\infty }}\mathbb{P}(X>t)dt,$ with $+\mathrm{\infty }$ allowed.

If $X$ is integrable and real, then $X^{+},X^{-}\in L^1$ and [L1] gives $X=X^{+}-X^{-}$. By step 1.1 applied to $X^{+}$ and $X^{-}$, $\mathbb{E}[X^{+}]={\int }_0^{\mathrm{\infty }}\mathbb{P}(X>t)dt,\mathbb{E}[X^{-}]={\int }_0^{\mathrm{\infty }}\mathbb{P}(X<-t)dt\mathrm{.}$ Subtracting these identities and using [L3] proves the second formula.

Rewriting expectation as the integral by [L1], linearity in [L2] gives $\mathbb{E}[aX+bY]=a\mathbb{E}[X]+b\mathbb{E}[Y]\mathrm{.}$

Applying the integral triangle inequality from [L2] after [L1] gives $\mathrm{\mid }\mathbb{E}[X]\mathrm{\mid }=\mid \int Xd\mathbb{P}\mid \le \int \mathrm{\mid }X\mathrm{\mid }d\mathbb{P}=\mathbb{E}[\mathrm{\mid }X\mathrm{\mid }]\mathrm{.}$

If $X\le Y$ almost surely, put $D=Y-X$, which is integrable by [L2]. Its positive part $D^{+}=\max \{D,0\}$ is measurable by [L4] and integrable because $D^{+}\le \mathrm{\mid }D\mathrm{\mid }$. Since $D\ge 0$ almost surely, $D^{+}=D$ almost surely, so [L3] gives $\int Dd\mathbb{P}=\int D^{+}d\mathbb{P}\ge 0$, where the last inequality concerns a pointwise nonnegative integrand. By [L1] and step 1.1, $0\le \mathbb{E}[Y-X]=\mathbb{E}[Y]-\mathbb{E}[X]$, so $\mathbb{E}[X]\le \mathbb{E}[Y]$.

Steps 1.1, 1.2, and 2.1 prove the three assertions.

Since the probability measure has total mass one, $2\mathrm{\mid }X\mathrm{\mid }\le 1+X^2$ makes $X$ integrable. Likewise $2\mathrm{\mid }XY\mathrm{\mid }\le X^2+Y^2$ makes $XY$ integrable. Finite linear combinations of square-integrable variables remain square-integrable by $(a+b{)}^2\le 2a^2+2b^2$, applied repeatedly. All expectations below are therefore finite.

Expanding $(X-\mathbb{E}[X]{)}^2$ and applying [L2] gives $\mathrm{Var}(X)=\mathbb{E}[X^2]-2\mathbb{E}[X]\mathbb{E}[X]+\mathbb{E}[X{]}^2=\mathbb{E}[X^2]-\mathbb{E}[X{]}^2\mathrm{.}$ Likewise, $\mathrm{Cov}(X,Y)=\mathbb{E}[XY]-\mathbb{E}[X]\mathbb{E}[Y]\mathrm{.}$

The covariance formula in step 2.1 is symmetric in $X$ and $Y$, so $\mathrm{Cov}(X,Y)=\mathrm{Cov}(Y,X)$. If $U={\sum }_{i<m}{a}_i{X}_i$ and $V={\sum }_{j<n}{b}_j{Y}_j$ are finite linear combinations of square-integrable real random variables, step 1.1 makes them square-integrable. Expanding $\mathbb{E}[UV]-\mathbb{E}[U]\mathbb{E}[V]$ and using [L2] gives $\mathrm{Cov}(U,V)={\sum }_{i<m}{\sum }_{j<n}{a}_i{b}_j\mathrm{Cov}(X_i,Y_j)\mathrm{.}$

On a finite full-power-set probability space, [L3] identifies the measure with its singleton weights. The integral of a finite-valued random variable is the sum of its values times those weights, so [L4] identifies the two expectations and the centered definitions. Thus the formulas in step 2.1 and step 3.1 agree with the finite identities.

Each variable is integrable because $2\mathrm{\mid }X\mathrm{\mid }\le 1+X^2$ on a probability space, and every product of two such variables is integrable because $2\mathrm{\mid }XY\mathrm{\mid }\le X^2+Y^2$. Finite real linear combinations remain square-integrable by repeated use of $(a+b{)}^2\le 2a^2+2b^2$. Thus all covariances and expectations below are defined and finite. Commutativity of real multiplication in [L2] gives $\mathrm{Cov}(X,Y)=\mathrm{Cov}(Y,X)$.

The finite linear combinations and their product have the required integrability by step 1.1. Substitute them into [L2] and distribute their pointwise product.

Applying [L1] to step 2.1 and collecting the $a_i{b}_j$ terms gives the displayed double sum.

If either index set is empty, both sides are zero; covariance with a constant is zero by [L1] and [L2].

The finite sum is square-integrable by repeated use of $(a+b{)}^2\le 2a^2+2b^2$. By [L1] and bilinearity, $\mathrm{Var}({\sum }_i{X}_i)=\mathrm{Cov}({\sum }_i{X}_i,{\sum }_j{X}_j)={\sum }_{i,j}\mathrm{Cov}(X_i,X_j)$.

Separate the diagonal terms, which are $\mathrm{Var}(X_i)$, from the off-diagonal ordered pairs. Symmetry pairs the latter into twice the sum over unordered pairs.

For an empty family every sum in step 1.1 is zero, and for a singleton only its diagonal term remains.

If $\phi (X)$ is nonnegative and $\mathbb{E}[\phi (X)]=+\mathrm{\infty }$, then the claimed inequality is automatic, because $\phi (\mathbb{E}[X])$ is a real number while the right-hand side is $+\mathrm{\infty }$.

In every remaining case, $\phi (X)$ is integrable: this is assumed directly, or follows from nonnegativity and finite expectation. Thus [L1] and [L2] applied to the probability measure $\mathbb{P}$ give $\phi (\mathbb{E}[X])=\phi (\int Xd\mathbb{P})\le \int \phi (X)d\mathbb{P}=\mathbb{E}[\phi (X)]\mathrm{.}$

Steps 1.1 and 1.2 cover the infinite and finite expectation cases.

Apply [L2] to the probability measure $\mathbb{P}$, the function $X$, and the threshold $a>0$. Rewriting the integral by [L1] gives $\mathbb{P}(X\ge a)\le \frac{\mathbb{E}[X]}{a}\mathrm{.}$

Step 1.1 is exactly Markov's inequality for random variables.

The random variable $Y:=(X-\mathbb{E}[X]{)}^2$ is nonnegative, and $\{Y\ge a^2\}=\{\mathrm{\mid }X-\mathbb{E}[X]\mathrm{\mid }\ge a\}\mathrm{.}$

Applying [L2] to $Y$ and using [L1] gives $\mathbb{P}(\mathrm{\mid }X-\mathbb{E}[X]\mathrm{\mid }\ge a)=\mathbb{P}(Y\ge a^2)\le \frac{\mathbb{E}[Y]}{a^2}=\frac{\mathrm{Var}(X)}{a^2}\mathrm{.}$