Finite Probability Spaces and Random Variables

The empty sum is $0$, while the sum over $\mathrm{\Omega }$ is $1$, so $\mathbb{P}(\mathrm{\varnothing })=0$ and $\mathbb{P}(\mathrm{\Omega })=1$.

Since every summand in $\mathbb{P}(A)$ is nonnegative, $0\le \mathbb{P}(A)$. Splitting the sum over $B$ into $A$ and $B\setminus A$ gives $\mathbb{P}(B)=\mathbb{P}(A)+\mathbb{P}(B\setminus A)$, so $\mathbb{P}(A)\le \mathbb{P}(B)$.

Taking $B=\mathrm{\Omega }$ in step 1.2 gives $\mathbb{P}(A^c)=1-\mathbb{P}(A)$; rearranging the same identity for general $A\subseteq B$ gives $\mathbb{P}(B\setminus A)=\mathbb{P}(B)-\mathbb{P}(A)$.

The definition permits zero weights, so a singleton outcome of weight zero is a nonempty event of probability zero. All displayed conclusions follow.

Pairwise disjointness makes $(i,\omega )\mapsto \omega$ a bijection from $\{(i,\omega ):i\in I,\omega \in A_i\}$ onto ${\bigcup }_{i\in I}{A}_i$.

Reindexing by this bijection and summing first over each fibre gives $\mathbb{P}({\bigcup }_i{A}_i)={\sum }_{i\in I}{\sum }_{\omega \in A_i}w(\omega )={\sum }_{i\in I}\mathbb{P}(A_i)$.

If $I=\mathrm{\varnothing }$, both sides are the empty sum $0$; if $I$ has one member, step 2.1 is the identity $\mathbb{P}(A)=\mathbb{P}(A)$.

The sets $A$ and $B\setminus A$ are disjoint and have union $A\cup B$, so $\mathbb{P}(A\cup B)=\mathbb{P}(A)+\mathbb{P}(B\setminus A)$.

Since $B\setminus A=B\setminus (A\cap B)$, one has $\mathbb{P}(B\setminus A)=\mathbb{P}(B)-\mathbb{P}(A\cap B)$.

Substitution of step 1.2 into step 1.1 gives the formula.

For $i<n$, put $B_i=A_i\setminus {\bigcup }_{j<i}{A}_j$. Then the $B_i$ are pairwise disjoint, $B_i\subseteq A_i$, and ${\bigcup }_{i<n}{B}_i={\bigcup }_{i<n}{A}_i$.

Hence $\mathbb{P}({\bigcup }_{i<n}{A}_i)={\sum }_{i<n}\mathbb{P}(B_i)\le {\sum }_{i<n}\mathbb{P}(A_i)$.

For $n=0$ step 2.1 reads $0\le 0$, and for $n=1$ it is equality.

Multiplying the identity in [L1] by the positive denominator gives $\mathbb{P}(A\cap B)=\mathbb{P}(A\mid B)\mathbb{P}(B)$.

For $n=0$ the empty intersection is $\mathrm{\Omega }$ and the empty product is $1$, while for $n=1$ the asserted identity is immediate.

Assume the chain formula holds for $n=k\ge 1$, and suppose the stated prefix probabilities are positive for $A_0,\dots ,A_k$.

Apply step 1.1 to $A=A_k$ and $B={\bigcap }_{i<k}{A}_i$, then substitute the induction hypothesis for $\mathbb{P}(B)$; this gives the chain formula for $k+1$.

Induction proves the formula for every finite $n$, and every displayed conditional probability has a positive conditioning event by hypothesis.

The events $A\cap B_i$ are pairwise disjoint and have union $A$, so $\mathbb{P}(A)={\sum }_{i<n}\mathbb{P}(A\cap B_i)$.

Each positive-probability cell satisfies $\mathbb{P}(A\cap B_i)=\mathbb{P}(A\mid B_i)\mathbb{P}(B_i)$.

Substitution gives the displayed formula. If $\mathbb{P}(B_i)=0$, [L3] gives $0\le \mathbb{P}(A\cap B_i)\le \mathbb{P}(B_i)=0$, so deleting that cell changes neither side of the unconditioned decomposition.

By conditional probability and symmetry of intersection, $\mathbb{P}(B_j\mid A)=\mathbb{P}(A\cap B_j)\mathrm{/}\mathbb{P}(A)$.

The numerator in step 1.1 is $\mathbb{P}(A\mid B_j)\mathbb{P}(B_j)$, and [L2] is the denominator.

Substitution yields the formula; its denominator is positive because it equals $\mathbb{P}(A)>0$.

If $A$ and $B$ are independent, divide the identity in [L2] by $\mathbb{P}(B)>0$ to obtain $\mathbb{P}(A\mid B)=\mathbb{P}(A)$.

The same calculation with $A$ and $B$ interchanged gives $\mathbb{P}(B\mid A)=\mathbb{P}(B)$.

Conversely, multiplying either conditional identity by its positive conditioning probability gives the product identity in [L2].

Thus each of conditions 2 and 3 is equivalent to condition 1, proving all three equivalent.

With no event complemented, every intersection identity required for a subfamily is already one of the identities required for the original family.

Assume that after complementing any chosen $r$ events, every resulting subfamily is mutually independent.

Complement one further event $A_i$. For any intersection $D$ of selected events other than $A_i$, the induction hypothesis gives $\mathbb{P}(D\cap A_i)=\mathbb{P}(D)\mathbb{P}(A_i)$. Since $D\cap A_i^c=D\setminus (D\cap A_i)$, [L1] gives $\mathbb{P}(D\cap A_i^c)=\mathbb{P}(D)(1-\mathbb{P}(A_i))=\mathbb{P}(D)\mathbb{P}(A_i^c)$. Thus every subfamily remains mutually independent after $r+1$ replacements.

Induction on the number of complemented events proves the assertion for every selection. The empty intersection has probability $1$ and the empty product is $1$.

For $I=\mathrm{\varnothing }$, the unique product outcome has weight $1$, so normalization holds.

If normalization holds for $I$ and another factor $({\mathrm{\Omega }}_k,w_k)$ is appended, finite Fubini gives ${\sum }_{(\omega ,u)}w(\omega )w_k(u)=({\sum }_{\omega }w(\omega ))({\sum }_u{w}_k(u))=1$.

For $J\subseteq I$, summing the product weights over ${\bigcap }_{j\in J}{C}_j$ factors coordinatewise: a coordinate in $J$ contributes ${\mathbb{P}}_j(A_j)$ and a coordinate outside $J$ contributes $1$.

Induction proves normalization for every finite product; nonnegativity follows from nonnegativity of all factor weights.

Thus $\mathbb{P}({\bigcap }_{j\in J}{C}_j)={\prod }_{j\in J}{\mathbb{P}}_j(A_j)$. The empty $J$ gives $1=1$, so [L3] proves mutual independence in all cases.

The nonempty fibres $X^{-1}(\{x\})$, for $x\in X(\mathrm{\Omega })$, form a finite partition of $\mathrm{\Omega }$.

Splitting the expectation over these fibres gives $\mathbb{E}[X]={\sum }_{x\in X(\mathrm{\Omega })}{\sum }_{\omega :X(\omega )=x}xw(\omega )$.

The inner sum is $x\mathbb{P}(X=x)$, which proves the formula. The range cannot be empty because total probability is $1$.

Expanding the left side gives ${\sum }_{\omega \in \mathrm{\Omega }}{\sum }_{i\in I}{a}_i{X}_i(\omega )w(\omega )$.

Finite Fubini and distributivity turn step 1.1 into ${\sum }_{i\in I}{a}_i{\sum }_{\omega \in \mathrm{\Omega }}{X}_i(\omega )w(\omega )={\sum }_{i\in I}{a}_i\mathbb{E}[X_i]$.

The calculation uses no independence identity. If $I=\mathrm{\varnothing }$, the two sums in step 2.1 are empty and equal $0$.

Nonnegative weights give $X(\omega )w(\omega )\le Y(\omega )w(\omega )$ for every $\omega$, so summing gives $\mathbb{E}[X]\le \mathbb{E}[Y]$.

The outcome set is nonempty because its nonnegative weights sum to $1$; let $m$ and $M$ be the minimum and maximum of the attained values of $X$.

The pointwise inequalities $m\le X\le M$ and step 1.1 give $m\le \mathbb{E}[X]\le M$, since the expectations of the constant variables are $m$ and $M$.

Outcomes attaining $m$ and $M$ therefore supply values at most and at least the expectation, including the constant case.

Expanding $\mathbb{E}[{\mathbf{1}}_A]$ leaves exactly the weights of outcomes in $A$, hence equals $\mathbb{P}(A)$.

At an outcome $\omega$, the product ${\prod }_i{\mathbf{1}}_{A_i}(\omega )$ is $1$ exactly when $\omega$ belongs to every $A_i$, and is otherwise $0$.

At $\omega$, each summand ${\mathbf{1}}_{A_i}(\omega )$ contributes one exactly when $\omega \in A_i$, so their sum counts those events.

Steps 1.2 and 1.3 also give the stated empty conventions by [L3].

Suppose the events are mutually independent. Every joint assignment ${\mathbf{1}}_{A_j}=b_j$, with $b_j\in \{0,1\}$, is an intersection of events $A_j$ and complements $A_j^c$, whose probability factors by [L1].

Conversely, if the indicators are mutually independent, specialize their joint-value identity to $b_j=1$ for every chosen index; [L2] gives the event-intersection product identity.

Hence the indicators are mutually independent by [L3].

Step 1.1 together with step 2.1 proves the forward direction, and step 1.2 proves the reverse direction. Restricting the same arguments to two indices proves the pairwise equivalence.

Grouping outcomes by the joint values $(x_i{)}_{i\in I}$ gives $\mathbb{E}[{\prod }_i{X}_i]={\sum }_{(x_i)}({\prod }_i{x}_i)\mathbb{P}(X_i=x_i\text{ for all }i)$.

Independence changes the last probability to ${\prod }_i\mathbb{P}(X_i=x_i)$.

Finite Fubini factors the resulting sum as ${\prod }_i{\sum }_{x_i}{x}_i\mathbb{P}(X_i=x_i)={\prod }_i\mathbb{E}[X_i]$. For $I=\mathrm{\varnothing }$, this calculation is the empty product identity $1=1$.

Expand $(X-\mathbb{E}[X]{)}^2=X^2-2\mathbb{E}[X]X+\mathbb{E}[X{]}^2$ and apply linearity to obtain $\mathrm{Var}(X)=\mathbb{E}[X^2]-\mathbb{E}[X{]}^2$.

Expanding $(X-\mathbb{E}[X])(Y-\mathbb{E}[Y])$ and applying linearity gives the covariance identity.

The centred square is pointwise nonnegative, so variance is nonnegative; if $X$ is constant it vanishes identically. If the variance is zero, every positive-weight centred-square summand is zero, while a zero-weight outcome is unrestricted.

Steps 1.1, 1.2, and 1.3 give all claims.

Commutativity of real multiplication in [L2] gives $\mathrm{Cov}(X,Y)=\mathrm{Cov}(Y,X)$.

Substitute the two finite linear combinations into [L2] and distribute their pointwise product.

Applying [L1] to step 1.2 and collecting the $a_i{b}_j$ terms gives the displayed double sum.

If either index set is empty, both sides are zero; covariance with a constant is zero by [L1] and [L2].

By [L1] and bilinearity, $\mathrm{Var}({\sum }_i{X}_i)=\mathrm{Cov}({\sum }_i{X}_i,{\sum }_j{X}_j)={\sum }_{i,j}\mathrm{Cov}(X_i,X_j)$.

Separate the diagonal terms, which are $\mathrm{Var}(X_i)$, from the off-diagonal ordered pairs. Symmetry pairs the latter into twice the sum over unordered pairs.

For an empty family every sum in step 1.1 is zero, and for a singleton only its diagonal term remains.

For distinct $i,j$, pairwise independence and [L1] give $\mathbb{E}[X_i{X}_j]=\mathbb{E}[X_i]\mathbb{E}[X_j]$, so $\mathrm{Cov}(X_i,X_j)=0$.

Substitute step 1.1 into [L2]; every off-diagonal term vanishes, leaving the displayed formula. The empty and singleton cases are included.

Since $X$ is $1$ with probability $p$ and $0$ otherwise, $\mathbb{E}[X]=p$ and $\mathbb{E}[X^2]=p$.

Hence $\mathrm{Var}(X)=p-p^2=p(1-p)$.

Linearity gives $\mathbb{E}[S]={\sum }_{i<n}p=np$, and pairwise-independent variance additivity gives $\mathrm{Var}(S)={\sum }_{i<n}p(1-p)=np(1-p)$.

At $p=0$ or $p=1$ the variance formula is zero, and at $n=0$ both finite sums are empty and equal zero.

Pointwise, $X\ge a{\mathbf{1}}_{\{X\ge a\}}$: on the event this is the threshold inequality, and off it the right side is zero while $X\ge 0$.

Taking expectations gives $\mathbb{E}[X]\ge a\mathbb{P}(X\ge a)$.

Division by $a>0$ gives the claimed inequality. The hypothesis $a>0$ is exactly what licenses the division.