Finite-alphabet IID information AEP with a variance bound

Define $Y_i={\mathrm{ı}}_p(X_i)$ when $X_i\in A_{+}$ and $Y_i=0$ otherwise. This is the same finite-valued measurable function of every $X_i$, so its range is finite. Take arbitrary distinct indices $i_1,\dots ,i_m$ and Borel sets $B_1,\dots ,B_m$. Then ${\bigcap }_{j=1}^m\{Y_{i_j}\in B_j\}={\bigcap }_{j=1}^m\{X_{i_j}\in {\mathrm{ı}}_p^{-1}(B_j)\}$; mutual independence in [F6] gives probability ${\prod }_{j=1}^m\mathbb{P}(X_{i_j}\in {\mathrm{ı}}_p^{-1}(B_j))$, and the common law $p$ makes every $Y_i$ have the same law. Thus the $Y_i$ are IID.

Since $\mathbb{P}(X_i=a)=p(a)$, zero-mass symbols contribute nothing, and finite summation using [F4] gives $\mathbb{E}[Y_i]=H(p)$; the definition of $V(p)$ in [F2] gives $\mathrm{Var}(Y_i)=V(p)$. Finite range gives square integrability.

For each fixed $n$, each event $\{X_i\mathrm{\notin }{A}_{+}\}$ has probability zero because its probability is ${\sum }_{a\mathrm{\notin }{A}_{+}}p(a)=0$; the finite union also has probability zero. On its complement, [F1] and [F5] give $I_n(X^n)=n^{-1}{\sum }_{i=1}^n{Y}_i$. Applying [F7] with mean $H(p)$ and variance $V(p)$ proves the displayed tail bound. Its strict-tail version and [F8] give convergence in probability.