Finite strongly typical sets with support restriction

Let $\mathcal{A}$ be a finite nonempty alphabet, let $P$ be a probability vector on $\mathcal{A}$ in the sense of Finite distributions, entropy, divergence, and stochastic matrices, and let $n\ge 1$. Write $\mathrm{supp}(P):=\{a\in \mathcal{A}:P(a)>0\}$. For $a^n=(a_1,\dots ,a_n)\in {\mathcal{A}}^n$, its empirical distribution is ${\widehat{P}}_{a^n}(a):=n^{-1}\mathrm{\mid }\{t:a_t=a\}\mathrm{\mid }$. For $\delta >0$, define the support-restricted strongly typical set by $T_{\delta }^{(n)}(P):=\{a^n\in \mathrm{supp}(P{)}^n:\mathrm{\mid }{\widehat{P}}_{a^n}(a)-P(a)\mathrm{\mid }\le \delta P(a)\text{ for every }a\text{ with }P(a)>0\}\mathrm{.}$ For a finite joint law $P_{XA}$ and a fixed $x^n$, write $T_{\delta }^{(n)}(P_{A\mathrm{\mid }X}\mid x^n):=\{a^n:(x^n,a^n)\in T_{\delta }^{(n)}(P_{XA})\}$. This conditional set is empty if $x^n$ is outside the support of $P_X$. Zero-probability letters are excluded exactly, so no logarithm of zero occurs in likelihood calculations on these sets.

The relative tolerance is used to keep support and likelihood ratios explicit. Each finite alphabet gives finitely many empirical distributions, and every sequence in $T_{\delta }^{(n)}(P)$ has its empirical frequencies within a stated multiplicative tolerance of $P$.