Finite-alphabet mean-field probability ensembles

Let $\mathcal{A}$ be a finite nonempty alphabet, let $p=(p(a){)}_{a\in \mathcal{A}}$ be a probability vector with $p(a)>0$ for every $a$, and write $\mathrm{\Delta }(\mathcal{A}):=\{q\in [0,1{]}^{\mathcal{A}}:{\sum }_{a\in \mathcal{A}}q(a)=1\}$ with the metric $d_1(q,r):={\sum }_{a\in \mathcal{A}}\mathrm{\mid }q(a)-r(a)\mathrm{\mid }$ and its induced topology, called the ${\mathrm{\ell }}^1$ topology. This is a metric: symmetry, nonnegativity and separation follow coordinatewise, and the triangle inequality follows by summing $\mathrm{\mid }q(a)-s(a)\mathrm{\mid }\le \mathrm{\mid }q(a)-r(a)\mathrm{\mid }+\mathrm{\mid }r(a)-s(a)\mathrm{\mid }$. For each integer $n\ge 1$, the configuration space is ${\mathrm{\Omega }}_n:={\mathcal{A}}^n$ with the full power-set $\sigma$-algebra. Its reference probability is the product law $p^{\otimes n}$, which gives the word $x^n=(x_1,\dots ,x_n)$ mass ${\prod }_{i=1}^{n}p(x_i)$. By The finite product law has iid coordinate maps, this mass assignment is normalized and is a probability law; explicitly, ${\sum }_{x^n\in {\mathcal{A}}^n}{\prod }_{i=1}^{n}p(x_i)={\prod }_{i=1}^n{\sum }_{a\in \mathcal{A}}p(a)=1$. The empirical macrostate is the probability vector $L_n(x^n)(a):=\frac{1}{n}\mathrm{\#}\{i\in \{1,\dots ,n\}:x_i=a\},a\in \mathcal{A}\mathrm{.}$ Thus $L_n$ is the empirical type defined by the finite-alphabet supplier.

For a continuous function $h:\mathrm{\Delta }(\mathcal{A})\to \mathbb{R}$ and a real parameter $\beta$, define the finite canonical normalizer and word law by $Z_{n,\beta }:={\sum }_{x^n\in {\mathrm{\Omega }}_n}{p}^{\otimes n}(x^n)e^{-n\beta h(L_n(x^n))},p_{n,\beta }(x^n):=\frac{p^{\otimes n}(x^n)e^{-n\beta h(L_n(x^n))}}{Z_{n,\beta }}\mathrm{.}$ The normalizer is finite and positive because the sum is finite and every summand is finite and positive. The canonical macrostate law is the probability measure on Borel subsets $B\subseteq \mathrm{\Delta }(\mathcal{A})$ that assigns mass $p_{n,\beta }(\{x^n\in {\mathrm{\Omega }}_n:L_n(x^n)\in B\})\mathrm{.}$

For a real-valued function $f$ on a nonempty set $S$, define ${\mathrm{argmin}}_{x\in S}f(x):=\{x\in S:f(x)={\inf }_{y\in S}f(y)\}$ and ${\mathrm{argmax}}_{x\in S}f(x):=\{x\in S:f(x)={\sup }_{y\in S}f(y)\}$; either set is empty if the corresponding extremum is not attained. For $q\in \mathrm{\Delta }(\mathcal{A})$, set $D(q\mathrm{\parallel }p):={\sum }_{a:q(a)>0}q(a)\ln \frac{q(a)}{p(a)},$ omitting zero-mass terms, as in the finite relative-entropy convention. For $u\in h(\mathrm{\Delta }(\mathcal{A}))$, define the constrained cost, entropy, canonical variational set, and constrained variational set by $d(u):={\inf }_{q\in \mathrm{\Delta }(\mathcal{A}):h(q)=u}D(q\mathrm{\parallel }p),s(u):=-d(u),$ ${\mathcal{M}}_{\beta }:={\mathrm{argmin}}_{q\in \mathrm{\Delta }(\mathcal{A})}[D(q\mathrm{\parallel }p)+\beta h(q)],{\mathcal{M}}_u:={\mathrm{argmin}}_{q\in \mathrm{\Delta }(\mathcal{A}):h(q)=u}D(q\mathrm{\parallel }p)\mathrm{.}$ For $A\subseteq {\mathrm{\Omega }}_n$ and an event $E\subseteq {\mathrm{\Omega }}_n$ with $p^{\otimes n}(E)>0$, define the conditional probability by $p^{\otimes n}(A\mid E):=p^{\otimes n}(A\cap E)\mathrm{/}{p}^{\otimes n}(E)$. For $\delta >0$, the microcanonical shell event is $E_{n,\delta ,u}:=\{x^n:\mathrm{\mid }h(L_n(x^n))-u\mathrm{\mid }<\delta \}$. Whenever $p^{\otimes n}(E_{n,\delta ,u})>0$, its microcanonical shell law is the conditional probability $p^{\otimes n}(\cdot \mid E_{n,\delta ,u})$.

Empirical types, n-types, and type classes

Finite product mass on words

The finite product law has iid coordinate maps

Finite distributions, entropy, divergence, and stochastic matrices