Spatial Measure And Information

On the component of configurations with exactly $n$ points, $N_B(x_1,\dots ,x_n)={\sum }_{j=1}^n{\mathbf{1}}_B(x_j)$ is Borel for every Borel $B\subseteq C$. The count of $C$ identifies the component $n$, so every count map is Borel on the full disjoint union. Thus the count-generated sigma-algebra is contained in the sorted-list Borel sigma-algebra.

Each $I_{m,k}$ is Borel as the intersection of a closed ray and an open ray. Distinct interval interiors are disjoint and an adjacent endpoint belongs only to the interval beginning there. Induction on the number of adjacent intervals gives ${\bigcup }_{k<j}{I}_{m,k}=[0,j{2}^{-m})$; at $j=2^m$ this is $[0,1)$. Thus they partition the reference interval. Each interval is the union of its two intervals at the next level, so the partition refines and its mesh tends to zero. Cartesian products give the corresponding Borel partitions of $C$.

Let $(T,\mathcal{T})$ be any measurable space. For $H\in \mathcal{B}(C)\otimes \mathcal{T}$, define $M_H(\eta ,t)={\sum }_{x\in \eta }{\mathbf{1}}_H(x,t)$. The class of such $H$ for which $M_H$ is measurable with respect to the count sigma-algebra times $\mathcal{T}$ contains every rectangle $A\times D$, since $M_{A\times D}(\eta ,t)=N_A(\eta ){\mathbf{1}}_D(t)$. It is closed under relative complements because $M_{(C\times T)\setminus H}=N_C-M_H$, and under disjoint countable unions by pointwise convergence of finite partial sums. Rectangles form a generating pi-system, so [F2] proves measurability for every product-measurable $H$.

Fix a component with $n$ distinct points and an index $k\le n$. Recover the coordinates of the $k$th lexicographic point recursively. Initially the selected prefix is empty and its rank is $r=k$. At coordinate $j$, with measurable prefix $p=(a_1,\dots ,a_{j-1})$, use step 2.1 on the Borel set $H_j=\{(x,p,t):x_{\mathrm{\ell }}=p_{\mathrm{\ell }}(\mathrm{\ell }<j),x_j<t\}$, where $t\in [0,1]$. For each dyadic interval $I=[u,v)$ in coordinate $j$, the number of points matching the prefix and having $x_j\in I$ is $M_{H_j}(\eta ,p,v)-M_{H_j}(\eta ,p,u)$, hence is measurable. The interval containing the $r$th such coordinate is the first whose cumulative count reaches $r$; the nested interval closures have diameters tending to zero, so their left endpoints converge to that coordinate $a_j$. The count of matching points with $x_j<a_j$ is $M_{H_j}(\eta ,p,a_j)$, measurable by step 2.1 after composition with the measurable prefix and $a_j$. Subtracting it from $r$ gives the positive rank among points with the extended prefix.

The recursive choice in step 3.1 is the usual lexicographic ordering rule: among points with a fixed prefix, sort by the next coordinate, retain the value at the current rank, then subtract the points strictly below it to obtain the rank among ties. After coordinate $d$, distinctness leaves exactly one point, namely $x_k$. Thus all coordinate functions of every sorted point are count-measurable. The Borel sigma-algebra on the sorted-list component is generated by its coordinate maps, so it is contained in the count-generated sigma-algebra there.

The count $N_C$ partitions the full configuration space into its countable components. Applying step 4.1 on each component shows that every sorted-list Borel set is count-measurable; step 1.1 proves the reverse inclusion. The displayed sigma-algebra equality follows.

For $P\ll Q$ with density $h$, the negative part of $h\log h$ is integrable because $-u\log u\le 1\mathrm{/}e$ for $0\le u\le 1$; the density definition is therefore unambiguous. Also $u\log u\ge u-1$ for $u\ge 0$, so $D(P\mathrm{\parallel }Q)=\int h\log hdQ\ge \int (h-1)dQ=0$. If $P\ll ̸Q$, the defined value is $+\mathrm{\infty }\ge 0$.

For bounded $\varphi$, let $Q_{\varphi }$ have density $e^{\varphi }\mathrm{/}Q(e^{\varphi })$ relative to $Q$. When $P\ll Q$ and $D(P\mathrm{\parallel }Q)<\mathrm{\infty }$, the density logarithm identity gives $D(P\mathrm{\parallel }{Q}_{\varphi })=D(P\mathrm{\parallel }Q)-P(\varphi )+\log Q(e^{\varphi })\ge 0$ by step 1.1. Thus every variational test is at most $D(P\mathrm{\parallel }Q)$; the same upper bound is immediate if $D=+\mathrm{\infty }$. If $P\ll ̸Q$, choose a measurable $Q$-null set $N$ with $P(N)>0$ and use $\varphi =t{\mathbf{1}}_N$: then $Q(e^{\varphi })=1$ and $P(\varphi )=tP(N)\to \mathrm{\infty }$.

In the remaining case $P\ll Q$, let $h=dP\mathrm{/}dQ$ and set ${\varphi }_m=\max (-m,\min (m,\log h))$, taking ${\varphi }_m=-m$ on $\{h=0\}$. Then $e^{{\varphi }_m}\to h$ and $e^{{\varphi }_m}\le 1+h$, so [F4] gives $Q(e^{{\varphi }_m})\to 1$. The negative part of $h{\varphi }_m$ is bounded by $1\mathrm{/}e$, and its positive part increases to $(h\log h{)}^{+}$; hence [F4] gives $P({\varphi }_m)=\int h{\varphi }_{m}dQ\to D(P\mathrm{\parallel }Q)$, including when the positive entropy part is infinite. Combining this with step 2.1 proves the variational identity.

If $T$ is measurable, every bounded test $\psi$ on the target pulls back to the bounded test $\psi \circ T$ on the source, and $Q(e^{\psi \circ T})=(Q\circ T^{-1})(e^{\psi })$ while $P(\psi \circ T)=(P\circ T^{-1})(\psi )$. Taking suprema in step 3.1 proves the restriction inequality.

Finiteness of $D(P_{S,Y}\mathrm{\parallel }{P}_S\otimes Q_Y)$ gives a joint density and its conditional kernel by [F2]. Pushing forward by $(s,y)\mapsto (r(s),y)$ and using step 4.1 gives finite $D(P_{R,Y}\mathrm{\parallel }{P}_R\otimes Q_Y)$, so [F2] supplies the corresponding conditional kernel for $R$ as well. Thus all conditional densities and references in this clause are defined. For a conditioning observation $R$, applying the variational identity pointwise to the conditional density $k_R(r,y)$ shows $G(R)={\sup }_{\varphi \in B_b}(E_P\varphi (R,Y)-E_P\text{\hspace{0.17em}⁣}[\log \int e^{\varphi (R,y)}{Q}_Y(dy)]),$ where the tests range over bounded measurable functions of $(R,Y)$. The upper bound follows from step 2.1 for each conditional law; the reverse bound follows by using the jointly measurable bounded truncations of $\log k_R$ from step 3.1 and then [F3]–[F4]. When $R$ is a measurable function of $S$, every test depending on $(R,Y)$ is an admissible test depending on $(S,Y)$ with the same objective value, since the $Q_Y$ normalizer still depends only on $R$. Thus $G(R)\le G(S)$; nonnegativity is step 1.1 applied conditionally.

Suppose $P_{XY}$ has density $h(x,y)$ relative to $Q_X\otimes Q_Y$. Put $h_X(x)=\int h(x,y)Q_Y(dy)$. By [F3], $h_X$ is a density for $P_X$. Where $0<h_X(x)<\mathrm{\infty }$, [F2] supplies the conditional density $k(x,y)=h(x,y)\mathrm{/}{h}_X(x)$ relative to $Q_Y$; elsewhere use its supplied filler. On the full $P_X$-measure set of finite positive $h_X$, $h=h_{X}k$ and $\log h=\log h_X+\log k$ $P_{XY}$-almost everywhere. Integrating this identity, with nonnegative conditional entropies and finite total entropy, gives the displayed chain rule. If a restriction entropy is infinite, step 4.1 makes the joint entropy infinite as well, so no subtraction of two infinite terms is used.

Steps 1.1, 2.1 and 3.1 prove nonnegativity and the variational identity, step 4.1 proves restriction monotonicity, step 5.2 proves the product chain rule, and step 5.1 proves monotonicity under additional conditioning.

Fix one unit cell $C_i$ and a finite Borel partition $B_0,\dots ,B_s$ of it. Put $v_j=\mathrm{\mid }{B}_j\mathrm{\mid }$, so ${\sum }_j{v}_j=1$. Conditional on the cell count $N=n$, the $n$ independent uniform points give category-count probability $n!{\prod }_j{v}_j^{n_j}\mathrm{/}{\prod }_j{n}_j!$ for ${\sum }_j{n}_j=n$. Multiplying by the Poisson count probability $e^{-\lambda }{\lambda }^n\mathrm{/}n!$ gives $e^{-\lambda }{\prod }_j(\lambda v_j{)}^{n_j}\mathrm{/}{n}_j!={\prod }_j{e}^{-\lambda v_j}(\lambda v_j{)}^{n_j}\mathrm{/}{n}_j!$, by [F3]. This factors into the Poisson probabilities of the disjoint cell pieces, including the complement $B_0$; hence their counts are independent Poisson variables with respective means $\lambda v_j$. Zero-volume pieces have count zero almost surely by the same formula.

Each bounded $A_j$ intersects only finitely many unit cells. Apply step 1.1 to the partition formed by the sets $A_j\cap C_i$ and their complement in each such cell. Independence within each cell and independence between cell configurations show that all component counts, indexed by both $j$ and $i$, are independent Poisson variables.

For fixed $j$, sum the component counts over its finitely many cells. The sum of independent Poisson variables of means ${\mu }_1,\dots ,{\mu }_q$ has mass at $n$ equal to $e^{-\sum {\mu }_i}(\sum {\mu }_i{)}^n\mathrm{/}n!$: expand $({\sum }_i{\mu }_i{)}^n$ by [F2], then use the product of the component masses. Thus $N_{A_j}$ is Poisson with mean $\lambda {\sum }_i\mathrm{\mid }{A}_j\cap C_i\mathrm{\mid }=\lambda \mathrm{\mid }{A}_j\mathrm{\mid }$. The groups of components for different $j$ are disjoint independent families, so their sums remain independent.

Put $\theta =\lambda \mathrm{\mid }A\mathrm{\mid }$. The count mass in step 3.1 gives, by nonnegative partial sums and [F5], $E_{Q_{\lambda }}{e}^{a{N}_A}={\sum }_{n=0}^{\mathrm{\infty }}{e}^{an}{e}^{-\theta }\frac{{\theta }^n}{n!}=e^{-\theta }{\sum }_{n=0}^{\mathrm{\infty }}\frac{(\theta e^a{)}^n}{n!}=\exp \text{\hspace{0.17em}⁣}(\theta (e^a-1))\mathrm{.}$ For $\theta =0$, the count is zero almost surely and both sides are one, so no ambiguous zero power is needed. This establishes the exponential-moment interface before entropy estimates use it.

Let $A_1,\dots ,A_r$ be pairwise disjoint and, for each $j$, consider a finite count-cylinder event in the restriction to $A_j$. Refine the finitely many counting sets in each event into their disjoint atoms inside $A_j$. Step 3.1 gives independent Poisson counts on all these atoms across every $j$; the event for each restricted configuration is a measurable function only of its own atom counts. Thus finite cylinder events factor across $j$. The restriction sigma-fields are generated by these cylinders, so repeated pi-lambda extension proves independence of the restrictions themselves.

For a finite family of bounded Borel sets $A_1,\dots ,A_s$, partition their union into the finitely many disjoint Boolean atoms formed by the sets and their complements. Every $N_{A_j}$ is a sum of the counts on these atoms. Translation preserves each atom's volume, so step 3.1 gives the same joint distribution of this whole count vector under $Q_{\lambda }$ and its translate. Thus the laws agree on every finite count cylinder. These cylinders generate the configuration sigma-algebra by [F4], and pi-lambda uniqueness identifies the laws. The one-set count law gives $E_{Q_{\lambda }}{N}_A=\lambda \mathrm{\mid }A\mathrm{\mid }$ for bounded $A$. For arbitrary Borel $A$, increase bounded cubes to all space and apply [F5] to the counting measures and Lebesgue measure to obtain the same extended identity. For disjoint Borel $A_n$, the pointwise counting identity $N_{{\cup }_n{A}_n}={\sum }_n{N}_{A_n}$ and [F5] show countable additivity of ${\mathrm{\Lambda }}_{Q_{\lambda }}$, while the empty set has expectation zero. Thus the defined intensity functional is a measure and equals $\lambda$ times Lebesgue measure.

If $G_1,G_2$ are disjoint bounded Borel sets, [F1] decomposes $D(P_{G_1\cup G_2}\mathrm{\parallel }{Q}_{G_1}\otimes Q_{G_2})$ into $D(P_{G_1}\mathrm{\parallel }{Q}_{G_1})$ plus the mean conditional relative entropy of $G_2$ given $G_1$. That conditional increment is at least $D(P_{G_2}\mathrm{\parallel }{Q}_{G_2})$ by the conditioning monotonicity in [F1], comparing observation by $G_1$ with no observation. Thus relative entropy is superadditive over disjoint regions; if the left side is infinite the inequality is immediate, and otherwise all terms in the chain rule are finite.

Fix an integer $m\ge 1$ and let $q_N=\lfloor N\mathrm{/}m\rfloor$. Tile $[0,q_{N}m{)}^d$ by $q_N^d$ disjoint translates of ${\mathrm{\Lambda }}_m$, leaving the Borel remainder ${\mathrm{\Lambda }}_N\setminus [0,q_{N}m{)}^d$. Iterating step 1.1 and using nonnegativity on the remainder gives $D(P_{{\mathrm{\Lambda }}_N}\mathrm{\parallel }{Q}_{{\mathrm{\Lambda }}_N})\ge q_N^{d}D(P_{{\mathrm{\Lambda }}_m}\mathrm{\parallel }{Q}_{{\mathrm{\Lambda }}_m}),$ because stationarity gives the same entropy for each translated $m$-cube. If $D(P_{{\mathrm{\Lambda }}_m}\mathrm{\parallel }{Q}_{{\mathrm{\Lambda }}_m})=+\mathrm{\infty }$, this forces $D(P_{{\mathrm{\Lambda }}_N}\mathrm{\parallel }{Q}_{{\mathrm{\Lambda }}_N})=+\mathrm{\infty }$ for every $N\ge m$.

Put $s={\sup }_{m\ge 1}{m}^{-d}D(P_{{\mathrm{\Lambda }}_m}\mathrm{\parallel }{Q}_{{\mathrm{\Lambda }}_m})$. If $s=+\mathrm{\infty }$, step 2.1 makes the liminf of $N^{-d}{D}_N$ exceed every finite bound, so the integer-cube limit is $+\mathrm{\infty }=s$. If $s<+\mathrm{\infty }$, step 2.1 gives ${\mathrm{lim\hspace{0.17em}inf}}_{N\to \mathrm{\infty }}{N}^{-d}{D}_N\ge m^{-d}D(P_{{\mathrm{\Lambda }}_m}\mathrm{\parallel }{Q}_{{\mathrm{\Lambda }}_m})$ for every $m$, since $q_N^d\mathrm{/}{N}^d\to m^{-d}$. Taking the supremum gives liminf at least $s$, while the definition of $s$ gives $N^{-d}{D}_N\le s$ for every integer $N$. Hence the integer-cube limit exists and equals $s$.

For real $L\to \mathrm{\infty }$, let $N=\lfloor L\rfloor$. The inclusions ${\mathrm{\Lambda }}_N\subseteq {\mathrm{\Lambda }}_L\subseteq {\mathrm{\Lambda }}_{N+1}$ and [F3] give $D_N\le D_L\le D_{N+1}$. Since $N\mathrm{/}L\to 1$ and $(N+1)\mathrm{/}L\to 1$, step 3.1 squeezes $L^{-d}{D}_L$ to the same extended limit $s$; if some integer-cube entropy is infinite, monotonicity gives the same conclusion directly.

If $I(P)<\mathrm{\infty }$ and $A$ is bounded, choose an integer cube ${\mathrm{\Lambda }}_N+x$ containing $A$. Stationarity [F2] and restriction monotonicity [F3] give $D(P_A\mathrm{\parallel }{Q}_A)\le D(P_{{\mathrm{\Lambda }}_N+x}\mathrm{\parallel }{Q}_{{\mathrm{\Lambda }}_N+x})=D(P_{{\mathrm{\Lambda }}_N}\mathrm{\parallel }{Q}_{{\mathrm{\Lambda }}_N})\le N^{d}I(P)<\mathrm{\infty }\mathrm{.}$ Thus every bounded restriction has finite relative entropy.

Let $A\subset F_{+}$ be finite. By [F1], $D(P_{A\cup C}\mathrm{\parallel }{Q}_{A\cup C})$ is finite, so the joint density $h_{A,C}$ relative to $Q_A\otimes Q_C$ exists. The marginal density $h_A$ is its $Q_C$ integral. On $0<h_A<\mathrm{\infty }$, [F2] gives the conditional density $k_A=h_{A,C}\mathrm{/}{h}_A$ relative to $Q_C$; the exceptional fibers have $P_A$-measure zero. The chain rule in [F2] gives $g(A)=E_{P}D(P_{C\mathrm{\mid }A}\mathrm{\parallel }{Q}_C)=D(P_{A\cup C}\mathrm{\parallel }{Q}_{A\cup C})-D(P_A\mathrm{\parallel }{Q}_A)$, so $g(A)$ is finite and nonnegative.

If $A\subseteq B\subset F_{+}$ are finite, apply the conditional-increment monotonicity in [F2] to observation of the cell configuration on $A$ and then on $B$. It yields $g(A)\le g(B)$.

To identify the supremum, let ${\mathrm{\Lambda }}_N=[0,N{)}^d$, the union of $N^d$ unit cells, and order those cells lexicographically. Repeated use of the conditional chain rule decomposes $D(P_{{\mathrm{\Lambda }}_N}\mathrm{\parallel }{Q}_{{\mathrm{\Lambda }}_N})$ as the sum of $g(B_i)$ over its cells $i$, where $B_i$ is the finite set of earlier cells in ${\mathrm{\Lambda }}_N$. Each $B_i$ is a finite subset of $F_{+}$ after translating cell $i$ to $C$, so $g(B_i)\le {\sup }_{A}g(A)$ by stationarity and step 2.1. Hence $D(P_{{\mathrm{\Lambda }}_N}\mathrm{\parallel }{Q}_{{\mathrm{\Lambda }}_N})\mathrm{/}{N}^d\le {\sup }_{A}g(A)$. Conversely, fix finite $A\subset F_{+}$. For every cell $i$ in the inner cube whose distance from the boundary exceeds the largest coordinate magnitude in $A$, the translated set $i+A$ lies in $B_i$. By step 2.1 and stationarity, $g(B_i)\ge g(A)$. The proportion of these interior cells tends to one, and all other increments are nonnegative by step 1.1. Taking $N\to \mathrm{\infty }$ and applying [F1] gives $\gamma \ge g(A)$. Taking the supremum over $A$ proves ${\sup }_{A}g(A)=\gamma$.

Choose an increasing sequence of finite pasts $A_m$ exhausting $F_{+}$ and set $R=P_{F_{+}}\otimes Q_C$. Lift each $k_{A_m}$ to the common space $({\mathrm{\Omega }}_{F_{+}}\times E_C,R)$. Consistency of the conditional densities shows $E_R[k_{A_{m+1}}\mid {\mathcal{G}}_m]=k_{A_m}$, where ${\mathcal{G}}_m$ is generated by $A_m$ and $C$; also $E_R{k}_{A_m}=1$. By step 3.1, $E_R[k_{A_m}\log k_{A_m}]=g(A_m)\le \gamma$. For $K>1$, splitting at $k_{A_m}=1$ and using $-u\log u\le 1\mathrm{/}e$ gives ${\int }_{\{k_{A_m}>K\}}{k}_{A_m}dR\le \frac{\gamma +1\mathrm{/}e}{\log K}\mathrm{.}$ Thus these martingale densities are uniformly integrable. By [F3] they converge almost surely and in $L^1(R)$ to $k_{\mathrm{\infty }}\ge 0$ with $E_R{k}_{\mathrm{\infty }}=1$.

For any finite-past cylinder event and any event in $C$, integrate $k_{\mathrm{\infty }}$ over their product. $L^1(R)$ convergence makes this equal the limit of the same integral against $k_{A_m}$, which is the corresponding joint probability under $P$. These rectangles form a generating pi-system, so [F6] identifies $k_{\mathrm{\infty }}R$ with $P_{F_{+}\cup C}$. Therefore $k_A=E_R[k_{\mathrm{\infty }}\mid {\mathcal{G}}_A]$ for every finite $A\subset F_{+}$. Fatou applied to $u\log u+1\mathrm{/}e\ge 0$ gives $D(P_{F_{+}\cup C}\mathrm{\parallel }{P}_{F_{+}}\otimes Q_C)\le {\mathrm{lim\hspace{0.17em}inf}}_{m}g(A_m)=\gamma$. Conversely, its variational supremum contains all finite-past tests, so step 3.1 gives the reverse inequality. Hence $E_P\log k_{\mathrm{\infty }}=\gamma$.

The $L^1(R)$ convergence of $k_{A_m}$ implies convergence in $R$-probability. Since the full joint law has density $k_{\mathrm{\infty }}$ relative to $R$, absolute continuity transfers this to convergence in $P$-probability. Also $k_{\mathrm{\infty }}>0$ $P$-almost surely, so continuity of $\log$ on $(0,\mathrm{\infty })$ gives $\log k_{A_m}\to \log k_{\mathrm{\infty }}$ in $P$-probability. For $t\ge 1$, $E_P\text{\hspace{0.17em}⁣}[(-\log k_A{)}^{+}{\mathbf{1}}_{\{(-\log k_A{)}^{+}>t\}}]={\int }_{\{0<k_A<e^{-t}\}}{k}_A(-\log k_A)dR\le t{e}^{-t}\mathrm{.}$ Thus the negative parts are uniformly integrable; [F4] gives their $L^1(P)$ convergence.

Since $E_P\log k_{A_m}=g(A_m)\to \gamma =E_P\log k_{\mathrm{\infty }}$ and the negative parts converge in $L^1$, the expectations of the positive parts converge as well. These positive parts converge in probability. For nonnegative variables $Y_m\to Y$ in probability with $E{Y}_m\to EY<\mathrm{\infty }$, bounded truncations satisfy $E(Y_m\wedge K)\to E(Y\wedge K)$: for any $\eta >0$ the difference is at most $\eta +KP(\mathrm{\mid }{Y}_m-Y\mathrm{\mid }>\eta )$. Consequently $E(Y_m-K{)}^{+}\to E(Y-K{)}^{+}$; choosing $K$ large and then bounding $Y_m{\mathbf{1}}_{Y_m>2K}\le 2(Y_m-K{)}^{+}$ proves uniform integrability, including the finite initial segment. Applying [F4] gives $L^1$ convergence of the positive parts. Together with the negative parts this proves $\log k_{A_m}\to \log k_{\mathrm{\infty }}$ in $L^1(P)$.

For any finite $B\supseteq A_m$, the tower property gives $k_B=E_R[k_{\mathrm{\infty }}\mid {\mathcal{G}}_B]$ and $k_{A_m}=E_R[k_{\mathrm{\infty }}\mid {\mathcal{G}}_m]$. Conditional expectation is an $L^1$ contraction, so ${\sup }_{B\supseteq A_m}\mathrm{\parallel }{k}_B-k_{\mathrm{\infty }}{\mathrm{\parallel }}_{L^1(R)}\le 2\mathrm{\parallel }{k}_{A_m}-k_{\mathrm{\infty }}{\mathrm{\parallel }}_{L^1(R)}⟶0.$ Absolute continuity of $P=k_{\mathrm{\infty }}R$ transfers this uniform probability convergence to $P$. The negative-part tail bound in step 6.1 is uniform over all finite $B$, while $g(A_m)\le g(B)\le \gamma$ by steps 2.1–3.1. The truncation argument in step 7.1 therefore applies to every sequence $B_m\supseteq A_m$ with $m\to \mathrm{\infty }$; if the asserted directed $L^1$ convergence were not uniform, choosing a contradicting sequence $B_m$ would violate that argument. This proves the directed form.