A finite deterministic copy has mutual information equal to entropy

On $\{0,1\}$, define $d(x,y)=0$ if $x=y$ and $d(x,y)=1$ otherwise; distinct endpoints force at least one of the two intermediate distances to be 1, so the triangle inequality holds. For a Cauchy sequence, tolerance $1\mathrm{/}2$ forces all sufficiently late terms to be equal, so the metric is complete. The finite set itself is a finite dense subset, and radius-$1\mathrm{/}2$ balls are singletons, so the topology is discrete. By the definitions in [F4], this is a Polish presentation of the discrete sigma-algebra, which is therefore standard Borel. If $p=0$ or $p=1$, X and Y are the same constant almost surely; their joint law and the product of their marginals are the same point mass. Thus [F1] gives a finite-alphabet mutual information, [F2] gives $I(X;Y)=0$, and [F3] gives the same entropy.

Suppose $0<p<1$. The joint law of $(X,Y)$ has masses $p$ at $(1,1)$ and $1-p$ at $(0,0)$, and zero elsewhere. The product law has masses $p^2$ at $(1,1)$ and $(1-p{)}^2$ at $(0,0)$. By [F1]–[F2], $I(X;Y)=p\ln \frac{p}{p^2}+(1-p)\ln \frac{1-p}{(1-p{)}^2}=-p\ln p-(1-p)\ln (1-p)\mathrm{.}$ The last quantity is $H(X)$ by [F3].

The endpoint cases in step 1.1 and the interior case in step 2.1 exhaust $p\in [0,1]$ and establish the claimed identity throughout.