A nonstationary independent source with no entropy-rate limit

Partition the coordinates into consecutive blocks of lengths $L_j=2^{2^j}$ for $j\ge 1$. In odd-numbered blocks set $X_i=0$ deterministically; in even-numbered blocks let the $X_i$ be fair independent bits. Independence gives the block law $p_{0:N-1}(x_0^{N-1})={\prod }_{i=0}^{N-1}{p}_i(x_i)$. Expanding its finite entropy sum over positive-probability words and using ${\log }_2{\prod }_i{p}_i(x_i)={\sum }_i{\log }_2{p}_i(x_i)$ yields $H(X_0^{N-1})={\sum }_{i=0}^{N-1}H(X_i),$ since each other coordinate probability sums to one. Thus the entropy contribution is zero on odd blocks and one bit per coordinate on even blocks.

Let $N_j={\sum }_{r=1}^j{L}_r$ be the endpoint of block $j$. For $j\ge 2$, ${\sum }_{r<j}{L}_r\le (j-1)2^{2^{j-1}}$, so $\frac{L_j}{{\sum }_{r<j}{L}_r}\ge \frac{2^{2^{j-1}}}{j-1}⟶\mathrm{\infty }\mathrm{.}$ Thus the previous blocks contribute a vanishing fraction of $N_j$. Therefore $H(X_0^{N_j-1})\mathrm{/}{N}_j\to 0$ along odd $j$ and tends to $1$ along even $j$. The block entropy per symbol has no limit, so the process has no entropy rate in the sense of [F2].