Multiuser Channel Networks

The integrability hypothesis makes $\int f\ln f$ a finite real number, so the displayed negative integral is a defined real number.

In the conditional definition, almost-everywhere existence of the conditional densities and integrability of their entropy values make the outer expectation a finite real number; changing a conditional density on a $P_S$-null set does not change it.

For each finite $A>0$, apply [F2] on the bounded interval $[-A\mathrm{/}\sqrt{2N},A\mathrm{/}\sqrt{2N}]$ to the linear substitution $x=\sqrt{2N}t$. It gives ${\int }_{-A}^A(2\pi N{)}^{-1\mathrm{/}2}{e}^{-x^2\mathrm{/}(2N)}dx=\frac{1}{\sqrt{\pi }}{\int }_{-A\mathrm{/}\sqrt{2N}}^{A\mathrm{/}\sqrt{2N}}{e}^{-t^2}dt\mathrm{.}$ As $A\to \mathrm{\infty }$, the right side tends to $\sqrt{\pi }\mathrm{/}\sqrt{\pi }=1$ by the improper-integral statement [F1]; hence the Gaussian density has total mass one.

Tonelli's theorem [F4] gives the product density ${\prod }_{i=1}^n(2\pi N{)}^{-1\mathrm{/}2}{e}^{-z_i^2\mathrm{/}(2N)}$ total mass one; its coordinates are independent and have the required Gaussian law.

The finite message sets, deterministic encoders, Borel decoder and channel law define a joint distribution and a measurable decoder error event; the displayed expected costs and error probability are therefore well-defined, and the closure convention specifies the operational region.

For each $i$, the domain $[M_j]\times {\mathbb{R}}^{i-1}$ records exactly user $j$'s message and the common output history strictly before time $i$, so the maps are causal and use no future output; Borel measurability makes each input a measurable function of that history.

Given a message pair and a past-output history, the Borel encoder maps determine the next inputs and the Gaussian channel supplies a measurable probability kernel. Iterating these kernels for the finite blocklength defines a normalized joint law; the Borel decoder makes the error event measurable, so its average error and the expected powers are well-defined as in the no-feedback model.

The convolution $f_Y(y)=\mathbb{E}[(2\pi N{)}^{-d\mathrm{/}2}{e}^{-\mathrm{\parallel }y-X{\mathrm{\parallel }}^2\mathrm{/}(2N)}]$ is measurable and strictly positive. Tonelli's theorem [F3] gives integral one. Direct Gaussian integration gives $\mathbb{E}\mathrm{\parallel }Z{\mathrm{\parallel }}^2=dN$, hence $\mathbb{E}\mathrm{\parallel }Y{\mathrm{\parallel }}^2<\mathrm{\infty }$.

Choose $K$ with $p_0:=\mathrm{Pr}(\mathrm{\parallel }X\mathrm{\parallel }\le K)>0$, which exists because the increasing union of the finite-radius balls is ${\mathbb{R}}^d$. For every $y$, $f_Y(y)\le (2\pi N{)}^{-d\mathrm{/}2}$ and $f_Y(y)\ge p_0(2\pi N{)}^{-d\mathrm{/}2}{e}^{-(\mathrm{\parallel }y\mathrm{\parallel }+K{)}^2\mathrm{/}(2N)}$. Thus $\mathrm{\mid }\ln f_Y(y)\mathrm{\mid }\le C(1+\mathrm{\parallel }y{\mathrm{\parallel }}^2)$ for a finite constant $C$; [F1] and step 1.1 give $\int f_Y\mathrm{\mid }\ln f_Y\mathrm{\mid }<\mathrm{\infty }$. The same argument applies to every coordinate marginal and, for almost every $s$, to the conditional law $P_{X\mathrm{\mid }S=s}$ using a radius $K_s$ with positive conditional mass.

For any coordinate density $f_{Y_k}$, let $\mu =\mathbb{E}{Y}_k$, $v=\mathrm{Var}{Y}_k>0$, and let ${\varphi }_{\mu ,v}$ be the matching Gaussian density. The inequality $t\ln t-t+1\ge 0$ for $t\ge 0$ follows by differentiation and its minimum at $1$; integrating it against ${\varphi }_{\mu ,v}$ gives $\int f_{Y_k}\ln (f_{Y_k}\mathrm{/}{\varphi }_{\mu ,v})\ge 0$. Step 2.1 gives integrability of the entropy term, and the second moment gives integrability of the Gaussian log density. Expanding that log density yields $h(Y_k)\le \frac{1}{2}\ln (2\pi ev)$.

Apply the same inequality to the ratio of the joint density of $Y$ to the product of its coordinate marginals. Step 2.1 makes every log term integrable, and expansion gives $h(Y)\le {\sum }_{k}h(Y_k)$.

Under the reference law $P_X(dx)f_Y(y)dy$, the ratio of the joint law of $(X,Y)$ to that reference is ${\varphi }_N^{\otimes d}(y-x)\mathrm{/}{f}_Y(y)$. Its mean is one; integrating $t\ln t-t+1\ge 0$ against the reference gives a nonnegative integral. The logarithm is integrable by step 2.1 and the finite Gaussian second moment. Expanding it gives $h(Y)-h(Z)\ge 0$, and direct integration of ${\varphi }_N$ gives $h(Z)=\frac{d}{2}\ln (2\pi eN)$. Steps 2.1, 3.1 and 3.2 prove the unconditional claims.

Independence of $Z$ from $(X,S)$ makes the conditional law of $Y$ given $S=s$ the convolution of $P_{X\mathrm{\mid }S=s}$ with the same $N(0,N{I}_d)$ density. For almost every $s$, repeat steps 1.1 and 2.1 with $P_{X\mathrm{\mid }S=s}$; applying steps 3.1 and 3.2 coordinatewise gives $h(Y\mathrm{\mid }S=s)\le {\sum }_{k=1}^d\frac{1}{2}\ln \text{\hspace{0.17em}⁣}(2\pi e[N+\mathrm{Var}(X_k\mathrm{\mid }S=s)])\mathrm{.}$ The conditional version of step 4.1 also gives $h(Y\mathrm{\mid }S=s)\ge h(Z)$, so these bounds and step 2.1 make the averaged conditional entropy finite. Averaging the upper bound and applying concavity of $\ln$ to each coordinate separately yields the displayed conditional inequality; each $\mathbb{E}\mathrm{Var}(X_k\mathrm{\mid }S)$ is finite because it is at most $\mathbb{E}{X}_k^2$.

For every $x$, summing $W_1(y_1\mid x)W_2(y_2\mid y_1)$ over $y_1,y_2$ gives ${\sum }_{y_1}{W}_1(y_1\mid x)=1$ by [F1], so the displayed joint law is a stochastic kernel.

The factorization makes the conditional law of $Y_2$ given $(X,Y_1)$ equal to $W_2(\cdot \mid Y_1)$; hence $X$ and $Y_2$ are conditionally independent given $Y_1$, which is exactly $X\to Y_1\to Y_2$.

For each input pair, $W(\cdot \mid x_1,x_2)$ is a probability vector by [F1].

Summing the product law over ${\mathcal{Y}}^n$ factors into $n$ row sums, each equal to one; therefore $W^n$ is a stochastic kernel and the block law is well-defined.

Write $K=\left(\begin{array}{cc}{P}_1& c\\ c& P_2\end{array}\right)$ with $c=\rho \sqrt{P_1{P}_2}$, $a=(1,1{)}^T$, and $D=N+a^{T}Ka$. Set $R=X-(Ka\mathrm{/}D)Y$. Then $\mathrm{Cov}(R,Y)=Ka-(Ka\mathrm{/}D)D=0$. Since $(R,Y)$ is jointly Gaussian, this block-diagonal covariance makes its joint characteristic function factor, so $R$ and $Y$ are independent. Thus $\mathbb{E}[X\mid Y]=KaY\mathrm{/}D$ and the residual covariance is $\mathrm{Cov}(R)=K-Ka{a}^{T}K\mathrm{/}D$.

Since $Ka=(P_1+c,P_2+c{)}^T$ and $N+a^{T}Ka=D$, subtracting the rank-one matrix gives diagonal entries $P_1[N+P_2(1-{\rho }^2)]\mathrm{/}D$, $P_2[N+P_1(1-{\rho }^2)]\mathrm{/}D$, and off-diagonal entry $[cN+c^2-P_1{P}_2]\mathrm{/}D$.

Dividing the off-diagonal covariance by the square root of the two diagonal variances and changing its sign yields the displayed ${\rho }_{\mathrm{n}\mathrm{e}\mathrm{w}}$; all denominators are positive because $N>0$.

Recursively in $i$, the message and past-output history determine $X_i=f_i(M,Y^{i-1})$, after which [F1] specifies a probability vector for $Y_i$. Multiplying these causal row probabilities and the uniform message mass $1\mathrm{/}M$ gives the displayed joint law. Summing first over $y_n$, then $y_{n-1}$ down to $y_1$, each last row sums to one; summing over the $M$ messages then gives total mass one. Thus the law is normalized, and its positive-probability history conditionals are the stated rows.

Hence the decoder error, rate and capacity convention are well-defined for every such finite code.

The single encoder fixes the input word for each message pair; the finite product law in [F1] then defines both receiver output sequences jointly.

Both decoder error events are measurable subsets of finite output alphabets, and their union is the stated joint error; the rate and closure conventions are therefore well-defined.

The independent uniform messages and the two deterministic maps induce separate input words; combining these with [F1] defines a probability law on the finite set of message, input and output tuples.

Since each input coordinate is a function only of its own independent message, $X_{1i}$ and $X_{2i}$ are independent for $1\le i\le n$. Because $n\ge 1$ and $M_j\ge 1$, each rate $n^{-1}\ln M_j$ is a finite nonnegative real number. The decoder event belongs to the finite probability space in step 1.1, so its probability is well-defined.

Given $({\mathsf{M}}_1,{\mathsf{M}}_2,Y^{i-1})$, both causal maps determine $(X_{1i},X_{2i})$, and [F1] supplies a probability vector for $Y_i$; recursion over $i$ therefore gives a normalized joint law.

Since $n\ge 1$ and $M_j\ge 1$, each rate $n^{-1}\ln M_j$ is a finite nonnegative real number. The joint-error event belongs to the finite probability space in step 1.1, so its probability is well-defined; the causal maps add no other random input to the model.

If ${\pi }_s(m)=\mathrm{Pr}[M=m\mid S=s]$ and $f_{m,s}$ is the conditional density of $Y$, then $f_s(y)={\sum }_m{\pi }_s(m)f_{m,s}(y)$ and $p_{M\mathrm{\mid }y,s}(m)={\pi }_s(m)f_{m,s}(y)\mathrm{/}{f}_s(y)$ whenever $f_s(y)>0$. This is a finite posterior probability vector, so [F1] makes its entropy well-defined and bounded by $\ln m$.

Put $E=\mathbf{1}\{\widehat{M}\mathrm{\ne }M\}$. For each fixed $(y,s)$, $E$ is a function of $M$; expanding the finite entropy sum first over $M$ and then over $E$ gives $H(M\mid y,s)=H(E\mid y,s)+H(M\mid E,y,s)$.

Let $q(y,s)=\mathrm{Pr}[E=1\mid Y=y,S=s]$. If $0<ϵ<1$, [F2] applied to the two-point laws $(q,1-q)$ and $(ϵ,1-ϵ)$ gives $q\ln (q\mathrm{/}ϵ)+(1-q)\ln ((1-q)\mathrm{/}(1-ϵ))\ge 0$. Rearranging and averaging, using $\mathbb{E}q=ϵ$, yields $\mathbb{E}{h}_b(q)\le h_b(ϵ)$. If $ϵ=0$ or $1$, then $q=ϵ$ almost surely and both binary entropies are zero.

On $E=0$, $M=\varphi (Y,S)$ and so its conditional entropy is zero; on $E=1$ and fixed $(Y,S)$, at most $m-1$ message values remain, so [F2] bounds that entropy by $\ln (m-1)$. Therefore $H(M\mid E,Y,S)\le ϵ\ln (m-1)$.

Averaging the identity of step 2.1 and applying steps 3.1 and 3.2 proves the sharp bound. Since $h_b(ϵ)\le \ln 2$ and $\ln (m-1)\le \ln m$, the weaker displayed bound follows. If $m=1$, $M$ is constant.

Put $a(r)=1-r^2$ and $F(r)=(1+s_{1}a(r))(1+s_{2}a(r))-(1+s_1+s_2+2r\sqrt{s_1{s}_2})$. Then $F(0)=s_1{s}_2>0$, $F(1)<0$, and $F^{\mathrm{\prime }}(r)=-2r[s_1(1+s_{2}a(r))+s_2(1+s_{1}a(r))]-2\sqrt{s_1{s}_2}<0.$ Continuity and strict decrease give a unique root ${\rho }_{\star }\in (0,1)$. Taking logarithms of its equation proves the rate-sum identity.

Choose any positive rates $0<R_j<\frac{1}{2}\ln (1+s_{j}a({\rho }_{\star }))$. By continuity of the unique root and rate expressions in $(P_1,P_2)$, choose temporary powers $0<P_j^{\mathrm{\prime }}<P_j$ close enough to $P_j$ that these strict inequalities still hold for $s_j^{\mathrm{\prime }}=P_j^{\mathrm{\prime }}\mathrm{/}N$ and their root ${\rho }^{\mathrm{\prime }}$. Continuity follows because every convergent subsequence of roots solves the limiting strictly decreasing equation, whose root is unique. Write ${\tilde{Z}}_i=Z_i\mathrm{/}\sqrt{N}$ and ${\tilde{Y}}_i=Y_i\mathrm{/}\sqrt{N}$, so ${\tilde{Y}}_i=V_{1i}+V_{2i}+{\tilde{Z}}_i$ and $\mathrm{Var}{\tilde{Z}}_i=1$. Use midpoint grids ${\theta }_j(m)=-1+(2m-1)\mathrm{/}{M}_j$ in $[-1,1]$, $M_j=\lceil e^{n{R}_j}\rceil$. The three startup transmissions in normalized units are $(V_{1,-2},V_{2,-2})=(0,{\theta }_2)$, $(V_{1,-1},V_{2,-1})=({\theta }_1,0)$ and $(V_{1,0},V_{2,0})=(0,0)$. User 1 learns ${\tilde{Z}}_{-1}$ by subtracting its own point from ${\tilde{Y}}_{-1}$; user 2 similarly learns ${\tilde{Z}}_{-2}$, and both learn ${\tilde{Z}}_0={\tilde{Y}}_0$ by feedback.

Set ${\lambda }^2={\rho }^{\mathrm{\prime }}\mathrm{/}(1-{\rho }^{\mathrm{\prime }})$, $U_1={\tilde{Z}}_{-1}+\lambda {\tilde{Z}}_0$, $U_2={\tilde{Z}}_{-2}+\lambda {\tilde{Z}}_0$, and $V_{j1}=\sqrt{s_j^{\mathrm{\prime }}\mathrm{/}(1+{\lambda }^2)}{U}_j$. Then both $U_j$ are known to their respective senders, have variance $1+{\lambda }^2$, and correlation ${\rho }^{\mathrm{\prime }}$, so $V_{j1}$ has variance $s_j^{\mathrm{\prime }}$ and the pair has correlation ${\rho }^{\mathrm{\prime }}$. For $i\ge 2$ define the causal innovations $E_{j,i-1}=V_{j,i-1}-\mathbb{E}[V_{j,i-1}\mid {\tilde{Y}}_1^{i-1}]$ using only the recursive outputs, and transmit $V_{1i}=\sqrt{s_1^{\mathrm{\prime }}\mathrm{/}\mathbb{E}{E}_{1,i-1}^2}{E}_{1,i-1}$ and $V_{2i}=-\sqrt{s_2^{\mathrm{\prime }}\mathrm{/}\mathbb{E}{E}_{2,i-1}^2}{E}_{2,i-1}$. Gaussian conditional means are affine functions of the observed histories, so these finite-message encoder maps are Borel-measurable; they are causal because the normalized history is an invertible rescaling of the physical output history.

The initial pair is jointly Gaussian and independent of the empty recursive history. Inductively, if the current pair is centered Gaussian with variances $s_1^{\mathrm{\prime }},s_2^{\mathrm{\prime }}$, correlation $r$, and independent of the earlier recursive outputs, its observation is the pair's sum plus fresh unit Gaussian noise. The Gaussian residual after conditioning on this observation is independent of the observation and earlier history. Formula [F3] therefore gives the next normalized opposite-sign correlation $r_{\mathrm{n}\mathrm{e}\mathrm{w}}=\frac{\sqrt{s_1^{\mathrm{\prime }}{s}_2^{\mathrm{\prime }}}(1-r^2)-r}{\sqrt{(1+s_1^{\mathrm{\prime }}(1-r^2))(1+s_2^{\mathrm{\prime }}(1-r^2))}},$ with both marginal variances restored to $s_j^{\mathrm{\prime }}$ by the chosen scales.

At $r={\rho }^{\mathrm{\prime }}$, the denominator in step 3.1 is $\sqrt{D}$, where $D=1+s_1^{\mathrm{\prime }}+s_2^{\mathrm{\prime }}+2{\rho }^{\mathrm{\prime }}\sqrt{s_1^{\mathrm{\prime }}{s}_2^{\mathrm{\prime }}}$. Expanding the root equation gives $(\sqrt{s_1^{\mathrm{\prime }}{s}_2^{\mathrm{\prime }}}(1-{\rho }^{\mathrm{\prime }2})-{\rho }^{\mathrm{\prime }}{)}^2={\rho }^{\mathrm{\prime }2}D\mathrm{.}$ The unsquared numerator is positive: let $r_0$ be the unique root of $r=\sqrt{s_1^{\mathrm{\prime }}{s}_2^{\mathrm{\prime }}}(1-r^2)$. Substitution gives $F(r_0)=r_0^2(1-s_1^{\mathrm{\prime }}-s_2^{\mathrm{\prime }})-2r_0\sqrt{s_1^{\mathrm{\prime }}{s}_2^{\mathrm{\prime }}}<0$; if $s_1^{\mathrm{\prime }}+s_2^{\mathrm{\prime }}<1$, use $r_0<\sqrt{s_1^{\mathrm{\prime }}{s}_2^{\mathrm{\prime }}}$, and otherwise both terms are nonpositive with the second strictly negative. Since $F$ is strictly decreasing, ${\rho }^{\mathrm{\prime }}<r_0$, so $\sqrt{s_1^{\mathrm{\prime }}{s}_2^{\mathrm{\prime }}}(1-{\rho }^{\mathrm{\prime }2})-{\rho }^{\mathrm{\prime }}>0$. Thus $r_{\mathrm{n}\mathrm{e}\mathrm{w}}={\rho }^{\mathrm{\prime }}$, and induction proves the same correlation and powers at every recursive use.

Conditional on the recursive past, the two current inputs are centered jointly Gaussian with variances $s_j^{\mathrm{\prime }}$ and correlation ${\rho }^{\mathrm{\prime }}$. Thus ${\tilde{Y}}_i\mid {\tilde{Y}}_1^{i-1}$ is Gaussian with variance $D$, while ${\tilde{Y}}_i\mid (U_j,{\tilde{Y}}_1^{i-1})$ is Gaussian with variance $v_j:=1+s_{3-j}^{\mathrm{\prime }}(1-{\rho }^{\mathrm{\prime }2})$ because $V_{ji}$ is determined by $(U_j,{\tilde{Y}}_1^{i-1})$. Define the conditional information here as the expectation of the log ratio of these two conditional Gaussian densities. Integrating their log densities, whose centered quadratic terms have expectation one, gives $I(U_j;{\tilde{Y}}_i\mid {\tilde{Y}}_1^{i-1})=\frac{1}{2}\ln (D\mathrm{/}{v}_j)$. The root equation then gives $I(U_j;{\tilde{Y}}_i\mid {\tilde{Y}}_1^{i-1})=\frac{1}{2}\ln \frac{D}{1+s_{3-j}^{\mathrm{\prime }}(1-{\rho }^{\mathrm{\prime }2})}=\frac{1}{2}\ln (1+s_j^{\mathrm{\prime }}(1-{\rho }^{\mathrm{\prime }2}))\mathrm{.}$ Factoring the positive conditional Gaussian densities over time and summing their log ratios gives the chain rule directly, so $I(U_j;{\tilde{Y}}_1^n)=\frac{n}{2}\ln (1+s_j^{\mathrm{\prime }}(1-{\rho }^{\mathrm{\prime }2}))$. Since $(U_j,{\tilde{Y}}_1^n)$ is jointly Gaussian, the same density-ratio calculation gives $I(U_j;{\tilde{Y}}_1^n)=\frac{1}{2}\ln \text{\hspace{0.17em}⁣}(\mathrm{Var}(U_j)\mathrm{/}\mathrm{Var}(U_j\mid {\tilde{Y}}_1^n))$: the conditional Gaussian variance is constant in the observed value, and the log-density ratio expectation is the difference of the two Gaussian entropies. Therefore $\mathrm{Var}(U_j\mid {\tilde{Y}}_1^n)=\mathrm{Var}(U_j)e^{-2I(U_j;{\tilde{Y}}_1^n)}$.

The receiver observes ${\tilde{Y}}_{-1}+\lambda {\tilde{Y}}_0={\theta }_1+U_1$ and ${\tilde{Y}}_{-2}+\lambda {\tilde{Y}}_0={\theta }_2+U_2$. It estimates ${\theta }_j$ by subtracting $\mathbb{E}[U_j\mid {\tilde{Y}}_1^n]$. The estimation error variance is the conditional variance from step 5.1. Since the adjacent grid spacing is $2\mathrm{/}{M}_j$, decode to a nearest grid point and resolve any tie by choosing the least-indexed point. Chebyshev's inequality makes the decoding error tend to zero whenever $R_j<\frac{1}{2}\ln (1+s_j^{\mathrm{\prime }}(1-{\rho }^{\mathrm{\prime }2}))$.

The $n$ recursive symbols use expected physical power $P_j^{\mathrm{\prime }}$ each, and the one nonzero startup symbol for user $j$ has physical energy at most $N$. Thus total expected energy is at most $n{P}_j^{\mathrm{\prime }}+N$, which is at most $(n+3)P_j$ for all sufficiently large $n$ because $P_j^{\mathrm{\prime }}<P_j$. The block has $n+3$ uses and rates $n{R}_j\mathrm{/}(n+3)\to R_j$. Letting $P_j^{\mathrm{\prime }}\uparrow P_j$ and then closing the strict rates proves achievability of the stated corner.