Entropy bounds for finite-power Gaussian convolutions

The convolution $f_Y(y)=\mathbb{E}[(2\pi N{)}^{-d\mathrm{/}2}{e}^{-\mathrm{\parallel }y-X{\mathrm{\parallel }}^2\mathrm{/}(2N)}]$ is measurable and strictly positive. Tonelli's theorem [F3] gives integral one. Direct Gaussian integration gives $\mathbb{E}\mathrm{\parallel }Z{\mathrm{\parallel }}^2=dN$, hence $\mathbb{E}\mathrm{\parallel }Y{\mathrm{\parallel }}^2<\mathrm{\infty }$.

Choose $K$ with $p_0:=\mathrm{Pr}(\mathrm{\parallel }X\mathrm{\parallel }\le K)>0$, which exists because the increasing union of the finite-radius balls is ${\mathbb{R}}^d$. For every $y$, $f_Y(y)\le (2\pi N{)}^{-d\mathrm{/}2}$ and $f_Y(y)\ge p_0(2\pi N{)}^{-d\mathrm{/}2}{e}^{-(\mathrm{\parallel }y\mathrm{\parallel }+K{)}^2\mathrm{/}(2N)}$. Thus $\mathrm{\mid }\ln f_Y(y)\mathrm{\mid }\le C(1+\mathrm{\parallel }y{\mathrm{\parallel }}^2)$ for a finite constant $C$; [F1] and step 1.1 give $\int f_Y\mathrm{\mid }\ln f_Y\mathrm{\mid }<\mathrm{\infty }$. The same argument applies to every coordinate marginal and, for almost every $s$, to the conditional law $P_{X\mathrm{\mid }S=s}$ using a radius $K_s$ with positive conditional mass.

For any coordinate density $f_{Y_k}$, let $\mu =\mathbb{E}{Y}_k$, $v=\mathrm{Var}{Y}_k>0$, and let ${\varphi }_{\mu ,v}$ be the matching Gaussian density. The inequality $t\ln t-t+1\ge 0$ for $t\ge 0$ follows by differentiation and its minimum at $1$; integrating it against ${\varphi }_{\mu ,v}$ gives $\int f_{Y_k}\ln (f_{Y_k}\mathrm{/}{\varphi }_{\mu ,v})\ge 0$. Step 2.1 gives integrability of the entropy term, and the second moment gives integrability of the Gaussian log density. Expanding that log density yields $h(Y_k)\le \frac{1}{2}\ln (2\pi ev)$.

Apply the same inequality to the ratio of the joint density of $Y$ to the product of its coordinate marginals. Step 2.1 makes every log term integrable, and expansion gives $h(Y)\le {\sum }_{k}h(Y_k)$.

Under the reference law $P_X(dx)f_Y(y)dy$, the ratio of the joint law of $(X,Y)$ to that reference is ${\varphi }_N^{\otimes d}(y-x)\mathrm{/}{f}_Y(y)$. Its mean is one; integrating $t\ln t-t+1\ge 0$ against the reference gives a nonnegative integral. The logarithm is integrable by step 2.1 and the finite Gaussian second moment. Expanding it gives $h(Y)-h(Z)\ge 0$, and direct integration of ${\varphi }_N$ gives $h(Z)=\frac{d}{2}\ln (2\pi eN)$. Steps 2.1, 3.1 and 3.2 prove the unconditional claims.

Independence of $Z$ from $(X,S)$ makes the conditional law of $Y$ given $S=s$ the convolution of $P_{X\mathrm{\mid }S=s}$ with the same $N(0,N{I}_d)$ density. For almost every $s$, repeat steps 1.1 and 2.1 with $P_{X\mathrm{\mid }S=s}$; applying steps 3.1 and 3.2 coordinatewise gives $h(Y\mathrm{\mid }S=s)\le {\sum }_{k=1}^d\frac{1}{2}\ln \text{\hspace{0.17em}⁣}(2\pi e[N+\mathrm{Var}(X_k\mathrm{\mid }S=s)])\mathrm{.}$ The conditional version of step 4.1 also gives $h(Y\mathrm{\mid }S=s)\ge h(Z)$, so these bounds and step 2.1 make the averaged conditional entropy finite. Averaging the upper bound and applying concavity of $\ln$ to each coordinate separately yields the displayed conditional inequality; each $\mathbb{E}\mathrm{Var}(X_k\mathrm{\mid }S)$ is finite because it is at most $\mathbb{E}{X}_k^2$.