Gaussian convolution has second-order KL and log-square error

At every zero of the nonnegative differentiable function $p$, $p^{\mathrm{\prime }}=0$. At each accumulation point of its zero set, differentiation along zeros of $p^{\mathrm{\prime }}$ gives $p^{\mathrm{\prime }\mathrm{\prime }}=0$; isolated zeros form a countable set. Thus $p^{\mathrm{\prime }\mathrm{\prime }}=0$ almost everywhere on $\{p=0\}$. Put $r=p^{\mathrm{\prime }\mathrm{\prime }}\mathrm{/}p$ on $\{p>0\}$ and zero elsewhere. Then $p^{\mathrm{\prime }\mathrm{\prime }}=rp$ almost everywhere, $\int r^{2}p=J$, and [F2] gives $\int \mathrm{\mid }{p}^{\mathrm{\prime }\mathrm{\prime }}\mathrm{\mid }\le \sqrt{J}$. Both $p$ and $p^{\mathrm{\prime }\mathrm{\prime }}$ are continuous and compactly supported.

Let $P_{t}f=f\ast {\varphi }_{\sqrt{t}}$ for $t>0$. Direct differentiation of [F1] gives ${\mathrm{\partial }}_t{\varphi }_{\sqrt{t}}=\frac{1}{2}{\mathrm{\partial }}_x^2{\varphi }_{\sqrt{t}}$. For fixed $x$ and $0<\epsilon <t$, the differentiated integrand in $P_{s}p(x)$ is bounded on $s\in [\epsilon ,t]$ and the compact support of $p$ by $C_{x,\epsilon ,t}\mathrm{\mid }p(z)\mathrm{\mid }$; differentiation under the integral [F5] and two integrations by parts [F6] therefore give ${\mathrm{\partial }}_s{P}_{s}p=\frac{1}{2}{P}_s{p}^{\mathrm{\prime }\mathrm{\prime }}$. The boundary terms vanish because $p,p^{\mathrm{\prime }}$ have compact support. Dominated convergence [F9] makes $s\mapsto P_s{p}^{\mathrm{\prime }\mathrm{\prime }}(x)$ continuous on $[\epsilon ,t]$, so [F7] integrates the derivative there. Gaussian approximate identities converge uniformly on continuous compactly supported functions: split their convolution integral at $\mathrm{\mid }z\mathrm{\mid }\le \delta$, where uniform continuity controls the integrand difference, and its complement, whose normal mass tends to zero by scaling and integrability of [F1]. Letting $\epsilon \downarrow 0$ consequently yields $p-P_{t}p=-\frac{1}{2}{\int }_0^t{P}_s{p}^{\mathrm{\prime }\mathrm{\prime }}ds$ pointwise; the omitted integral over $[0,\epsilon ]$ tends to zero because $\mathrm{\mid }{P}_s{p}^{\mathrm{\prime }\mathrm{\prime }}(x)\mathrm{\mid }\le \mathrm{\parallel }{p}^{\mathrm{\prime }\mathrm{\prime }}{\mathrm{\parallel }}_{\mathrm{\infty }}$.

Fix $t>0$ and put $q=P_{t}p$. It is positive everywhere because $p$ is a nonzero nonnegative density and the Gaussian kernel is positive; Tonelli [F3] and $\int {\varphi }_{\sqrt{t}}=1$ show $q$ integrates to one. For $0<s\le t$, the Gaussian formula gives ${\varphi }_{\sqrt{t}}(z)\ge \sqrt{s\mathrm{/}t}{\varphi }_{\sqrt{s}}(z)$ for every $z$, hence $q\ge \sqrt{s\mathrm{/}t}{P}_{s}p$. Applying [F2] inside the convolution and then [F3] gives $\mathrm{\mid }{P}_s{p}^{\mathrm{\prime }\mathrm{\prime }}{\mathrm{\mid }}^2\le (P_{s}p)P_s(r^{2}p)$ and therefore $\int \mathrm{\mid }{P}_s{p}^{\mathrm{\prime }\mathrm{\prime }}{\mathrm{\mid }}^2\mathrm{/}q\le \sqrt{t\mathrm{/}s}\int P_s(r^{2}p)=\sqrt{t\mathrm{/}s}J$. These inequalities remain valid if one of the pointwise nonnegative integrals is infinite.

Apply [F2] to the time integral in step 2.1 with weights $w(s)=(t\mathrm{/}s{)}^{1\mathrm{/}4}$. Since ${\int }_0^{t}w(s)ds=4t\mathrm{/}3$, it gives $\mathrm{\mid }p-q{\mathrm{\mid }}^2\le \frac{1}{4}(4t\mathrm{/}3){\int }_0^{t}w(s{)}^{-1}\mathrm{\mid }{P}_s{p}^{\mathrm{\prime }\mathrm{\prime }}{\mathrm{\mid }}^{2}ds$. Divide by $q$, integrate and use [F3] and step 2.2. The result is $\int (p-q{)}^2\mathrm{/}q\le \frac{1}{4}(4t\mathrm{/}3)J{\int }_0^t(t\mathrm{/}s{)}^{1\mathrm{/}4}ds=(4\mathrm{/}9)t^{2}J$. In particular all these weighted integrals are finite.

For $u>0$, the function $u-1-\log u$ has value zero at $1$ and derivative $(u-1)\mathrm{/}u$, so $\log u\le u-1$ and $u\log u-u+1\le (u-1{)}^2$. For $t\ge 1$, $t-t^{-1}-2\log t$ has value zero at $1$ and derivative $(t-1{)}^2\mathrm{/}{t}^2\ge 0$; replacing $t$ by $1\mathrm{/}t$ gives $2\mathrm{\mid }\log t\mathrm{\mid }\le \mathrm{\mid }t-t^{-1}\mathrm{\mid }$ also for $0<t<1$. With $u=t^2$, this yields $u(\log u{)}^2\le (u-1{)}^2$. At $u=0$, use the conventions $u\log u=u(\log u{)}^2=0$, for which both inequalities hold. Apply these pointwise bounds to $u=p\mathrm{/}q$ and integrate against $q$. Since $p,q$ integrate to one, [F4] identifies the first integral as KL; its negative part is integrable because $u\log u\ge -e^{-1}$. Both requested logarithmic quantities are at most the chi-square integral from step 3.1. Taking $t={\sigma }^2$ proves the Statement.