Gaussian maximizes differential entropy at fixed positive-definite covariance

Let $g$ be the everywhere-positive Gaussian density in [F1] with mean $m$ and covariance $K$. Since $f$ is a Lebesgue density and $g>0$ everywhere, the law $P_f$ is absolutely continuous with respect to $P_g$. The log ratio is integrable: $\int f\ln f$ is finite by hypothesis, while $\ln g$ is a constant plus a quadratic form and $X$ has finite second moments. Hence $D(P_f\mathrm{\parallel }{P}_g)=\int f\ln (f\mathrm{/}g)=-h(X)+\frac{d}{2}\ln (2\pi )+\frac{1}{2}\ln \det K+\frac{1}{2}\mathbb{E}[(X-m{)}^T{K}^{-1}(X-m)]\mathrm{.}$ The last expectation is $\mathrm{tr}(K^{-1}K)=d$, by expanding the finite quadratic form and using the covariance entries.

By [F2], the left side in step 1.1 is nonnegative, which rearranges to the displayed entropy bound. Equality holds exactly when $D(P_f\mathrm{\parallel }{P}_g)=0$, which by [F2] is equivalent to equality of the laws; thus $X\sim N_d(m,K)$. Conversely, substituting the density $g$ into the entropy integral gives equality because its centered quadratic expectation is $d$.