Entropy bounds for a finite mixture of states

Omit zero weights. For each $i$ with $p_i>0$, $\bar{\rho }-p_i{\rho }_i={\sum }_{j\mathrm{\ne }i}{p}_j{\rho }_j$ is non-negative. If $v\in \ker \bar{\rho }$, then $0=⟨\bar{\rho }v,v⟩=p_i⟨{\rho }_{i}v,v⟩+{\sum }_{j\mathrm{\ne }i}{p}_j⟨{\rho }_{j}v,v⟩$. All terms are nonnegative, so $p_i>0$ implies $⟨{\rho }_{i}v,v⟩=0$ and [F5] gives ${\rho }_{i}v=0$. Hence $\mathrm{supp}{\rho }_i\subseteq \mathrm{supp}\bar{\rho }$ and every $D({\rho }_i\mathrm{\parallel }\bar{\rho })$ is finite. The trace formula [F6], linearity of trace [F4], and $\bar{\rho }={\sum }_i{p}_i{\rho }_i$ give ${\sum }_i{p}_{i}D({\rho }_i\mathrm{\parallel }\bar{\rho })=S(\bar{\rho })-{\sum }_i{p}_{i}S({\rho }_i)\ge 0$ by [F2].

For the upper bound, diagonalize each ${\rho }_i={\sum }_j{\lambda }_{ij}\mathrm{\mid }{v}_{ij}⟩⟨v_{ij}\mathrm{\mid }$, omitting zero weights and eigenvalues, and purify the mixture as $\mathrm{\mid }v⟩={\sum }_{ij}\sqrt{p_i{\lambda }_{ij}}\mathrm{\mid }{v}_{ij}⟩\otimes \mathrm{\mid }i,j⟩$. Its label marginal $G$ is a state with diagonal $q_{ij}=p_i{\lambda }_{ij}$ and has the same nonzero eigenvalues as $\bar{\rho }$ by Schmidt decomposition. The logarithm product law in [F6] gives $H(q)=H(p)+{\sum }_i{p}_{i}S({\rho }_i)$.

For $\mathrm{\Delta }=\mathrm{diag}(q)$, its kernel is spanned by coordinate labels with $q_{ij}=0$. Each such basis vector $e_{ij}$ has $⟨G{e}_{ij},e_{ij}⟩=q_{ij}=0$, so [F5] gives $G{e}_{ij}=0$. Hence $G$ annihilates $\ker \mathrm{\Delta }$ and $\mathrm{supp}G\subseteq \mathrm{supp}\mathrm{\Delta }$. The formula [F6] now gives $0\le D(G\mathrm{\parallel }\mathrm{\Delta })=H(q)-S(G)=H(p)+{\sum }_i{p}_{i}S({\rho }_i)-S(\bar{\rho })\mathrm{.}$ This proves both bounds, including singleton/zero-weight cases.