Robust Covariance and Sparse Robust Estimation

We first record two elementary trace estimates. If $A=U\mathrm{diag}(a_j)V^{\ast }$ is a singular value decomposition, cyclically expanding the finite diagonal sum gives $\mathrm{tr}(AB)={\sum }_j{a}_j⟨V^{\ast }BU{e}_j,e_j⟩$, so $\mathrm{\mid }\mathrm{tr}(AB)\mathrm{\mid }\le \mathrm{\parallel }A{\mathrm{\parallel }}_1\mathrm{\parallel }B{\mathrm{\parallel }}_{\mathrm{o}\mathrm{p}}$. Also $\mathrm{tr}(AB)={\sum }_{i,j}{A}_{ij}{B}_{ji}$, so scalar Cauchy–Schwarz gives $\mathrm{\mid }\mathrm{tr}(AB)\mathrm{\mid }\le \mathrm{\parallel }A{\mathrm{\parallel }}_2\mathrm{\parallel }B{\mathrm{\parallel }}_2$, since $\mathrm{\parallel }A{\mathrm{\parallel }}_2^2=\mathrm{tr}(A^{\ast }A)={\sum }_{i,j}\mathrm{\mid }{A}_{ij}{\mathrm{\mid }}^2$ by the singular value decomposition. These arguments also prove the trace cyclicity used here by interchanging two finite sums.

Set $F(z)=\exp (zH\mathrm{/}2)\exp (zK\mathrm{/}2)$ and $M=\mathrm{\parallel }\exp (H\mathrm{/}2)\exp (K\mathrm{/}2){\mathrm{\parallel }}_2>0$. The function $F$ is entrywise entire and uniformly bounded on the closed strip: for $z=x+iy$ each exponential is the product of a bounded positive real exponential with a unitary imaginary exponential of the same Hermitian matrix. On the line $x=0$, $\mathrm{\parallel }F(iy){\mathrm{\parallel }}_{\mathrm{o}\mathrm{p}}=1$. On $x=1$, the identity $F(1+iy)=\exp (iyH\mathrm{/}2)\exp (H\mathrm{/}2)\exp (K\mathrm{/}2)\exp (iyK\mathrm{/}2)$ and unitary invariance give $\mathrm{\parallel }F(1+iy){\mathrm{\parallel }}_2=M$.

We need a finite eigenvalue comparison only for a diagonalizable invertible matrix $A$. Order its eigenvalue moduli and its singular values decreasingly. For $1\le k\le p$, the $k$th exterior space has the orthonormal wedge basis $e_{j_1}\wedge \cdots \wedge e_{j_k}$, $j_1<\cdots <j_k$, and its induced map sends a wedge to the wedge of the images. This defines the finite matrix ${\bigwedge }^{k}A$ directly by multilinearity and alternation. The singular value decomposition makes its norm ${\prod }_{j=1}^k{s}_j(A)$: wedges of the two singular-vector orthonormal bases give diagonal coefficients consisting of all $k$-fold singular-value products. Wedges of the first $k$ independent eigenvectors are nonzero eigenvectors with eigenvalue ${\prod }_{j=1}^k{\lambda }_j(A)$; nonvanishing follows by expanding independent vectors in a basis and noting that some $k$-minor is nonzero. The norm bounds the modulus of an eigenvalue, proving ${\prod }_{j=1}^k\mathrm{\mid }{\lambda }_j(A)\mathrm{\mid }\le {\prod }_{j=1}^k{s}_j(A)$. This uses only finite bases and finite determinants.

Fix $0<s<1$, put $q=2\mathrm{/}s$ and $q^{\mathrm{\prime }}=2\mathrm{/}(2-s)$, and choose a singular value decomposition $F(s)=U\mathrm{diag}(a_j)V^{\ast }$. All $a_j$ are positive because $F(s)$ is invertible. Set $N=({\sum }_j{a}_j^q{)}^{1\mathrm{/}q}>0$ and $b_j=(a_j\mathrm{/}N{)}^{q-1}$, so ${\sum }_j{b}_j^{q^{\mathrm{\prime }}}=1$. Define $G(z)=V\mathrm{diag}(\exp (q^{\mathrm{\prime }}(1-z\mathrm{/}2)\log b_j))U^{\ast }$ and $g(z)=\mathrm{tr}(G(z)F(z))$. These functions are continuous and bounded on the closed strip and holomorphic inside: their entries are finite sums of entire exponentials and their moduli are bounded uniformly in the imaginary coordinate. On its left boundary $\mathrm{\parallel }G(iy){\mathrm{\parallel }}_1={\sum }_j{b}_j^{q^{\mathrm{\prime }}}=1$, and on its right boundary $\mathrm{\parallel }G(1+iy){\mathrm{\parallel }}_2=({\sum }_j{b}_j^{q^{\mathrm{\prime }}}{)}^{1\mathrm{/}2}=1$. The two trace estimates of step 1.1 therefore bound $\mathrm{\mid }g\mathrm{\mid }$ by $1$ and $M$ on the respective boundaries. Since $q^{\mathrm{\prime }}(1-s\mathrm{/}2)=1$, one has $g(s)={\sum }_j{a}_j{b}_j=N$. Applying [F3] yields $\mathrm{\parallel }F(s){\mathrm{\parallel }}_{2\mathrm{/}s}\le M^s$.

Put $x_j=\log \mathrm{\mid }{\lambda }_j(A)\mathrm{\mid }$ and $y_j=\log s_j(A)$ in decreasing order. Step 1.3 gives ${\sum }_{j=1}^k{x}_j\le {\sum }_{j=1}^k{y}_j$ for every $k$. For any real $u$, the positive terms form an initial segment, so ${\sum }_j(x_j-u{)}_{+}={\max }_{0\le k\le p}({\sum }_{j=1}^k{x}_j-ku)\le {\sum }_j(y_j-u{)}_{+}$. For every $c>0$, multiply by $c^2{e}^{cu}$ and integrate: $c^2{\int }_{-\mathrm{\infty }}^x(x-u)e^{cu}du=e^{cx}$, as follows by evaluating the elementary exponential primitive at finite lower limits and sending that limit to $-\mathrm{\infty }$. The finite sums can be integrated termwise and all integrands are nonnegative, giving ${\sum }_j\mathrm{\mid }{\lambda }_j(A){\mathrm{\mid }}^c\le {\sum }_j{s}_j(A{)}^c$.

For each integer $n\ge 2$, let $A_n=F(1\mathrm{/}n)$. It is similar to the positive definite matrix $\exp (K\mathrm{/}4n)\exp (H\mathrm{/}2n)\exp (K\mathrm{/}4n)$, since conjugating $A_n$ by $\exp (K\mathrm{/}4n)$ gives that matrix. Thus [F2] makes $A_n$ diagonalizable with positive eigenvalues. Apply step 2.2 with $c=2n$ and step 2.1 with $s=1\mathrm{/}n$ to obtain $\mathrm{tr}(A_n^{2n})\le {\sum }_j{s}_j(A_n{)}^{2n}=\mathrm{\parallel }{A}_n{\mathrm{\parallel }}_{2n}^{2n}\le M^2$.

The exponential series gives $A_n=I+(H+K)\mathrm{/}(2n)+O(n^{-2})$ and $B_n:=\exp ((H+K)\mathrm{/}(2n))=I+(H+K)\mathrm{/}(2n)+O(n^{-2})$ in operator norm, where the constants depend only on the fixed matrices. Also $\mathrm{\parallel }{A}_n\mathrm{\parallel },\mathrm{\parallel }{B}_n\mathrm{\parallel }\le e^{c\mathrm{/}n}$ for one fixed $c$. The telescoping identity $A^N-B^N={\sum }_{j=0}^{N-1}{A}^j(A-B)B^{N-1-j}$, valid by cancellation without commutativity, gives $\mathrm{\parallel }{A}_n^{2n}-B_n^{2n}\mathrm{\parallel }=O(n^{-1})$. The addition law for exponential series of scalar multiples of one matrix gives $B_n^{2n}=\exp (H+K)$. Therefore $\mathrm{tr}\exp (H+K)={\lim }_n\mathrm{tr}(A_n^{2n})\le M^2=\mathrm{tr}(\exp (H)\exp (K))$, using $\mathrm{\mid }\mathrm{tr}C\mathrm{\mid }\le p\mathrm{\parallel }C{\mathrm{\parallel }}_{\mathrm{o}\mathrm{p}}$ and cyclicity to evaluate the Hilbert–Schmidt square $M^2$. This is the claimed inequality.

Since the standard-normal density is bounded by $1\mathrm{/}\sqrt{2\pi }$, $\mathrm{\mid }\mathrm{\Phi }(t+h)-\mathrm{\Phi }(t)\mathrm{\mid }\le \mathrm{\mid }h\mathrm{\mid }\mathrm{/}\sqrt{2\pi }$; hence $\mathrm{\Phi }$ is continuous. The density is positive, so its integral over every nonempty interval is positive and $\mathrm{\Phi }$ is strictly increasing. Evenness and total mass one give $\mathrm{\Phi }(0)=1\mathrm{/}2$. Applying [F2] to $(-\mathrm{\infty },n]$ as $n\to \mathrm{\infty }$ gives $\mathrm{\Phi }(n)\to 1$.

Choose an integer $n$ with $\mathrm{\Phi }(n)>3\mathrm{/}4$. By [F3] on $[0,n]$, some $q\in (0,n)$ satisfies $\mathrm{\Phi }(q)=3\mathrm{/}4$; strict increase from step 1.1 makes $q$ unique. Then $\beta =q^2>0$ is the unique positive value with $\mathrm{\Phi }(\sqrt{\beta })=3\mathrm{/}4$.

The supremum is measurable. Fix $S$ with $d=\mathrm{\mid }S\mathrm{\mid }$. By [F9], rational vectors supported in $S$ can be enumerated by enumerating their finite tuples of rational indices in increasing sum; their nonzero normalizations form a countable subclass of $V_S$. For $u\in V_S$ and each $k\ge 1$, choose the first rational tuple $q_k$ in that enumeration satisfying $\mathrm{\mid }(q_k{)}_i-u_i\mathrm{\mid }<1\mathrm{/}(4d{k}^2)$ for $i\in S$. Such a tuple exists by rational density, and $\mathrm{\parallel }{q}_k-u{\mathrm{\parallel }}_2<1\mathrm{/}(4k^2)$, so $q_k\mathrm{\ne }0$. With $u_k=q_k\mathrm{/}\mathrm{\parallel }{q}_k{\mathrm{\parallel }}_2$, [F7] gives $\mathrm{\parallel }{u}_k-u{\mathrm{\parallel }}_2\le \mathrm{\mid }1-\mathrm{\parallel }{q}_k{\mathrm{\parallel }}_2\mathrm{\mid }+\mathrm{\parallel }{q}_k-u{\mathrm{\parallel }}_2\le 2\mathrm{\parallel }{q}_k-u{\mathrm{\parallel }}_2\le k^{-2}$. For $t\ge 0$, choose the first rational $t_k\in (t+1\mathrm{/}k,t+2\mathrm{/}k)$ in the fixed enumeration. Cauchy--Schwarz yields $\mathrm{\mid }(u_k^{T}x{)}^2-(u^{T}x{)}^2\mathrm{\mid }\le 2\mathrm{\parallel }x{\mathrm{\parallel }}_2^2{k}^{-2}\mathrm{.}$ Thus ${\mathbf{1}}_{A_{u_k,t_k}}(x)\to {\mathbf{1}}_{A_{u,t}}(x)$ for every $x$, including boundary points: at equality the threshold margin $1\mathrm{/}k$ eventually exceeds this error, while away from equality convergence preserves the strict sign. By [F3] their $P$-probabilities converge; finite empirical probabilities converge too. Negative thresholds give the empty set, already represented by rational thresholds. Hence the supremum equals that over the countable subclass of normalized rational vectors and rational thresholds, and is measurable. The first-element rules use no choice axiom.

Fix a support $S$ of size $k\le s$. Trace patterns of affine halfspaces on $r$ fixed points are sign patterns of $r$ affine hyperplanes in a $(k+1)$-dimensional parameter space. The number of open regions is at most ${\sum }_{\mathrm{\ell }=0}^{k+1}\left(\genfrac{}{}{0px}{}{r}{\mathrm{\ell }}\right)$: adding one hyperplane creates at most the number of regions cut on that hyperplane, so induction on $r$ and the dimension gives Pascal's recurrence and this sum. Weak-boundary traces are also counted, since on a finite point set a halfspace threshold can be increased slightly without changing its trace.

A slab $A_{u,t}$ with $t\ge 0$ is the intersection of the halfspaces $u^{T}x\le \sqrt{t}$ and $-u^{T}x\le \sqrt{t}$ in the same $k$ coordinates; when $t<0$ it is empty. Hence its trace count on $r$ points is at most $({\sum }_{\mathrm{\ell }=0}^{k+1}\left(\genfrac{}{}{0px}{}{r}{\mathrm{\ell }}\right){)}^2$. For $r\ge q:=k+1$, set $x=q\mathrm{/}r$. Since $x^{\mathrm{\ell }}\ge x^q$ for $\mathrm{\ell }\le q$, ${\sum }_{\mathrm{\ell }=0}^q\left(\genfrac{}{}{0px}{}{r}{\mathrm{\ell }}\right)\le x^{-q}(1+x{)}^r\le (er\mathrm{/}q{)}^q\mathrm{.}$ If $r\ge 16q$, then $2^r>(er\mathrm{/}q{)}^{2q}$: the logarithmic difference is positive at $r\mathrm{/}q=16$ and has positive derivative thereafter. Thus $r$ points cannot be shattered by the slab class, so its VC dimension is at most $16(k+1)\le 16(s+1)$.

Let ${\epsilon }_i$ be independent uniform signs, independent of the sample, and let $X_1^{\mathrm{\prime }},\dots ,X_N^{\mathrm{\prime }}$ be an independent copy. Jensen's inequality [F5], conditional on the original sample, gives $\mathbb{E}{\sup }_A\mathrm{\mid }{P}_N(A)-P(A)\mathrm{\mid }\le \mathbb{E}{\sup }_A\mathrm{\mid }{P}_N(A)-P_N^{\mathrm{\prime }}(A)\mathrm{\mid }\mathrm{.}$ Each pair $(X_i,X_i^{\mathrm{\prime }})$ is exchangeable, so independently swapping its coordinates with sign ${\epsilon }_i$ leaves the joint law unchanged. The triangle inequality then gives $\mathbb{E}{\sup }_A\mathrm{\mid }{P}_N(A)-P(A)\mathrm{\mid }\le 2\mathbb{E}{\sup }_A\mid N^{-1}{\sum }_{i=1}^N{\epsilon }_i{\mathbf{1}}_A(X_i)\mid \mathrm{.}$ The ghost sample and signs exist on finite product spaces by [F6]; all suprema use the countable class from step 1.1.

Replacing one observation changes every empirical set probability by at most $1\mathrm{/}N$, so the supremum $F$ changes by at most $1\mathrm{/}N$. In the Doob martingale of $F$, each conditional increment therefore has a range of width at most $1\mathrm{/}N$. If a random variable lies in an interval of width $c$, the log-mgf of its centered version has second derivative equal to a tilted variance at most $c^2\mathrm{/}4$; integrating twice gives $\mathbb{E}{e}^{\lambda (Y-\mathbb{E}Y)}\le e^{{\lambda }^2{c}^2\mathrm{/}8}$. Iterating these bounds and optimizing the exponential Markov inequality yields $\mathbb{P}(F-\mathbb{E}F>t)\le e^{-2N{t}^2}$.

On a fixed $N$-point sample the trace class is finite. For $0<\rho \le 1$, if $M$ traces are pairwise separated by more than $\rho$ in $d(A,B)=\sqrt{P_N(A\mathrm{△}B)}$, sample $r=\lceil 3{\rho }^{-2}\log (2M)\rceil$ coordinates uniformly with replacement. For a fixed pair the probability that the traces agree at all sampled coordinates is at most $e^{-r{\rho }^2}$; a union bound over pairs shows that some coordinate sample distinguishes all $M$ traces. Put $D=16(s+1)$. If $r<D$, then $M\le 2^r\le 2^D$. If $r\ge D$, step 2.1 bounds the trace count by $(er\mathrm{/}q{)}^{2q}\le (er\mathrm{/}D{)}^{2D}$, where $q=\mathrm{\mid }S\mathrm{\mid }+1\le D$; the inequality follows because $a\mapsto a\log (er\mathrm{/}a)$ is increasing for $0<a\le r$. Writing $y=\log (M)\mathrm{/}D$, the choice of $r$ gives $r\le C{\rho }^{-2}D(y+1)$ and hence $y\le 2\log (C{\rho }^{-2}(y+1))$. Put $A=\log (C{\rho }^{-2})\ge 1$. If $y\ge 10$, then $\log (y+1)\le y\mathrm{/}4$, so $y\le 4A$; if $y<10$, then $y<10A$. Thus $y\le 10A\le 20\log (C_0\mathrm{/}\rho )$ for a universal $C_0$. After increasing $C_0$, the same bound covers $r<D$. A maximal $\rho$-separated set is a $\rho$-net, so $\mathcal{N}(\rho )\le (C_0\mathrm{/}\rho {)}^{20D}$.

To obtain the sign MGF bound used for chaining, for $t\ge 0$ put $h(t)=t-\tanh t$. Then $h(0)=0$ and $h^{\mathrm{\prime }}(t)=1-{\mathrm{sech}}^{2}t={\tanh }^{2}t\ge 0$ by [F8]; the mean value theorem gives $h(t)\ge 0$. Put $g(t)=t^2\mathrm{/}2-\log (\cosh t)$. Then $g(0)=0$ and $g^{\mathrm{\prime }}(t)=h(t)\ge 0$, so $g(t)\ge 0$; evenness extends the bound to every real $t$. Hence $\cosh t\le e^{t^2\mathrm{/}2}$. Conditional on the sample, the sign process $Z_A=N^{-1}{\sum }_i{\epsilon }_i{\mathbf{1}}_A(X_i)$ therefore has increments satisfying ${\mathbb{E}}_{\epsilon }{e}^{\lambda (Z_A-Z_B)}\le e^{{\lambda }^{2}d(A,B{)}^2\mathrm{/}(2N)}$, by multiplying the independent sign MGFs with coefficients $\lambda ({\mathbf{1}}_A(X_i)-{\mathbf{1}}_B(X_i))\mathrm{/}N$. Choose empirical $L^2$ nets $T_j$ at radii $2^{-j}$. Telescope each trace through its nearest net points; adjacent levels differ by at most $3\cdot 2^{-j}$ in $d$. For $L$ centered sub-Gaussian variables with common proxy ${\sigma }^2$, $\mathbb{E}{\max }_{\mathrm{\ell }\le L}\mathrm{\mid }{W}_{\mathrm{\ell }}\mathrm{\mid }\le \sigma \sqrt{2\log (2L)}$ follows by Jensen's inequality applied to the log-sum-exp of $e^{\lambda W_{\mathrm{\ell }}}$ and $e^{-\lambda W_{\mathrm{\ell }}}$, followed by minimization in $\lambda$. Thus the conditional expected supremum is at most $\frac{C}{\sqrt{N}}(1+{\sum }_{j\ge 1}{2}^{-j}\sqrt{\log (2\mathcal{N}(2^{-j})\mathcal{N}(2^{-j+1}))})\mathrm{.}$ The finite trace class makes the sum terminate.

From step 3.1, $\log \mathcal{N}(\rho )\le 20D\log (C_0\mathrm{/}\rho )$. Therefore the dyadic sum in step 4.1 is at most $C\sqrt{D}{\sum }_{j\ge 1}{2}^{-j}\sqrt{j+1}\le C^{\mathrm{\prime }}\sqrt{D}$; the base trace contributes at most $N^{-1\mathrm{/}2}$ by Cauchy--Schwarz. Combining the symmetrization bound from step 2.2 with the chaining bound from step 4.1 yields $\mathbb{E}{\sup }_A\mathrm{\mid }{P}_N(A)-P(A)\mathrm{\mid }\le C^{\mathrm{\prime }\mathrm{\prime }}\sqrt{(s+1)\mathrm{/}N}$.

Apply step 2.3 with $t=\sqrt{\log (m\mathrm{/}\delta )\mathrm{/}(2N)}$ separately to each support class and take a union bound over its $m$ supports. Combining with step 5.1 proves the displayed probability bound after increasing the universal constant $C$.

For any $u,t$, $\{x:\mathrm{\mid }{u}^{T}x{\mathrm{\mid }}^2<t\}={\bigcup }_{k\ge 1}\{x:\mathrm{\mid }{u}^{T}x{\mathrm{\mid }}^2\le t-1\mathrm{/}k\},$ an increasing union. Continuity from below [F4] makes both $P$ and $P_N$ probabilities converge to those of the strict lower-tail set. The deviation bound in step 6.1 holds at every threshold, so it passes to this limit. Taking complements gives the same bound for the upper weak tails $\{\mathrm{\mid }{u}^{T}x{\mathrm{\mid }}^2\ge t\}$.

Fix $0<s<3\mathrm{/}R$. If $\mathrm{\mid }x\mathrm{\mid }\le R$ and $k\ge 2$ is an integer, then $x^k\le R^{k-2}{x}^2$, including odd $k$ and negative $x$. Applying the spectral calculus and taking expectations gives $\mathbb{E}{B}_i^k⪯R^{k-2}\mathbb{E}{B}_i^2$. Since $k!\ge 2\cdot 3^{k-2}$ for every $k\ge 2$ (start at $k=2$ and multiply by $k+1\ge 3$), the exponential series and $\mathbb{E}{B}_i=0$ yield $\mathbb{E}\exp (s{B}_i)⪯I+g(s)\mathbb{E}{B}_i^2⪯e^{g(s)v}I$, where $g(s)=s^2\mathrm{/}[2(1-sR\mathrm{/}3)]$. The first inequality is the sum of the convergent geometric majorant ${\sum }_{k\ge 2}{s}^k{R}^{k-2}\mathrm{/}k!\le g(s)$, and the second uses $0⪯\mathbb{E}{B}_i^2⪯vI$ and the scalar inequality $1+a\le e^a$. Termwise expectation is justified by the uniform norm majorant ${\sum }_k(sR{)}^k\mathrm{/}k!$.

Let $Y_j={\sum }_{i=1}^j{B}_i$, with $Y_0=0$. For each fixed value of $Y_{j-1}$, [F2] and integration over the independent $B_j$ give ${\mathbb{E}}_{B_j}\mathrm{tr}\exp (s{Y}_{j-1}+s{B}_j)\le \mathrm{tr}(\exp (s{Y}_{j-1})\mathbb{E}\exp (s{B}_j))\le e^{g(s)v}\mathrm{tr}\exp (s{Y}_{j-1})$. The last implication follows because $P⪰0$ and $Q⪯cI$ imply $\mathrm{tr}(PQ)\le c\mathrm{tr}P$: diagonalize $P$ and sum its nonnegative eigenvalues times the bounded diagonal quadratic forms of $Q$. Successive integration using [F3] and iteration from $\mathrm{tr}I=q$ prove $\mathbb{E}\mathrm{tr}\exp (sY)\le q{e}^{mg(s)v}$.

By [F1], $\mathrm{tr}\exp (sY)\ge \exp (s{\lambda }_{\max }(Y))$. Markov's inequality [F4] and step 2.1 therefore give $\mathbb{P}\{{\lambda }_{\max }(Y)\ge u\}\le q\exp (-su+mg(s)v)$ for every $u>0$ and $0<s<3\mathrm{/}R$. If $v>0$, choose $s=u\mathrm{/}(mv+Ru\mathrm{/}3)$, which belongs to that interval. Substitution gives $-su+mg(s)v=-u^2\mathrm{/}[2(mv+Ru\mathrm{/}3)]$, proving the stated tail bound. The same fixed-parameter bound with $>$ in place of $\ge$ will also be used below.

For $b>0$ set $u=\sqrt{2mvb}+2Rb\mathrm{/}3$. Expanding its square shows $u^2\ge 2b(mv+Ru\mathrm{/}3)$: with $a=\sqrt{2mvb}$ and $c=2Rb\mathrm{/}3$, the two sides are respectively $a^2+2ac+c^2$ and $a^2+ac+c^2$. Thus step 3.1 bounds the one-sided failure by $q{e}^{-b}$. Applying the same argument to $-B_i$ and taking the union of the two failures bounds the norm failure by $2q{e}^{-b}$, since the norm of a Hermitian matrix is the larger of its maximum eigenvalue and the maximum eigenvalue of its negative.

Finally, if $v=0$, then for each coordinate vector $e_j$, $\mathbb{E}\mathrm{\parallel }{B}_i{e}_j{\mathrm{\parallel }}_2^2=e_j^{\ast }(\mathbb{E}{B}_i^2)e_j=0$. A nonnegative random variable with zero expectation vanishes almost surely: otherwise one of the events where it exceeds $1\mathrm{/}n$ would have positive probability and positive contribution to the expectation. Intersecting the finitely many full-probability coordinate events gives $B_i=0$ almost surely; a further finite intersection covers all $i$. The trace mgf is then $q$ and both strict positive-threshold events are empty. This covers the remaining case and completes all conclusions.

For each coordinate, $\mathrm{\mid }{X}_j\mathrm{\mid }\le \mathrm{\parallel }X{\mathrm{\parallel }}_2$ and $\mathrm{\mid }x\mathrm{\mid }\le (1+x^2)\mathrm{/}2$ imply $\mathbb{E}\mathrm{\mid }{X}_j\mathrm{\mid }<\mathrm{\infty }$, so every ${\mu }_j$ exists. Also $\mathbb{E}(X_j-{\mu }_j{)}^2<\mathrm{\infty }$, since $(X_j-{\mu }_j{)}^2\le 2X_j^2+2{\mu }_j^2$.

For every pair $j,k$, $2\mathrm{\mid }(X_j-{\mu }_j)(X_k-{\mu }_k)\mathrm{\mid }\le (X_j-{\mu }_j{)}^2+(X_k-{\mu }_k{)}^2$. Thus every covariance entry is integrable; all entries are finite and symmetry follows from commutativity of multiplication.

For $a\in {\mathbb{R}}^p$, $a^T\mathrm{\Sigma }a=\mathbb{E}[(a^T(X-\mu ){)}^2]\ge 0$, so $\mathrm{\Sigma }$ is positive semidefinite. Summing the diagonal entries gives $\mathrm{tr}(\mathrm{\Sigma })={\sum }_j\mathbb{E}(X_j-{\mu }_j{)}^2=\mathbb{E}{\sum }_j(X_j-{\mu }_j{)}^2=\mathbb{E}\mathrm{\parallel }X-\mu {\mathrm{\parallel }}_2^2$; the sum is finite.