Almost-sure Marchenko–Pastur law for finite-fourth-moment sample covariance matrices

Fix $z=u+i\eta$ with $\eta >0$. Write the transpose of row $i$ of $X_n$ as $y_i\in {\mathbb{R}}^{p_n}$ and set $Q=(S_n-zI{)}^{-1}$, $m_n=p_n^{-1}\mathrm{tr}Q=m_{{\mu }_n}(z)$ by [F1] and [F2]. For each $i$, let $S_i=S_n-y_i{y}_i^{\mathsf{T}}\mathrm{/}n$, $Q_i=(S_i-zI{)}^{-1}$ and ${\beta }_i=n^{-1}{y}_i^{\mathsf{T}}{Q}_i{y}_i$. The matrices $S_n,S_i$ are nonnegative symmetric, so their resolvents exist; diagonalizing $S_i$ expresses ${\beta }_i$ as a sum of nonnegative weights times $(\lambda -z{)}^{-1}$ with $\lambda \ge 0$. Thus $\mathrm{Im}{\beta }_i\ge 0$ and $\mathrm{Im}(z{\beta }_i)\ge 0$, since $\mathrm{Im}[z\mathrm{/}(\lambda -z)]=\eta \lambda \mathrm{/}\mathrm{\mid }\lambda -z{\mathrm{\mid }}^2\ge 0$. Consequently $\mathrm{Im}[z(1+{\beta }_i)]\ge \eta$, so $\mathrm{\mid }1+{\beta }_i\mathrm{\mid }\ge \eta \mathrm{/}\mathrm{\mid }z\mathrm{\mid }>0$. Applying [F4] with $x=y_i\mathrm{/}\sqrt{n}$ gives $Q=Q_i-Q_i{y}_i{y}_i^{\mathsf{T}}{Q}_i\mathrm{/}[n(1+{\beta }_i)]$, so $n^{-1}{y}_i^{\mathsf{T}}Q{y}_i={\beta }_i\mathrm{/}(1+{\beta }_i)$. Since $S_{n}Q=I+zQ$, taking normalized traces yields $1+z{m}_n=\frac{1}{p_n}{\sum }_{i=1}^n\frac{{\beta }_i}{1+{\beta }_i}\mathrm{.}$

Put ${\gamma }_n=p_n\mathrm{/}n$, $v_n={\gamma }_n{m}_n=n^{-1}\mathrm{tr}Q$, and ${\mu }_4=\mathbb{E}{\xi }^4$ for the common entry law. Both ${\beta }_i$ and $v_n$ belong to the cone ${\mathcal{C}}_z=\{w:\mathrm{Im}w\ge 0,\mathrm{Im}(zw)\ge 0\}$: for $v_n$ use the same eigenvalue calculation on $Q$. For every $w\in {\mathcal{C}}_z$, $\mathrm{Im}[z(1+w)]\ge \eta$, so $\mathrm{\mid }1+w{\mathrm{\mid }}^{-1}\le \mathrm{\mid }z\mathrm{\mid }\mathrm{/}\eta$. Conditional on $Q_i$, row $y_i$ is independent of it by [F12]. Applying [F3] conditionally and using $\mathrm{\parallel }{Q}_i{\mathrm{\parallel }}_F^2\le p_n\mathrm{/}{\eta }^2$ gives $\mathbb{E}\mid {\beta }_i-\frac{1}{n}\mathrm{tr}{Q}_i\mid \le \frac{\sqrt{C{p}_n}}{n\eta },C=\max \{{\mu }_4-1,2\}\mathrm{.}$ By [F4], $\mathrm{\mid }\mathrm{tr}{Q}_i-\mathrm{tr}Q\mathrm{\mid }\le 1\mathrm{/}\eta$, hence $\mathbb{E}\mathrm{\mid }{\beta }_i-v_n\mathrm{\mid }\le (\sqrt{C{p}_n}+1)\mathrm{/}(n\eta )$. For $f(w)=w\mathrm{/}(1+w)$ and $w,v\in {\mathcal{C}}_z$, $\mathrm{\mid }f(w)-f(v)\mathrm{\mid }=\mathrm{\mid }w-v\mathrm{\mid }\mathrm{/}(\mathrm{\mid }1+w\mathrm{\mid }\mathrm{\mid }1+v\mathrm{\mid })\le K_z\mathrm{\mid }w-v\mathrm{\mid }$, where $K_z=(\mathrm{\mid }z\mathrm{\mid }\mathrm{/}\eta {)}^2$. Therefore [step 1.1] implies $\mathbb{E}\mid 1+z{m}_n-\frac{m_n}{1+{\gamma }_n{m}_n}\mid \le K_z\frac{\sqrt{C{p}_n}+1}{p_n\eta }⟶0.$

Apply [F5] to the independent vectors $y_i\mathrm{/}\sqrt{n}$, whose rank-one sum is $S_n$. For the fixed $z$, $m_n-a_n\to 0$ almost surely, where $a_n=\mathbb{E}{m}_n$. Also $\mathbb{E}\mathrm{\mid }{m}_n-a_n\mathrm{\mid }\to 0$: for every $\epsilon >0$, $\mathbb{E}\mathrm{\mid }{m}_n-a_n\mathrm{\mid }\le \epsilon +(2\mathrm{/}\eta )\mathrm{Pr}(\mathrm{\mid }{m}_n-a_n\mathrm{\mid }>\epsilon )$, and the tail bound in [F5] makes the second term tend to zero. Define $g_n(w)=1+zw-w\mathrm{/}(1+{\gamma }_{n}w)$. The cone ${\mathcal{C}}_z$ is closed under nonnegative scaling and expectations, so it contains $m_n,a_n,{\gamma }_n{m}_n$ and ${\gamma }_n{a}_n$. For $w,v\in {\mathcal{C}}_z$, the denominator estimate of step 2.1 applied to ${\gamma }_{n}w,{\gamma }_{n}v$ gives $\mathrm{\mid }{g}_n(w)-g_n(v)\mathrm{\mid }\le (\mathrm{\mid }z\mathrm{\mid }+K_z)\mathrm{\mid }w-v\mathrm{\mid }$. Taking expectations in the estimate of step 2.1 gives $g_n(a_n)\to 0$.

The sequence $(a_n)$ is bounded by $1\mathrm{/}\eta$. Moreover $\mathrm{Im}(z{m}_n)=p_n^{-1}{\sum }_j\eta {\lambda }_j(S_n)\mathrm{/}\mathrm{\mid }{\lambda }_j(S_n)-z{\mathrm{\mid }}^2\ge 0$, so $\mathrm{Im}(1-{\gamma }_n-z-{\gamma }_{n}z{a}_n)\le -\eta$. The same closed-cone bound gives $\mathrm{\mid }1+{\gamma }_n{a}_n\mathrm{\mid }\ge \eta \mathrm{/}\mathrm{\mid }z\mathrm{\mid }$. Every convergent subsequence of $a_n$ therefore has a limit $a$, with $\mathrm{\mid }1+\gamma a\mathrm{\mid }\ge \eta \mathrm{/}\mathrm{\mid }z\mathrm{\mid }$, satisfying $1+za=a\mathrm{/}(1+\gamma a)$ and $\mathrm{Im}(1-\gamma -z-\gamma za)\le -\eta$. The two roots of the associated quadratic have corresponding values $D_j=1-\gamma -z-\gamma z{a}_j$ whose sum is $1-\gamma -z$, with imaginary part $-\eta$. Thus at most one root can have $\mathrm{Im}{D}_j\le -\eta$. By [A2] and [F6], that root is $m_{\gamma }(z)$. All subsequential limits are consequently $m_{\gamma }(z)$, so $a_n\to m_{\gamma }(z)$ and step 3.1 gives $m_n(z)\to m_{\gamma }(z)$ almost surely.

Perform steps 1.1–4.1 for each $z$ in the countable dense set $\{u+i\eta :u,\eta \in \mathbb{Q},\eta >0\}$ and intersect their probability-one events. Also let $T_n=(n{p}_n{)}^{-1}{\sum }_{i,j}{X}_{n,ij}^2$. Its mean is one and its variance is $({\mu }_4-1)\mathrm{/}(n{p}_n)$. Since $p_n\mathrm{/}n\to \gamma >0$, [F9] gives a summable bound $\mathrm{Pr}(\mathrm{\mid }{T}_n-1\mathrm{\mid }>\epsilon )\le ({\mu }_4-1)\mathrm{/}({\epsilon }^{2}n{p}_n)$ for every $\epsilon >0$; [F10] over positive rational $\epsilon$ yields $T_n\to 1$ almost surely.

On this probability-one event, $\int xd{\mu }_n(x)=T_n$ and ${\mu }_n$ is supported on $[0,\mathrm{\infty })$ by [F1]. For every $R>0$, ${\mu }_n((R,\mathrm{\infty }))\le T_n\mathrm{/}R$, so the sequence is tight. By [A1] and [F8], every subsequence has a further weakly convergent subsequence with limit $\nu$. Along it, [F11] applied to $x\mapsto (x-z{)}^{-1}$ shows $m_{\nu }(z)=m_{\gamma }(z)$ on the dense set from step 5.1. The transforms are continuous because $\mathrm{\mid }{m}_{\nu }(z)-m_{\nu }(w)\mathrm{\mid }\le \mathrm{\mid }z-w\mathrm{\mid }\mathrm{/}(\mathrm{Im}z\mathrm{Im}w)$ for $z,w\in {\mathbb{C}}^{+}$; equality therefore extends to all $z\in {\mathbb{C}}^{+}$.

By [A2] and [F7], every weak subsequential limit $\nu$ in step 6.1 equals ${\mu }_{\gamma }$. If ${\mu }_n$ did not converge weakly to ${\mu }_{\gamma }$, some bounded continuous test function and a subsequence would have integrals separated from its ${\mu }_{\gamma }$ integral by a fixed positive amount; tightness would give a further weakly convergent subsequence, contradicting its limit ${\mu }_{\gamma }$. Therefore ${\mu }_n\Rightarrow {\mu }_{\gamma }$ almost surely.