Gaussian sample covariance converges in operator norm when dimension is little-o of sample size

Fix $k$, abbreviate $d=d_k$, $n=n_k$, and take a fixed unit $v\in {\mathbb{R}}^d$. The row products $Z_i=(G_{k}v{)}_i$ are independent $N(0,1)$ by [A1]. From [F1], $\mathbb{E}{e}^{t{Z}_i^2}=(1-2t{)}^{-1\mathrm{/}2}$ for $t<1\mathrm{/}2$; applying [F2] at $t=\delta \mathrm{/}(2(1+\delta ))$ gives $\mathbb{P}(n^{-1}{\sum }_i{Z}_i^2\ge 1+\delta )\le e^{-n[\delta -\log (1+\delta )]\mathrm{/}2}\le e^{-n{\delta }^2\mathrm{/}8}$ for $0<\delta \le 1$, since $\delta -\log (1+\delta )={\int }_0^{\delta }t\mathrm{/}(1+t)dt\ge {\delta }^2\mathrm{/}4$. For $0<\delta <1$, applying [F2] at $u=\delta \mathrm{/}(2(1-\delta ))$ gives $\mathbb{P}(n^{-1}{\sum }_i{Z}_i^2\le 1-\delta )\le e^{-n[-\delta -\log (1-\delta )]\mathrm{/}2}\le e^{-n{\delta }^2\mathrm{/}4}$, since $-\delta -\log (1-\delta )={\int }_0^{\delta }t\mathrm{/}(1-t)dt\ge {\delta }^2\mathrm{/}2$; at $\delta =1$ this event requires all $Z_i=0$ and has probability zero. Thus $\mathbb{P}(\mathrm{\mid }{v}^{\mathsf{T}}(G_k^{\mathsf{T}}{G}_k\mathrm{/}n-I_d)v\mathrm{\mid }>\delta )\le 2e^{-n{\delta }^2\mathrm{/}8}$.

Fix $0<\epsilon <1$ and greedily add unit vectors at distance greater than $1\mathrm{/}4$ from those already chosen whenever possible. The open balls of radius $1\mathrm{/}8$ around a finite separated set are disjoint and lie in the radius-$9\mathrm{/}8$ ball, so volume comparison gives $\mathrm{\mid }\mathcal{N}\mathrm{\mid }(1\mathrm{/}8{)}^d\le (9\mathrm{/}8{)}^d$; hence the process terminates within $9^d$ choices and its maximal set is a $1\mathrm{/}4$-net. Applying step 1.1 with $\delta =\epsilon \mathrm{/}2$ and taking a union bound yields $\mathbb{P}({\max }_{v\in \mathcal{N}}\mathrm{\mid }{v}^{\mathsf{T}}(G_k^{\mathsf{T}}{G}_k\mathrm{/}n-I_d)v\mathrm{\mid }>\epsilon \mathrm{/}2)\le 2\exp (d\log 9-n{\epsilon }^2\mathrm{/}32)\to 0$ because $d\mathrm{/}n\to 0$.

For any symmetric $H$, [F3] gives $\mathrm{\parallel }H{\mathrm{\parallel }}_{\mathrm{o}\mathrm{p}}={\sup }_{\mathrm{\parallel }x{\mathrm{\parallel }}_2=1}\mathrm{\mid }{x}^{\mathsf{T}}Hx\mathrm{\mid }$. For each unit $x$, choose $v\in \mathcal{N}$ with $\mathrm{\parallel }x-v{\mathrm{\parallel }}_2\le 1\mathrm{/}4$. By [F4] and the operator-norm definition, $\mathrm{\mid }{x}^{\mathsf{T}}Hx-v^{\mathsf{T}}Hv\mathrm{\mid }=\mathrm{\mid }(x-v{)}^{\mathsf{T}}Hx+v^{\mathsf{T}}H(x-v)\mathrm{\mid }\le 2(1\mathrm{/}4)\mathrm{\parallel }H{\mathrm{\parallel }}_{\mathrm{o}\mathrm{p}}\mathrm{.}$ Hence $\mathrm{\parallel }H{\mathrm{\parallel }}_{\mathrm{o}\mathrm{p}}\le 2{\max }_{v\in \mathcal{N}}\mathrm{\mid }{v}^{\mathsf{T}}Hv\mathrm{\mid }$. On the complementary event in step 2.1, take $H=G_k^{\mathsf{T}}{G}_k\mathrm{/}n-I_d$ to obtain $\mathrm{\parallel }H{\mathrm{\parallel }}_{\mathrm{o}\mathrm{p}}\le \epsilon$. Since this holds for every $0<\epsilon <1$, larger tolerances follow by monotonicity, and the claimed convergence in probability follows.