Uniform concentration of Gaussian sample partial correlations

Put $Y=(X_i,X_j)$ and $Z=X_S$. For nonempty $S$, define $B={\mathrm{\Sigma }}_{Y,S}{\mathrm{\Sigma }}_{S,S}^{-1}$ and $U=Y-{\mu }_Y-B(Z-{\mu }_S)$. Covariance algebra gives $\mathrm{Cov}(U)=K_{ij\mid S}$ and $\mathrm{Cov}(U,Z)=0$. The vector $(U,Z)$ is jointly Gaussian by [F5], since every scalar projection is an affine scalar projection of $X$. Its covariance is block diagonal and positive definite by [F3] and positive definiteness of ${\mathrm{\Sigma }}_{S,S}$. Its density from [F4] factors into the Gaussian densities of $U$ and $Z$: both the determinant and the quadratic form split by blocks. Integrating over rectangles proves independence of $U$ and $Z$. For empty $S$, take $U=Y-{\mu }_Y$ with covariance $K_{ij\mid \mathrm{\varnothing }}$; independence from the empty conditioning tuple is automatic. Across observations the residual pairs are independent, and their entire sample is independent of the conditioning sample, because the joint observation densities multiply. The $S$ coordinates have a nondegenerate Gaussian density because their covariance is a positive-definite principal submatrix. If $\mathrm{\mid }S\mathrm{\mid }=0$, the intercept design has rank one. If $s=\mathrm{\mid }S\mathrm{\mid }>0$, successive independent sample points in ${\mathbb{R}}^s$ are affinely independent until there are $s+1$: after $r\le s$ affinely independent points, their affine span has dimension $r-1<s$, and the next point has a density, so lies in that span with probability zero by [F4]. Thus the intercept and $S$ design has rank $s+1$ almost surely. Conditional on that design, an orthonormal basis of its orthogonal complement has dimension $m=n-\mathrm{\mid }S\mathrm{\mid }-1$. In that basis the projected residual pairs are $m$ independent copies of this Gaussian residual; standardizing the two coordinates does not change their sample correlation. Thus the sample partial correlation has the same distribution as $R_m=\frac{{\sum }_{r=1}^m{U}_r{V}_r}{({\sum }_{r=1}^m{U}_r^2{)}^{1\mathrm{/}2}({\sum }_{r=1}^m{V}_r^2{)}^{1\mathrm{/}2}},m=n-\mathrm{\mid }S\mathrm{\mid }-1,$ for $m$ independent copies $(U_r,V_r)$ of the standardized pair, whose correlation is $\rho ={\rho }_{ij\mid S}$ by [F1]. The hypothesis gives $m\ge 3$. Their covariance block is positive definite by [F3], so the first residual sample column is nonzero almost surely, and conditional on that column the second has a density in ${\mathbb{R}}^m$. Its probability of lying in the first column's one-dimensional span is zero by [F4], since $m\ge 2$. Thus the two residual columns have rank two almost surely. Together with the full-rank intercept and $S$ design, this makes the centered sample covariance block on $\{i,j\}\cup S$ positive definite. Therefore $R_m\in (-1,1)$ almost surely and the Fisher transform is defined.

Represent $U_r=X_r$ and $V_r=\rho X_r+\sqrt{1-{\rho }^2}{E}_r$ with independent standard normal vectors $X,E\in {\mathbb{R}}^m$. Put $A=\mathrm{\parallel }X\mathrm{\parallel }$, $Z=⟨E,X\mathrm{/}A⟩$, and $W=\mathrm{\parallel }E-ZX\mathrm{/}A\mathrm{\parallel }$. Rotational invariance of the standard Gaussian density implies that $A^2\sim {\chi }_m^2$, $Z\sim N(0,1)$, and $W^2\sim {\chi }_{m-1}^2$ are independent. Direct substitution gives $R_m=\frac{q}{\sqrt{q^2+W^2}},q=\frac{\rho A}{\sqrt{1-{\rho }^2}}+Z,$ and hence $g(R_m)=\mathrm{arsinh}(q\mathrm{/}W)$. Rotational invariance follows directly from the standard Gaussian density, and it makes $A,Z,W$ independent. Also $\mathrm{arsinh}(\rho \mathrm{/}\sqrt{1-{\rho }^2})=g(\rho )$.

Let $B=M\mathrm{/}\sqrt{1-M^2}$ and $L=4B+2$. On the event $\mathrm{\mid }A\mathrm{/}\sqrt{m}-1\mathrm{\mid }\le \delta ,\mathrm{\mid }W\mathrm{/}\sqrt{m}-1\mathrm{\mid }\le \delta ,\mathrm{\mid }Z\mathrm{\mid }\mathrm{/}\sqrt{m}\le \delta ,$ where $0<\delta \le 1\mathrm{/}2$, one has $\mid \frac{q}{W}-\frac{\rho }{\sqrt{1-{\rho }^2}}\mid \le \frac{(2B+1)\delta }{1-\delta }\le L\delta \mathrm{.}$ Since $\mathrm{\mid }{\mathrm{arsinh}}^{\mathrm{\prime }}(x)\mathrm{\mid }=1\mathrm{/}\sqrt{1+x^2}\le 1$, step 2.1 yields $\mathrm{\mid }g(R_m)-g(\rho )\mathrm{\mid }\le L\delta$.

For $Q\sim {\chi }_k^2$, its moment generating function, obtained by multiplying the $k$ one-dimensional Gaussian integrals, is $\mathbb{E}{e}^{sQ}=(1-2s{)}^{-k\mathrm{/}2}$ for $s<1\mathrm{/}2$. For $\delta >0$, Markov's inequality optimized at $s=\delta \mathrm{/}[2(1+\delta )]$ gives $\mathbb{P}(Q\mathrm{/}k\ge 1+\delta )\le e^{-k(\delta -\log (1+\delta ))\mathrm{/}2}$. For $0<\delta <1$, the negative choice $s=-\delta \mathrm{/}[2(1-\delta )]$ gives $\mathbb{P}(Q\mathrm{/}k\le 1-\delta )\le e^{-k(-\delta -\log (1-\delta ))\mathrm{/}2}$. Put ${\delta }_{+}=2u+u^2\le 3$ and ${\delta }_{-}=2u-u^2\ge u$. The inequalities $\delta -\log (1+\delta )={\int }_0^{\delta }x\mathrm{/}(1+x)dx\ge {\delta }^2\mathrm{/}8$ for $0\le \delta \le 3$ and $-\delta -\log (1-\delta )\ge {\delta }^2\mathrm{/}2$ imply $\mathbb{P}(\mathrm{\mid }\sqrt{Q\mathrm{/}k}-1\mathrm{\mid }>u)\le 2e^{-k{u}^2\mathrm{/}8}(0<u<1)\mathrm{.}$ Applying this to $A^2$ and $W^2$, and using $\mathrm{\mid }\sqrt{(m-1)\mathrm{/}m}-1\mathrm{\mid }\le 1\mathrm{/}m\le \delta \mathrm{/}2$, shows that $\mathrm{\mid }W\mathrm{/}\sqrt{m}-1\mathrm{\mid }>\delta$ implies $\mathrm{\mid }W\mathrm{/}\sqrt{m-1}-1\mathrm{\mid }>\delta \mathrm{/}2$. Thus for $2\mathrm{/}m\le \delta \le 1\mathrm{/}2$, $\mathbb{P}(\mathrm{\mid }A\mathrm{/}\sqrt{m}-1\mathrm{\mid }>\delta \text{ or }\mathrm{\mid }W\mathrm{/}\sqrt{m}-1\mathrm{\mid }>\delta )\le 4e^{-m{\delta }^2\mathrm{/}64}\mathrm{.}$ The standard normal moment generating function $\mathbb{E}{e}^{sZ}=e^{s^2\mathrm{/}2}$, with Markov's inequality optimized at $s=\delta \sqrt{m}$, gives $\mathbb{P}(\mathrm{\mid }Z\mathrm{\mid }\mathrm{/}\sqrt{m}>\delta )\le 2e^{-m{\delta }^2\mathrm{/}2}$. For $0<\delta <2\mathrm{/}m$, the trivial bound $1\le 6e^{-m{\delta }^2\mathrm{/}64}$ applies. Thus the complement of the event in step 3.1 has probability at most $6e^{-m{\delta }^2\mathrm{/}64}$ for every $0<\delta \le 1\mathrm{/}2$.

Set $\delta =t\mathrm{/}L$. Since $L\ge 2$ and $0<t\le 1$, step 3.1 applies and its complement contains the event in the Statement. Step 4.1 therefore bounds that event by $6\exp (-m{t}^2\mathrm{/}(64L^2))$. Taking $C_M=6$ and $c_M=(64L^2{)}^{-1}$ proves the claim.