Studentized permutation tests under weak parameter nulls — Examples

The kernel is symmetric in $(x_1,x_2)$ and in $(y_1,y_2)$, and exchanging the two blocks changes its sign. By independence within each product block, [F1] gives $\theta =(E_{P}X{)}^2-(E_{Q}Y{)}^2=(1\mathrm{/}2{)}^2-(-1\mathrm{/}2{)}^2=0$.

Under the mixed assignment $(X_1,X_2)\sim P^{\otimes 2}$ and $(Y_1,Y_2)\sim P\otimes Q$, independence gives $Eh=(E_{P}X{)}^2-(E_{P}X)(E_{Q}Y)=1\mathrm{/}4-(-1\mathrm{/}4)=1\mathrm{/}2\mathrm{\ne }0$. Thus the weak null does not force all mixed-assignment means to vanish.

Every law in $\{P,Q\}$ is bounded and assigns positive mass both to zero and to a nonzero value. Therefore every product of two independently assigned variables has positive probability at zero and positive probability at a nonzero value, so it has positive finite variance. For any two assignments, $h$ is the difference of two independent such products; its second moment is finite and strictly positive.

The direct projections are $g_X(x)=x\mathrm{/}2-1\mathrm{/}4$ and $g_Y(y)=1\mathrm{/}4+y\mathrm{/}2$. Each has mean zero and variance $3\mathrm{/}16$ under its respective law, so both direct first-projection variances are positive. At mixture weight $p=1\mathrm{/}4$, $\bar{P}=pP+(1-p)Q$ has mean $-1\mathrm{/}4$ and positive variance; its pooled projection is $g_{\bar{P}}(z)=(-1\mathrm{/}4)(z+1\mathrm{/}4)$, which also has positive variance.

The example satisfies block symmetry and antisymmetry, the weak parameter null, all mixed second-moment requirements, and positive direct and pooled first-projection variances. Step 1.2 nevertheless gives a mixed assignment with nonzero mean, refuting the Statement. This is a gap in the cited coupling route, not a counterexample to the broader asymptotic theorem by itself.

The kernel is symmetric within each block and changes sign when the blocks are exchanged. Since each Rademacher variable has mean zero, [F1] gives $\theta =E[X_1{X}_2]-E[Y_1{Y}_2]=0$.

The products $A=X_1{X}_2$ and $B=Y_1{Y}_2$ are independent Rademacher variables. Hence $E[h^2]=E[(A-B{)}^2]=E[A^2]+E[B^2]-2E[AB]=2$. Because $P=Q$, this same strictly positive finite second moment holds for every assignment of the four kernel arguments.

For every fixed $x$, $E[h(x,X_2;Y_1,Y_2)]=xE[X_2]-E[Y_1]E[Y_2]=0$; for every fixed $y$, $E[h(X_1,X_2;y,Y_2)]=E[X_1]E[X_2]-yE[Y_2]=0$. Thus $g_X=g_Y=0$ and both first-projection variances vanish.

By [F2], $E[U_{m,n}^2]=O(N^{-2})$, so $\sqrt{m}{U}_{m,n}\to 0$ in $L^2$ when $m\mathrm{/}N\to p\in (0,1)$. The positive kernel second moment therefore does not provide a nondegenerate normal pivot; the explicit positive first-projection assumptions in the asymptotic theorem cannot be dropped.

Both means are zero. The variances are ${\sigma }_P^2=1$ and ${\sigma }_Q^2=2$, so [F1] gives $\theta =0$. The kernel is antisymmetric, and under every P/Q assignment its expectation is the difference of two zero means, hence zero. Its second moment is the sum of the two positive variances, so it is finite and strictly positive for every assignment.

For $k$ observations $W_1,\dots ,W_k$, the sample variance is ${\widehat{\sigma }}_k^2=\frac{k}{k-1}\{k^{-1}{\sum }_i{W}_i^2-(k^{-1}{\sum }_i{W}_i{)}^2\}$ for $k\ge 2$. Under either finite-support law $P$ or $Q$, Chebyshev [F6] applied to the averages of $W_i$ and $W_i^2$ gives convergence in probability to their expectations, so ${\widehat{\sigma }}_m^2\to 1$ and ${\widehat{\sigma }}_n^2\to 2$. Under IID ${\bar{P}}_N$ rows, $\mathrm{\mid }{W}_i\mathrm{\mid }\le 2$ uniformly in $N$, so the variances of both averages are bounded by constants divided by their row sizes; [F6] again gives convergence to their row expectations. These means are $0$ and $p_N(1)+(1-p_N)(2)\to 7\mathrm{/}4$, so both mixture-sample variance estimates converge to $7\mathrm{/}4$.

From [F2], $g_X(x)=x$ and $g_Y(y)=-y$, so their variances are $1$ and $2$. The pooled mixture has mean zero and variance ${\sigma }_{{\bar{P}}_N}^2=p_N{\sigma }_P^2+(1-p_N){\sigma }_Q^2=2-p_N\to \frac{7}{4}\mathrm{.}$ It is positive.

Under AC [A1], the hypotheses of [F4] now hold, including positive direct and pooled projection variances, so the studentized statistic has standard-normal true-sampling and permutation limits. This example has a weak parameter null while $P\mathrm{\ne }Q$.

Under AC [A1], step 1.2 and [F4] give $\sqrt{m}{V}_{m,n}^{\mathrm{S}\mathrm{E}}{\to }_P\sqrt{1+\frac{p}{1-p}2}=\sqrt{5\mathrm{/}3}$ and $S_{m,n}\Rightarrow N(0,1)$ under the true sampling law. Thus $\sqrt{m}{U}_{m,n}=(\sqrt{m}{V}_{m,n}^{\mathrm{S}\mathrm{E}})S_{m,n}\Rightarrow N(0,5\mathrm{/}3)$ by Slutsky. The raw permutation limit instead has variance $\frac{{\sigma }_{\bar{P}}^2}{1-p}=\frac{7\mathrm{/}4}{3\mathrm{/}4}=\frac{7}{3}$ by [F3]. To specify the resulting two-sided permutation test, let ${\widehat{R}}_N(t)=P_{\pi }(\sqrt{m}{U}_{m,n}^{\pi }\le t\mid Z)$ and $q_N(u)=\inf \{t:{\widehat{R}}_N(t)\ge u\}$. For $\alpha \in (0,1)$, reject when $\sqrt{m}{U}_{m,n}<q_N(\alpha \mathrm{/}2)$ or $\sqrt{m}{U}_{m,n}>q_N(1-\alpha \mathrm{/}2)$. The conditional CDF convergence in [F3] to the continuous strictly increasing law $N(0,7\mathrm{/}3)$ implies $q_N(\alpha \mathrm{/}2){\to }_P-z_{1-\alpha \mathrm{/}2}\sqrt{7\mathrm{/}3}$ and $q_N(1-\alpha \mathrm{/}2){\to }_P{z}_{1-\alpha \mathrm{/}2}\sqrt{7\mathrm{/}3}$. Since the true-sampling statistic converges to $N(0,5\mathrm{/}3)$ and this limit has no atoms at either limiting critical value, the rejection probability tends to $P(\mathrm{\mid }N(0,5\mathrm{/}3)\mathrm{\mid }>z_{1-\alpha \mathrm{/}2}\sqrt{7\mathrm{/}3})=2(1-\mathrm{\Phi }\left(z_{1-\alpha \mathrm{/}2}\sqrt{\frac{7}{5}}\right))<\alpha ,$ where $z_{1-\alpha \mathrm{/}2}>0$ is the standard-normal quantile. Thus the unstudentized equal-tail permutation test is conservative in this example, and studentization corrects this variance mismatch.

In both cases, $E_{P}X=E_{Q}Y=0$, so [F1] gives $\theta =0$. The kernel is antisymmetric, and for either assignment of P and Q to its two arguments its mean is $0-0=0$. All mixed second moments are finite and positive because the two laws have finite positive variances.

In the first case $P=Q$, so the joint law of the pooled IID sample is invariant under coordinate permutations; [F2] gives finite-sample superuniformity and exactness after tie randomization. In the second case $P\mathrm{\ne }Q$ (for example, $P(X=0)=0$ and $Q(Y=0)=1\mathrm{/}2$), so swapping coordinates across groups does not preserve the sampling law. The parameter null alone therefore does not supply the finite-sample exchangeability premise of [F2].

Under AC [A1], in each case the two population variances are positive, and the pooled-mixture variance is positive. For any bounded law $R$, [F5] applied to $W$ and $W^2$ shows that their sample averages converge to $E_{R}W$ and $E_R{W}^2$; the identity $\stackrel{}{\mathrm{Var}}(W)=\frac{k}{k-1}\{\overline{W^2}-{\bar{W}}^2\}$ then proves sample-variance consistency. For the triangular pooled laws ${\bar{P}}_N$, the same Chebyshev bounds are uniform because the support stays in $\{-2,-1,0,1,2\}$, and their variances converge to $1$ in the first case and $\frac{3}{2}$ in the second. Thus the estimators in [F4] satisfy the conditions of [F3]. In the second case, studentization has the asymptotic standard-normal calibration even though finite-sample group exchangeability is absent.

The kernel is antisymmetric. For $X\sim P$, its conditional mean against $Q$ is $g_X(0)=1\mathrm{/}2$ and $g_X(2)=-1\mathrm{/}2$, so $\theta (P,Q;h)=\frac{1}{2}(1\mathrm{/}2)+\frac{1}{2}(-1\mathrm{/}2)=0$. For the reverse P/Q assignment, the mean is $-\theta =0$ by antisymmetry. For P/P and Q/Q assignments, the two arguments are IID with the same law, so swapping them leaves their joint law unchanged while negating $h$; each such mean is therefore zero. Also, ${\sigma }_X^2=E_P[g_X^2]=1\mathrm{/}4$. For $Y\sim Q$, the conditional means are $g_Y(-1)=-1$, $g_Y(1)=0$, and $g_Y(3)=1$, so $E_Q{g}_Y=0$ and ${\sigma }_Y^2=1\mathrm{/}2$. Every P/Q or Q/P pair is distinct, so its second moment is $1$. Under P/P the second moment is the probability of unequal values, $1\mathrm{/}2$; under Q/Q it is $1-\{(1\mathrm{/}4{)}^2+(1\mathrm{/}2{)}^2+(1\mathrm{/}4{)}^2\}=5\mathrm{/}8$. All four assignment-wise second moments are positive and finite.

For $p=1\mathrm{/}2$, the pooled law $\bar{P}=(P+Q)\mathrm{/}2$ assigns masses $1\mathrm{/}8,1\mathrm{/}4,1\mathrm{/}4,1\mathrm{/}4,1\mathrm{/}8$ to $-1,0,1,2,3$. The pooled projection $g_{\bar{P}}(z)=E_{\bar{P}}h(z,Y)$ has values $7\mathrm{/}8,1\mathrm{/}2,0,-1\mathrm{/}2,-7\mathrm{/}8$ at those points. Its mean is zero and ${\sigma }_{\bar{P}}^2=E_{\bar{P}}[g_{\bar{P}}^2]=\frac{49}{256}+\frac{1}{8}=\frac{81}{256}>0.$ For general $p_N\to 1\mathrm{/}2$, the same finite support probabilities and projection values vary continuously with $p_N$, so the pooled projection variance tends to this positive value.

For $x$ in the finite support of $P$, set ${\widehat{g}}_{X,i}=n^{-1}{\sum }_{j}h(X_i,Y_j)-U_{m,n}$; for $y$ in the finite support of $Q$, set ${\widehat{g}}_{Y,j}=m^{-1}{\sum }_{i}h(X_i,Y_j)-U_{m,n}$. Define ${\widehat{\sigma }}_m^2$ and ${\widehat{\sigma }}_n^2$ as the sample variances of these estimated projection values. They are nonnegative measurable statistics. In either sampling regime, each empirical frequency on the finite support converges to its row probability by [F8], uniformly over that finite support. Since $h$ is bounded, the finite sums expressing $U_{m,n}$ and each empirical section mean then converge to their corresponding product-law means. Under the true samples these limits are $\theta =0$, $E_{Q}h(x,Y)$, and $E_{P}h(X,y)$; under two IID ${\bar{P}}_N$ samples the limits are $0$ and the pooled section integrals, because the pooled parameter is zero by antisymmetry. Subtracting $U_{m,n}$ therefore makes the estimated projections converge uniformly on each support to their centered section-integral values. The sample averages of their squares converge to the corresponding projection second moments, again by finite support and [F8]; their sample means are zero by construction. Thus the variance estimates converge to ${\sigma }_X^2,{\sigma }_Y^2$ under the true laws and to ${\sigma }_{{\bar{P}}_N}^2\to 81\mathrm{/}256$ under the pooled-mixture rows.

For each support point $z\in \{-1,0,1,2,3\}$, the empirical frequency of $z$ in an IID ${\bar{P}}_N$ group converges to ${\bar{P}}_N(\{z\})$ by the bounded Bernoulli Chebyshev estimate [F8]. Applying [F9] to this single-group statistic and taking a finite union over points and both groups proves that the two permuted groups' empirical frequencies converge jointly to the corresponding pooled frequencies. Their conditional failure probabilities given the pooled sample tend to zero in probability, by Markov's inequality applied to the unconditional failure probabilities. The two variance estimates, though depending on both groups, are finite polynomial expressions in those frequencies times $m\mathrm{/}(m-1)$ or $n\mathrm{/}(n-1)$; the kernel is bounded and the finite row probabilities converge. Therefore both estimates converge to ${\sigma }_{{\bar{P}}_N}^2$ in conditional permutation probability, in probability over the data. This directly verifies the joint-estimator premise of [F5].

Under AC [A1], steps 1.1, 1.2, 2.1, 3.1 verify assignment-wise centering, finite positive mixed second moments, positive direct and pooled projection variances, original-sample consistency and conditional permutation consistency of the joint estimates. Thus [F5] applies: both the true-sampling pivot and its conditional permutation distribution converge to the standard normal despite $P\mathrm{\ne }Q$.

The two population means are zero and the two variances are one, so $\theta (P,Q;h)=0$ by [F1]. The kernel is antisymmetric, and both P/Q assignments have mean zero. For each assignment, $h^2$ equals $0$ on the two equal-sign pairs and $4$ on the two opposite-sign pairs, each of probability $1\mathrm{/}4$, so $E[h^2]=2$. The first projections are $g_X(x)=E[x-Y]-\theta =x$ and $g_Y(y)=E[X-y]-\theta =-y$; each has variance $1$.

Under AC [A1], the sample variances are measurable finite-sample statistics. Here $W_i^2=1$ exactly and, by [F4], the sample mean $\bar{W}$ converges in probability to $0$; the identity $\stackrel{}{\mathrm{Var}}(W)=\frac{k}{k-1}(1-{\bar{W}}^2)$ then gives consistency to $1$. The pooled mixture is the same Rademacher law, so this consistency holds in both permuted groups and the positive pooled projection variance is also $1$. Thus [F6] applies and gives the standard-normal limits for $S_{m,n}$ itself and its conditional permutation distribution.

For every fixed $M>0$ and every $\epsilon >0$, eventually $M\mathrm{/}\sqrt{m}<\epsilon$, hence $P(\mathrm{\mid }\sqrt{m}{S}_{m,n}\mathrm{\mid }\le M)\le P(\mathrm{\mid }{S}_{m,n}\mathrm{\mid }\le \epsilon )\mathrm{.}$ By the true-sampling convergence in step 1.2, the limsup of the right side is at most $\mathrm{\Phi }(\epsilon )-\mathrm{\Phi }(-\epsilon )$. Letting $\epsilon \downarrow 0$ proves $P(\mathrm{\mid }\sqrt{m}{S}_{m,n}\mathrm{\mid }\le M)\to 0$; thus $\mathrm{\mid }\sqrt{m}{S}_{m,n}\mathrm{\mid }\to \mathrm{\infty }$ in probability.

Under AC [A1], let $F_N^{\pi }$ be the conditional permutation CDF of $S_{m,n}^{\pi }$. For fixed $\epsilon >0$, eventually $M\mathrm{/}\sqrt{m}<\epsilon$, so $P_{\pi }(\mathrm{\mid }\sqrt{m}{S}_{m,n}^{\pi }\mathrm{\mid }\le M\mid Z)\le F_N^{\pi }(\epsilon )-F_N^{\pi }(-\epsilon -),$ where $F_N^{\pi }(-\epsilon -)$ denotes the left limit of the conditional CDF. By [F6], the right side converges in probability to $\mathrm{\Phi }(\epsilon )-\mathrm{\Phi }(-\epsilon )$. Letting $\epsilon \downarrow 0$ shows that the conditional probability of any fixed bounded interval tends to zero in probability. The standard-normal pivot is $S_{m,n}$; the extra $\sqrt{m}$ factor is erroneous when the denominator is already a standard error.