Exact permutation tests and random-permutation Monte Carlo — Examples

All four masses are nonnegative and sum to $(q+(1-q){)}^2=1$, so [F4] makes them a finite probability law. The swap fixes $(0,0)$ and $(1,1)$ and exchanges $(0,1)$ with $(1,0)$, which have equal mass. Thus $P_q(s^{-1}A)=P_q(A)$ for every $A\subseteq \mathrm{\Omega }$, and the group action is invariant for every $q\in [0,1]$.

The subset $S$ is uniform and independent of $X$ by the Example, so it satisfies the without-replacement design premise in [F5]. In the $\mathbb{Z}\mathrm{/}6\mathbb{Z}$ example, addition by any fixed group element permutes the six equally likely outcomes, so the null law is invariant. If $X=0$, only the identity in $W$ has score at least $T(X)=1$, and the without-replacement p-value is $1\mathrm{/}3$. If $X\mathrm{\ne }0$, then $T(X)=0$ and every element of $W$ has score at least zero, so the p-value is $1$. Its distribution is therefore $1\mathrm{/}3$ with probability $1\mathrm{/}6$ and $1$ with probability $5\mathrm{/}6$; its CDF is superuniform.

At $(0,0)$ and $(1,1)$, both transformed scores are $0$, giving $p_{\mathrm{o}\mathrm{r}\mathrm{b}}=1$ and $p_{\mathrm{t}\mathrm{i}\mathrm{e}}=U$. At $(1,0)$ the two scores are $1,-1$, giving $p_{\mathrm{o}\mathrm{r}\mathrm{b}}=1\mathrm{/}2$ and $p_{\mathrm{t}\mathrm{i}\mathrm{e}}=U\mathrm{/}2$. At $(0,1)$ they are $-1,1$, giving $p_{\mathrm{o}\mathrm{r}\mathrm{b}}=1$ and $p_{\mathrm{t}\mathrm{i}\mathrm{e}}=(1+U)\mathrm{/}2$.

Put $a=q(1-q)$. Then $p_{\mathrm{o}\mathrm{r}\mathrm{b}}=1\mathrm{/}2$ with probability $a$ and equals $1$ otherwise. Its CDF is $0$ for $u<1\mathrm{/}2$, $a\le 1\mathrm{/}4\le u$ for $1\mathrm{/}2\le u<1$, and $1$ at $u=1$, so it is superuniform. For $0\le u\le 1\mathrm{/}2$, the tie-randomized CDF is $2au+(1-2a)u=u$: the $(1,0)$ branch contributes $2au$, the $(0,1)$ branch contributes zero except at a probability-zero endpoint, and the tied branches contribute $(1-2a)u$. For $1\mathrm{/}2\le u\le 1$, it is $a+a(2u-1)+(1-2a)u=u$. Thus $p_{\mathrm{t}\mathrm{i}\mathrm{e}}$ is uniform, in agreement with [F3].

In the weak mean-difference example, $E[X]=E[Y]=0$. But the coordinate swap sends $(1,0)$ to $(0,1)$, and the product law assigns these points probabilities $P(X=1)Q(Y=0)=1\mathrm{/}4$ and $P(X=0)Q(Y=1)=0$, respectively. Hence this weak null does not make the pooled law invariant under the swap, so the invariant-null premise of [F3] cannot be inferred from equality of means.

The assignment weights are nonnegative and sum to one. Under the sharp null the same fixed response vector $y$ is available at both assignments, so the two score values in the Example are imputable before observing $Z$. The published weighted randomization theorem [F3] therefore applies to this schedule and this predeclared ordering.

If $Z=a$, the only assignment with score at least $1$ is $a$, so $p_{\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{d}}(a,y)=0.9$. If $Z=b$, both assignments have score at least $0$, so $p_{\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{d}}(b,y)=0.9+0.1=1$.

The p-value is $0.9$ with probability $0.9$ and $1$ with probability $0.1$. Its CDF is $0$ for $u<0.9$, $0.9\le u$ for $0.9\le u<1$, and $1$ at $u=1$, hence it is superuniform. The assignment law is not invariant under the swap $a\leftrightarrow b$, since the two assignment masses differ, so the group-invariance theorem [F4] is not the justification here. This example uses the Fisher sharp null and makes no claim under the weaker average-effect null [F5].

If $X=0$, both transformations have score at least $T(X)=0$, so ${\tilde{p}}_1=1$. If $X=1$, the identity gives an indicator of $1$ and the swap gives an indicator of $0$. Therefore ${\tilde{p}}_1=0$ exactly on $\{X=1,g_1=s\}$. By [F5], [F6], and independence [F7], this event has probability $(1\mathrm{/}2)(1\mathrm{/}2)=1\mathrm{/}4$.

Consequently $P({\tilde{p}}_1\le 0)=1\mathrm{/}4>0$, contradicting the necessary superuniform bound [F4]. Thus the uncorrected fraction can equal zero and is anti-conservative at cutoff $0$.

In the same witness the corrected value is $p_{+}=(1+\mathbf{1}\{T(g_1\cdot X)\ge T(X)\})\mathrm{/}2$, so it equals $1\mathrm{/}2$ with probability $1\mathrm{/}4$ and $1$ with probability $3\mathrm{/}4$. Its CDF is $0$ below $1\mathrm{/}2$, $1\mathrm{/}4\le u$ for $1\mathrm{/}2\le u<1$, and $1$ at $u=1$, consistent with the general identity-corrected result [F3].