Statistical tests, rejection regions, power functions and size

Let $(S,\mathcal{S})$ be a measurable space, the data space (Measurable spaces and measurable sets, Sigma-algebras); let $\mathrm{\Theta }$ be a nonempty set, the parameter set; let $({\mathbb{P}}_{\theta }{)}_{\theta \in \mathrm{\Theta }}$ be a family of probability measures on $(S,\mathcal{S})$ (Probability measures and probability spaces), the model family; and let ${\mathrm{\Theta }}_0$ be a nonempty subset of $\mathrm{\Theta }$, the null set. A test of the null set ${\mathrm{\Theta }}_0$ within the family $({\mathbb{P}}_{\theta }{)}_{\theta \in \mathrm{\Theta }}$ is a set $R\in \mathcal{S}$, the rejection region; the test rejects the null set on the event that the datum $s\in S$ lies in $R$, and retains it otherwise.

The power function of the test is $\beta :\mathrm{\Theta }⟶[0,1],\beta (\theta ):={\mathbb{P}}_{\theta }(R),$ and the size of the test is $\alpha :={\sup }_{\theta \in {\mathrm{\Theta }}_0}\beta (\theta )\in [0,1]\mathrm{.}$ The test has level ${\alpha }^{\mathrm{\prime }}$ when its size is at most ${\alpha }^{\mathrm{\prime }}$; the smallest such ${\alpha }^{\mathrm{\prime }}$ is the size.

Well-definedness and conventions

For each $\theta$ the value ${\mathbb{P}}_{\theta }(R)$ is defined because $R\in \mathcal{S}$ and ${\mathbb{P}}_{\theta }$ is a measure on $\mathcal{S}$. The set $\{\beta (\theta ):\theta \in {\mathrm{\Theta }}_0\}$ is nonempty because ${\mathrm{\Theta }}_0\mathrm{\ne }\mathrm{\varnothing }$, and it is bounded above by $1$; the least-upper-bound property of the complete ordered field of reals (Complete ordered field (least-upper-bound property)) therefore supplies its supremum $\alpha \in [0,1]$, uniquely by Complete ordered field (least-upper-bound property). The size is the supremum of the rejection probabilities over the null set for the specified rejection region and model family; no alternative set $\mathrm{\Theta }\setminus {\mathrm{\Theta }}_0$, no prior distribution on $\mathrm{\Theta }$, and no observed datum enter its definition.

The rejection region, the family $({\mathbb{P}}_{\theta })$ and the null set are data of the test, not consequences of it. Changing the rejection region or model family may change $\beta$ and $\alpha$; changing the null set may change $\alpha$ but leaves $\beta$ unchanged; a statement that a test has a given size is a statement about the specified triple $(S,\mathcal{S},\{{\mathbb{P}}_{\theta }\})$, the specified ${\mathrm{\Theta }}_0$ and the specified $R$.