Test size and power

Let $\mathsf{E}=((\mathcal{X},\mathcal{A}),\mathrm{\Theta },(P_{\theta }{)}_{\theta \in \mathrm{\Theta }})$ be a statistical experiment (Statistical experiments). Let ${\mathrm{\Theta }}_0,{\mathrm{\Theta }}_1$ be nonempty disjoint subsets of $\mathrm{\Theta }$, called the null and alternative hypotheses. A hypothesis is simple when it is a singleton and composite when it has at least two elements.

For a randomized test with test function $\phi :\mathcal{X}\to [0,1]$ (Randomized tests), its power function is $\beta (\theta ):=E_{\theta }[\phi (X)](\theta \in \mathrm{\Theta })\mathrm{.}$ Its size is ${\sup }_{\theta \in {\mathrm{\Theta }}_0}\beta (\theta ),$ and it has level $\alpha$ when this supremum is at most $\alpha \in [0,1]$. The expectations exist and lie in $[0,1]$ because $0\le \phi \le 1$ and each $P_{\theta }$ is a probability measure (Probability measures and probability spaces, Expectation of a nonnegative or integrable random variable). The supremum exists because ${\mathrm{\Theta }}_0$ is nonempty, the values lie in $[0,1]$. Here The real numbers identifies $\mathbb{R}$ with the Cauchy-sequence reals, The reals form a totally ordered field gives their ordered-field structure, and The Cauchy-sequence reals have the least-upper-bound property establishes their least-upper-bound property (Complete ordered field (least-upper-bound property)).

For the binary action $a\in \{0,1\}$, where $1$ means reject, the zero-one error loss is $1$ when $a=1$ and $\theta \in {\mathrm{\Theta }}_0$, or when $a=0$ and $\theta \in {\mathrm{\Theta }}_1$, and is $0$ otherwise. The decision-rule risk from Loss functions and risk integrates this loss against the action kernel and data law. For each fixed $\theta$, its section on the binary action space is measurable. The risk identities proved in Binary zero-one loss risks are the testing error probabilities show that risk equals $\beta (\theta )$ on ${\mathrm{\Theta }}_0$ and $1-\beta (\theta )$ on ${\mathrm{\Theta }}_1$. Thus size is the worst null error risk, and power is the rejection probability under each alternative parameter.

The supremum need not be attained. No measurability structure on $\mathrm{\Theta }$ or on the subsets ${\mathrm{\Theta }}_0,{\mathrm{\Theta }}_1$ is required; the definition uses only the individual laws $P_{\theta }$ and the pointwise supremum.