Null-valid p-values and familywise and false-discovery error rates

Let $\mathsf{E}=((\mathcal{X},\mathcal{A}),\mathrm{\Theta },(P_{\theta }{)}_{\theta \in \mathrm{\Theta }})$ be a statistical experiment (Statistical experiments), and fix an integer $m\ge 1$ before observing the data. Write $[m]=\{1,\dots ,m\}$. A finite family of null hypotheses is a fixed list $H_{01},\dots ,H_{0m}$ with each $H_{0i}\subseteq \mathrm{\Theta }$. For each $i\in [m]$, let $p_i:\mathcal{X}\to [0,1]$ be a p-value statistic (Valid p-values). It is valid for $H_{0i}$ when

$P_{\theta }\{x:p_i(x)\le u\}\le u(\theta \in H_{0i},0\le u\le 1)\mathrm{.}$

If $H_{0i}=\mathrm{\varnothing }$, this universal condition is vacuous and $i$ never belongs to a true-null index set. For a nonempty null it agrees with the published valid-p-value definition. The family may have any joint dependence among its p-values. Validity is required only under the corresponding null laws; a p-value statistic alone does not imply validity.

For $\theta \in \mathrm{\Theta }$, the true-null index set is $I_0(\theta ):=\{i\in [m]:\theta \in H_{0i}\}\mathrm{.}$ A fixed deterministic multiple-testing rule is a family of measurable rejection events $D_i\in \mathcal{A}$, one for each $i\in [m]$. Its total number of rejections and number of false rejections at parameter $\theta$ are $R:={\sum }_{i=1}^m{\mathbf{1}}_{D_i},V_{\theta }:={\sum }_{i\in I_0(\theta )}{\mathbf{1}}_{D_i}\mathrm{.}$ Its familywise error rate (FWER) at $\theta$ is $P_{\theta }(V_{\theta }\ge 1)$, and its false discovery rate (FDR) at $\theta$ is $E_{\theta }\text{\hspace{0.17em}⁣}\left[\frac{V_{\theta }}{R\vee 1}\right],$ where $R\vee 1=\max \{R,1\}$. The random variable $V_{\theta }\mathrm{/}(R\vee 1)$ is the false discovery proportion (FDP). Strong FWER control at level $\alpha$ means that this FWER is at most $\alpha$ for every $\theta \in \mathrm{\Theta }$, including parameters for which only some of the null hypotheses are true. The denominator convention makes the FDP zero when no hypothesis is rejected.

If the testing rule uses auxiliary randomization, include that randomizer in the observation and its sigma-algebra before defining the rejection events $D_i$.

These quantities are well-defined: the finite sums of indicators are measurable, $0\le V_{\theta }\le R$, and $0\le V_{\theta }\mathrm{/}(R\vee 1)\le 1$, so the ratio is integrable under each probability law (Expectation of a nonnegative or integrable random variable).