Wald, score, and likelihood-ratio statistics for regular nulls

Assume AC for the supplied density representatives in Score and Fisher information and The Axiom of Choice. Let $X_1,\dots ,X_n$ be iid from a dominated parametric model on an open $\mathrm{\Theta }\subset {\mathbb{R}}^d$, and let ${\mathrm{\ell }}_n(\theta )={\sum }_{i=1}^n\log p_{\theta }(X_i)$. For any nested null ${\mathrm{\Theta }}_0\subseteq \mathrm{\Theta }$ for which the maxima exist, the likelihood-ratio statistic is $L_n=2\{{\sup }_{\theta \in \mathrm{\Theta }}{\mathrm{\ell }}_n(\theta )-{\sup }_{\theta \in {\mathrm{\Theta }}_0}{\mathrm{\ell }}_n(\theta )\}\mathrm{.}$

For the Wald and score statistics, specialize to a regular equality null of codimension $q$ at an interior point ${\theta }_0$: this is a null set which, in a $C^1$ local parameter chart on an open neighborhood $V$ of ${\theta }_0$, $\theta =(\psi ,\eta )\in {\mathbb{R}}^q\times {\mathbb{R}}^{d-q}$, is $\psi =0$ near ${\theta }_0=(0,{\eta }_0)$. Its local tangent is the $\eta$-coordinate subspace. Assume the QMD information $I({\theta }_0)$ is positive definite. Let ${\widehat{\theta }}_n$ maximize ${\mathrm{\ell }}_n$ over the full parameter space $\mathrm{\Theta }$ and ${\tilde{\theta }}_n$ maximize it over the null ${\mathrm{\Theta }}_0$, so the general likelihood-ratio definition is $L_n=2\{{\mathrm{\ell }}_n({\widehat{\theta }}_n)-{\mathrm{\ell }}_n({\tilde{\theta }}_n)\}$.

For the score statistic, assume $\log p_{\theta }(x)$ is differentiable in the local chart at the parameter values where it is evaluated, and write $s_{\theta }(x)={\mathrm{\nabla }}_{\theta }\log p_{\theta }(x)$. Require explicitly that this pointwise derivative at ${\theta }_0$ agrees $P_{{\theta }_0}$-almost surely with the QMD score; this compatibility is an assumption, not a consequence of QMD alone. The pointwise score and its information-matrix convention are those in Score and Fisher information.

Partition the score and information in $(\psi ,\eta )$ coordinates: $s=(s_{\psi },s_{\eta })$ and $I=\left(\begin{array}{cc}{I}_{\psi \psi }& I_{\psi \eta }\\ I_{\eta \psi }& I_{\eta \eta }\end{array}\right),I_{\mathrm{e}\mathrm{f}\mathrm{f}}=I_{\psi \psi }-I_{\psi \eta }{I}_{\eta \eta }^{-1}{I}_{\eta \psi }\mathrm{.}$ The matrix $I_{\mathrm{e}\mathrm{f}\mathrm{f}}$ is the information for the tested coordinates after accounting for the nuisance coordinates. If $d=q$, interpret empty-block products as zero, so $I_{\mathrm{e}\mathrm{f}\mathrm{f}}=I_{\psi \psi }=I$ and $C$ below is the empty $q\times 0$ matrix. If $d>q$, then $I_{\eta \eta }$ is positive definite; for $z\mathrm{\ne }0$, the quadratic form of $I$ at $(z,-I_{\eta \eta }^{-1}{I}_{\eta \psi }z)$ equals $z^T{I}_{\mathrm{e}\mathrm{f}\mathrm{f}}z>0$. Thus $I_{\mathrm{e}\mathrm{f}\mathrm{f}}$ is positive definite in either case. Put

$$U_n(\theta )=n^{-1\mathrm{/}2}\sum _{i=1}^n{s}_{\theta }(X_i)\mathrm{.}$$

For a consistent estimator ${\widehat{I}}_{\mathrm{e}\mathrm{f}\mathrm{f},n}$ of $I_{\mathrm{e}\mathrm{f}\mathrm{f}}$ and a consistent estimator ${\widehat{C}}_n$ of $C=I_{\psi \eta }{I}_{\eta \eta }^{-1}$, when ${\tilde{\theta }}_n\in V$ set $U_{\mathrm{e}\mathrm{f}\mathrm{f},n}=U_{\psi ,n}({\tilde{\theta }}_n)-{\widehat{C}}_n{U}_{\eta ,n}({\tilde{\theta }}_n)$. The score statistic is $S_n=U_{\mathrm{e}\mathrm{f}\mathrm{f},n}^T{\widehat{I}}_{\mathrm{e}\mathrm{f}\mathrm{f},n}^{-1}{U}_{\mathrm{e}\mathrm{f}\mathrm{f},n}$. For a consistent covariance estimator ${\widehat{V}}_{\psi ,n}$ of the $\psi$-block of $I^{-1}$, when ${\widehat{\theta }}_n\in V$ the Wald statistic is $W_n=n{\widehat{\psi }}_n^T{\widehat{V}}_{\psi ,n}^{-1}{\widehat{\psi }}_n$, where ${\widehat{\psi }}_n$ is the tested coordinate of ${\widehat{\theta }}_n$. Set $W_n=0$ when ${\widehat{\theta }}_n\mathrm{\notin }V$ and $S_n=0$ when ${\tilde{\theta }}_n\mathrm{\notin }V$, without evaluating coordinates or scores outside the chart. When a displayed inverse does not exist, set that statistic to zero. Under the regularity conditions used in the later calibration theorem, consistency places both maximizers in $V$ and the estimated inverses exist with probability tending to one. This definition gives the statistics and their normalizations; it asserts no finite-sample or asymptotic distribution.