Influence Functions and Efficient Scores

Let $H$ be a real Hilbert space, let $\mathcal{T}\subseteq H$ be a closed linear subspace, and let $\dot{\psi }:\mathcal{T}\to {\mathbb{R}}^k$ be continuous linear with an ambient representer ${\varphi }_0\in H^k$, meaning $\dot{\psi }(s)=⟨{\varphi }_0,s{⟩}_H$ coordinatewise for all $s\in \mathcal{T}$. Assume Countable Choice. Then all ambient representers are exactly

${\varphi }_0+({\mathcal{T}}^{\perp }{)}^k\mathrm{.}$

The unique representer in ${\mathcal{T}}^k$ is ${\varphi }_{\mathrm{c}\mathrm{a}\mathrm{n}}={\mathrm{\Pi }}_{\mathcal{T}}{\varphi }_0$; when $\dot{\psi }$ is a statistical path derivative, this is its canonical gradient. Projection is applied coordinatewise.

The theorem is a Hilbert-space statement; its only choice use is Countable Choice for the projection decomposition.

Fix a law $P$ and an iid sample $X_1,\dots ,X_n\sim P$. For a measurable vector function $\phi :\mathcal{X}\to {\mathbb{R}}^k$, write $P\phi =E_P[\phi (X)]$ and $P_n\phi =n^{-1}{\sum }_{i=1}^n\phi (X_i)$, where $P_n=n^{-1}{\sum }_{i=1}^n{\delta }_{X_i}$ is the empirical measure. For a vector remainder $R_n$, $R_n=o_P(n^{-1\mathrm{/}2})$ means $\sqrt{n}{R}_n\to 0$ in $P$-probability.

A measurable estimator sequence ${\widehat{\psi }}_n$ of $\psi (P)\in {\mathbb{R}}^k$ is asymptotically linear at $P$ with estimator influence function $\phi$ if

${\widehat{\psi }}_n-\psi (P)=(P_n-P)\phi +o_P(n^{-1\mathrm{/}2}),$

where $\phi \in L_0^2(P{)}^k$: $E_P\phi =0$ and $E_P\mathrm{\parallel }\phi {\mathrm{\parallel }}^2<\mathrm{\infty }$. The space $L_0^2(P)$ and its almost-sure equivalence convention are as in Mean-zero square-integrable functions. This definition alone does not assert regularity under local alternatives, pathwise differentiability of $\psi$, or asymptotic normality.

The term “influence function” here describes an estimator expansion. The neighboring pathwise-differentiability definition describes a functional's derivative along model paths. The two notions are distinct, though they can coincide for a particular estimator and target.

Let $P$ be a probability law for one observation, let $\mathcal{P}$ be a model of laws containing $P$, and let $\psi :\mathcal{P}\to {\mathbb{R}}^k$ be a target functional, where $k\ge 1$ is finite. A one-sided path $t\mapsto P_t$, $0\le t<ϵ$, through $P=P_0$ is differentiable in quadratic mean (DQM) with score $s\in L^2(P)$ when its densities $p_t,p$ relative to one common sigma-finite measure $\mu$ satisfy

${\parallel \sqrt{p_t}-\sqrt{p}-\frac{t}{2}s\sqrt{p}\parallel }_{L^2(\mu )}=o(t)(t\downarrow 0)\mathrm{.}$

The score initially belongs to $L^2(P)$; A DQM score has mean zero and is unique proves it is unique modulo $P$-almost-sure equality and has mean zero, so it lies in $L_0^2(P)$ as defined by Mean-zero square-integrable functions. The allowed scores form the tangent set of the designated paths (Tangent set of a statistical model at a law); their tangent space $\mathcal{T}$ is the closure of their real linear span in $L_0^2(P)$ (Closed tangent space). It is a closed linear subspace by A tangent space is a closed linear subspace, and hence a Hilbert space under Countable Choice (The mean-zero L² space is a Hilbert space).

The functional $\psi$ is pathwise differentiable at $P$ relative to $\mathcal{T}$ if the right derivative ${\dot{\psi }}_P(s):={\lim }_{t\downarrow 0}(\psi (P_t)-\psi (P))\mathrm{/}t$ exists for every allowed path, depends only on its score, and extends to a continuous linear map ${\dot{\psi }}_P:\mathcal{T}\to {\mathbb{R}}^k$. A vector $\varphi \in L_0^2(P{)}^k$ is a parameter influence curve (or representer) when

${\dot{\psi }}_P(s)=E_P[\varphi s](s\in \mathcal{T}),$

with the expectation interpreted coordinatewise. The Hilbert-space Riesz theorem gives a unique representer in ${\mathcal{T}}^k$; it is the canonical gradient. This representation requires Countable Choice, as in Riesz representation for Hilbert spaces and The Axiom of Countable Choice (${\mathrm{A}\mathrm{C}}_{\omega }$). The supplied density versions along a path use the separate Axiom of Choice in A dominated differentiable-in-quadratic-mean submodel and The Axiom of Choice. The full set of ambient representers and its projection characterization are proved in Influence-curve representers form an affine orthogonal-complement class.

The tangent space is the closed linear span of allowed scores, not necessarily the full $L_0^2(P)$; restricting or enlarging the allowed paths can therefore change pathwise differentiability and the canonical gradient. A parameter influence curve is a derivative representer. This definition does not assert that an estimator attaining a bound exists.

Let $\theta \in {\mathbb{R}}^p$, $p\ge 1$ finite, be a parameter of interest at a law $P$. Assume its score vector $S_{\theta }=(S_{\theta ,1},\dots ,S_{\theta ,p}{)}^T$ has components in $L_0^2(P)$; under a dominated differentiable parametric submodel these are the coordinate log-density scores of Score and Fisher information. Let ${\mathcal{T}}_{\eta }\subseteq L_0^2(P)$ be the closed nuisance tangent space of Nuisance tangent space. The ambient $L_0^2(P)$ is as in Mean-zero square-integrable functions. The score-density convention uses the Axiom of Choice, while Hilbert projection uses Countable Choice; both dependencies are retained separately.

The Hilbert decomposition Orthogonal decomposition by a closed subspace defines ${\mathrm{\Pi }}_{{\mathcal{T}}_{\eta }}f$ as the unique $g\in {\mathcal{T}}_{\eta }$ with $f-g\in {\mathcal{T}}_{\eta }^{\perp }$, where orthogonality and ${\mathcal{T}}_{\eta }^{\perp }$ have the meanings in Orthogonality and the orthogonal complement. The efficient score vector and efficient information matrix are

$S_{\mathrm{e}\mathrm{f}\mathrm{f}}=S_{\theta }-{\mathrm{\Pi }}_{{\mathcal{T}}_{\eta }}{S}_{\theta },I_{\mathrm{e}\mathrm{f}\mathrm{f}}=E_P[S_{\mathrm{e}\mathrm{f}\mathrm{f}}{S}_{\mathrm{e}\mathrm{f}\mathrm{f}}^T],$

where ${\mathrm{\Pi }}_{{\mathcal{T}}_{\eta }}$ is the Hilbert-space orthogonal projection applied coordinatewise. This projection exists and is unique because ${\mathcal{T}}_{\eta }$ is a closed linear subspace; the projection theorem used by Orthogonal decomposition by a closed subspace assumes Countable Choice (The Axiom of Countable Choice (${\mathrm{A}\mathrm{C}}_{\omega }$)).

The formula does not assert nonsingularity. If $I_{\mathrm{e}\mathrm{f}\mathrm{f}}$ is singular, the inverse-information expression is not defined. The finite-dimensional canonical-gradient formula requires the full score-space decomposition and nonsingularity as additional hypotheses, established in the accompanying theorem on this page.

Assume the Axiom of Choice and Countable Choice.

Let $H=L_0^2(P)$, let $S_{\theta }\in H^p$ be a finite-dimensional interest-score vector, and let ${\mathcal{T}}_{\eta }\subseteq \mathcal{T}\subseteq H$ be closed linear nuisance and full tangent spaces, with ${\mathcal{T}}_{\eta }$ a subspace of $\mathcal{T}$. Assume the designated allowed score set is exactly

$\mathcal{S}=\{a^T{S}_{\theta }+t:a\in {\mathbb{R}}^p,t\in {\mathcal{T}}_{\eta }\},\mathcal{T}={\bar{\mathcal{S}}}^{L^2(P)},$

and every allowed one-sided path with score $a^T{S}_{\theta }+t\in \mathcal{S}$ has right target derivative $a$ for the parameter $\theta$. If $I_{\mathrm{e}\mathrm{f}\mathrm{f}}=E_P[S_{\mathrm{e}\mathrm{f}\mathrm{f}}{S}_{\mathrm{e}\mathrm{f}\mathrm{f}}^T]$ is nonsingular, then $\theta$ is pathwise differentiable and its canonical gradient is

${\varphi }_{\mathrm{c}\mathrm{a}\mathrm{n}}=I_{\mathrm{e}\mathrm{f}\mathrm{f}}^{-1}{S}_{\mathrm{e}\mathrm{f}\mathrm{f}}\mathrm{.}$

Its covariance matrix is $E_P[{\varphi }_{\mathrm{c}\mathrm{a}\mathrm{n}}{\varphi }_{\mathrm{c}\mathrm{a}\mathrm{n}}^T]=I_{\mathrm{e}\mathrm{f}\mathrm{f}}^{-1}$. This theorem does not assert that an estimator attaining the covariance bound exists.