Influence Functions and Efficient Scores — Examples

For $r>1$ and $k\ge 2$, monotonicity gives $k^{-r}\le {\int }_{k-1}^k{x}^{-r}dx$, so ${\sum }_{k\ge 1}{k}^{-r}\le 1+{\int }_1^{\mathrm{\infty }}{x}^{-r}dx<\mathrm{\infty }$. Taking $r=3$ shows $c$ is well-defined and the displayed masses sum to one; taking $r=2$ gives $\mu =c{\sum }_{k\ge 1}{k}^{-2}<\mathrm{\infty }$, so $P\in {\mathcal{M}}_X$. But $E_P{X}^2=c{\sum }_{k\ge 1}{k}^{-1}=\mathrm{\infty }$, because each dyadic block $2^{j-1}<k\le 2^j$ contributes at least $1\mathrm{/}2$.

For bounded centered $s$, choose $t\ge 0$ small enough that $1+ts\ge 0$. By [F7], $E_P(1+ts)=1$, so [F9] gives a probability law $P_{t,s}$ with density $1+ts$. The identity $\sqrt{1+u}-1-u\mathrm{/}2=-u^2\mathrm{/}[2(1+\sqrt{1+u}{)}^2]$ bounds the square-root remainder for $u=ts$ in $L^2(P)$ by $t^2\mathrm{\parallel }{s}^2{\mathrm{\parallel }}_2\mathrm{/}2=o(t)$, since $s$ is bounded. Thus these paths are DQM with score $s$ under [F1]. To see their scores are dense in $L_0^2(P)$, take $g\in L_0^2(P)$ and truncate to $g_m=\max (-m,\min (g,m))$. Dominated convergence gives $g_m\to g$ in $L^2(P)$ because $\mathrm{\mid }{g}_m-g{\mathrm{\mid }}^2\le g^2$ and the difference tends pointwise to zero. Cauchy--Schwarz and $\mathrm{\parallel }1{\mathrm{\parallel }}_2=1$ give $E_P{g}_m\to 0$, so $g_m-E_P{g}_m$ are bounded centered scores converging to $g$. Hence the tangent space is $L_0^2(P)$.

Step 1.1 gives $P\in {\mathcal{M}}_X$ and $E_{P}X<\mathrm{\infty }$; since $s$ is bounded, [F8] gives the finite expectation $E_{P_{t,s}}X=E_P[X(1+ts)]$, so every designated path lies in ${\mathcal{M}}_X$. Linearity [F7] then gives $\psi (P_{t,s})=E_{P_{t,s}}X=\mu +t{E}_P[Xs]$, so the right derivative on each allowed score is ${\dot{\psi }}_P(s)=E_P[Xs]$.

For $m\ge 2$ let $s_m=(X\wedge m)-E_P[X\wedge m]$. These are bounded centered scores. Their norms are positive because $X\wedge m$ takes the distinct values $1$ and $2$ on the positive-mass points $k=1$ and $k=2$. Direct summation gives $E_P[X{s}_m]=c{H}_m+cm{\sum }_{k>m}{k}^{-2}-\mu E_P[X\wedge m]\ge c{H}_m-{\mu }^2$, since $0\le X\wedge m\le X$. Also $\mathrm{\parallel }{s}_m{\mathrm{\parallel }}_2^2=E_P[(X\wedge m{)}^2]-(E_P[X\wedge m]{)}^2\le c{H}_m+c{m}^2{\sum }_{k>m}{k}^{-3}\le c{H}_m+c\mathrm{/}2,$ where the last bound follows from ${\sum }_{k>m}{k}^{-3}\le {\int }_m^{\mathrm{\infty }}{x}^{-3}dx=1\mathrm{/}(2m^2)$. The dyadic-block argument in step 1.1 shows $H_m\to \mathrm{\infty }$, so $\mathrm{\mid }{E}_P[X{s}_m]\mathrm{\mid }\mathrm{/}\mathrm{\parallel }{s}_m{\mathrm{\parallel }}_2\to \mathrm{\infty }$. The derivative on the dense score set is therefore unbounded in the tangent norm and cannot extend continuously, contrary to the requirement in [F1].

If a finite-variance influence curve $\varphi \in L_0^2(P)$ represented this derivative, then for every $m\ge 2$ Cauchy--Schwarz would give $\mathrm{\mid }{E}_P[X{s}_m]\mathrm{\mid }=\mathrm{\mid }{E}_P[\varphi s_m]\mathrm{\mid }\le \mathrm{\parallel }\varphi {\mathrm{\parallel }}_2\mathrm{\parallel }{s}_m{\mathrm{\parallel }}_2$, contradicting step 3.1. Thus no such influence curve exists.

Since $X\in L^2(P)$ and $P$ is a probability, Cauchy--Schwarz [F5] gives $\mu =E_{P}X<\mathrm{\infty }$. For bounded centered $s$ and sufficiently small $t\ge 0$, $1+ts\ge 0$ and $E_P(1+ts)=1+t{E}_{P}s=1$, so $P_{t,s}$ is a probability law with density $1+ts$ by [F9]. Direct rationalization gives $\sqrt{1+u}-1-u\mathrm{/}2=-u^2\mathrm{/}[2(1+\sqrt{1+u}{)}^2]$; for $u=ts$ the $L^2(P)$ norm of the DQM remainder in [F10] is at most $t^2\mathrm{\parallel }{s}^2{\mathrm{\parallel }}_2\mathrm{/}2=o(t)$ because $s$ is bounded. Thus the score is $s$.

Every bounded mean-zero $s$ is an allowed score. If $g\in L_0^2(P)$, truncate it to $g_m=\max (-m,\min (g,m))$. Then $g_m\to g$ in $L^2(P)$ by [F6], since $\mathrm{\mid }{g}_m-g{\mathrm{\mid }}^2\le g^2$ and the difference tends pointwise to zero. Also $E_P{g}_m\to E_{P}g=0$ by Cauchy–Schwarz and [F3], [F5]. Hence $s_m=g_m-E_P{g}_m$ is bounded and mean-zero, and $s_m\to g$ in $L^2(P)$. Thus the score closure is exactly $L_0^2(P)$ by the tangent-space convention [F10].

Since $P\in {\mathcal{M}}_X$, $E_{P}X<\mathrm{\infty }$; with bounded $s$, [F8] gives the finite expectation $E_{P_{t,s}}X=E_P[X(1+ts)]$, so every designated path lies in ${\mathcal{M}}_X$. Linearity [F7] then yields $\psi (P_{t,s})=E_{P_{t,s}}X=\mu +t{E}_P[Xs]$. Since $E_{P}s=0$, the right derivative is $E_P[(X-\mu )s]$.

Cauchy–Schwarz gives $\mathrm{\mid }{E}_P[(X-\mu )s]\mathrm{\mid }\le \mathrm{\parallel }X-\mu {\mathrm{\parallel }}_2\mathrm{\parallel }s{\mathrm{\parallel }}_2$, so the derivative extends continuously to $\mathcal{T}$. The vector $X-\mu$ is mean-zero and square-integrable, hence belongs to $\mathcal{T}$. If another $\tilde{\phi }\in \mathcal{T}$ represents the same derivative, then $h=\tilde{\phi }-(X-\mu )\in \mathcal{T}$ is orthogonal to every bounded mean-zero score; by density and continuity it is orthogonal to itself, so $E_P{h}^2=0$ and $h=0$. By [F10], $X-\mu$ is the unique representer in $\mathcal{T}$ and therefore the canonical gradient.

By the empirical-measure convention in [F2] and $E_P(X-\mu )=0$, $(P_n-P)(X-\mu )=n^{-1}{\sum }_{i=1}^n(X(X_i)-\mu )={\bar{X}}_n-\mu$ exactly. Step 2.1 gives $X-\mu \in L_0^2(P)$. The remainder is zero, so the sample mean is asymptotically linear with estimator influence function $X-\mu$ under [F2]. This algebraic identity invokes no central limit theorem.

For each parameter in the stated domain, $\mathrm{\mid }\theta (u+v)+\eta u\mathrm{\mid }\le 2\mathrm{\mid }\theta \mathrm{\mid }+\mathrm{\mid }\eta \mathrm{\mid }<1$, so each mass is positive. Their sum is one because ${\sum }_{u,v}(u+v)={\sum }_{u,v}u=0$ and the four constant terms sum to four. Thus the family consists of probability laws by [F5].

Finite averaging, using [F6], gives $E_{P}U=E_{P}V=0$, $E_P{U}^2=E_P{V}^2=1$, and $E_{P}UV=0$. Thus $w=a(U+V)+bU$ is bounded and centered. Along $({\theta }_t,{\eta }_t)=(at,bt)$ the likelihood ratio is $1+tw$, which is nonnegative and integrates to one for sufficiently small $t\ge 0$. The square-root identity $\sqrt{1+z}-1-z\mathrm{/}2=-z^2\mathrm{/}[2(1+\sqrt{1+z}{)}^2]$ gives an $L^2(P)$ remainder bounded by $t^2\mathrm{\parallel }{w}^2{\mathrm{\parallel }}_2\mathrm{/}2=o(t)$, so the path is DQM with score $w$ by [F2]. Direct differentiation of the log density gives interest score $U+V$ and nuisance score $U$ by [F3]. Thus the designated nuisance scores span $\{U\}$, which is closed since $\mathrm{\parallel }cU{\mathrm{\parallel }}_2=\mathrm{\mid }c\mathrm{\mid }$, and all designated scores $a(U+V)+bU$ span $\{U,V\}$, which is closed because $\mathrm{\parallel }aU+bV{\mathrm{\parallel }}_2^2=a^2+b^2$.

Since $U\in {\mathcal{T}}_{\eta }$ and $E_P[(U+V-U)U]=E_P[VU]=0$, $U$ is the orthogonal projection of $S_{\theta }=U+V$ onto ${\mathcal{T}}_{\eta }$. Hence $S_{\mathrm{e}\mathrm{f}\mathrm{f}}=V$ and $I_{\mathrm{e}\mathrm{f}\mathrm{f}}=E_P{V}^2=1$ by [F1].

Along each direction $(at,bt)$, the target parameter is ${\theta }_t=at$, so its right derivative is $a$. Step 1.2 gives the full score-space decomposition $\mathcal{T}=\overline{\{a{S}_{\theta }+bU:a,b\in \mathbb{R}\}}$; step 2.1 gives nonsingular $I_{\mathrm{e}\mathrm{f}\mathrm{f}}=1$. The finite densities are explicit, and the theorem's stated axiom inputs are Axiom of Choice [A1] and Countable Choice [A2]. Therefore the hypotheses of [F4] apply, and the canonical gradient is $I_{\mathrm{e}\mathrm{f}\mathrm{f}}^{-1}{S}_{\mathrm{e}\mathrm{f}\mathrm{f}}=V$.