Scores and Information Bounds — Examples

Put $\epsilon :=\frac{1}{2}\min \{p,1-p\}>0$, the open neighborhood $U:=(p-\epsilon ,p+\epsilon )$, and its compact closure $K:=[p-\epsilon ,p+\epsilon ]\subset (0,1)$. For $q\in K$, the masses $p_q(1)=q$ and $p_q(0)=1-q$ are positive and twice continuously differentiable on the fixed support $S=\{0,1\}$, with common exceptional set $N=\mathrm{\varnothing }$. With counting measure $\mathrm{\#}$, their first derivatives are $1$ and $-1$, and their second derivatives are $0$. Thus the common envelopes $g_1(0)=g_1(1)=1$ and $g_2\equiv 0$ dominate all first and second derivatives on $U$; they are integrable because $\int g_{1}d\mathrm{\#}=2$ and $\int g_{2}d\mathrm{\#}=0$.

The map $T(x)=x$ is real-valued and measurable on the full power set of $\{0,1\}$. For every $q\in (0,1)$, the masses sum to $1$, and [F2] gives ${\mathbb{E}}_{q}T=q$ and ${\mathbb{E}}_q{T}^2=q<\mathrm{\infty }$; hence $T=X$ is unbiased for $\psi (q)=q$ throughout the parameter interval, including a neighborhood of $p$. At $p$, finite summation gives ${\mathrm{Var}}_p(T)=p(1-p)$. The log density is ${\mathrm{\ell }}_p(x)=x\log p+(1-x)\log (1-p)$. Differentiating gives $S_p(x)=x\mathrm{/}p-(1-x)\mathrm{/}(1-p)=(x-p)\mathrm{/}(p(1-p))$ and ${\mathrm{\ell }}_p^{\mathrm{\prime }\mathrm{\prime }}(x)=-x\mathrm{/}{p}^2-(1-x)\mathrm{/}(1-p{)}^2$. Since the two score values are finite, its second moment is finite. Finite summation yields ${\mathbb{E}}_p{S}_p=0$ and $I(p)={\mathbb{E}}_p{S}_p^2=1\mathrm{/}[p(1-p)]=-{\mathbb{E}}_p{\mathrm{\ell }}_p^{\mathrm{\prime }\mathrm{\prime }}$, agreeing with [F4].

Since $S_p=(X-p)\mathrm{/}(p(1-p))$, [F5] gives ${\mathrm{Cov}}_p(X,S_p)={\mathrm{Var}}_p(X)\mathrm{/}[p(1-p)]=1$. On the open parameter interval $(0,1)$, $\psi (p)=p$ has difference quotient $1$ at every $x\mathrm{\ne }p$, so [F8] gives ${\psi }^{\mathrm{\prime }}(p)=1$; the score-covariance identity required by [F6] holds.

The bound from [F6] is ${\mathrm{Var}}_p(T)\ge 1\mathrm{/}I(p)=p(1-p)$, and step 1.2 gives equality. Moreover $T-\psi (p)=X-p=p(1-p)S_p=({\psi }^{\mathrm{\prime }}(p)\mathrm{/}I(p))S_p$, exactly its equality condition.

Let $Z\sim N(0,1)$ have density $\varphi$, and put $G(z):=z\varphi (z)$. By [F9], ${\varphi }^{\mathrm{\prime }}(z)=-z\varphi (z)$ and $G^{\mathrm{\prime }}(z)=\varphi (z)-z^2\varphi (z)$. These functions are smooth.

For each integer $n\ge 1$, $G^{\mathrm{\prime }}$ is continuous on $[-n,n]$, hence integrable there by [F10]. Applying the second FTC and then the Riemann–Lebesgue integral identity in [F10] gives ${\int }_{[-n,n]}(\varphi -z^2\varphi )d\lambda =G(n)-G(-n)=2n\varphi (n)$. Linearity [F3] therefore yields ${\int }_{[-n,n]}{z}^2\varphi d\lambda ={\int }_{[-n,n]}\varphi d\lambda -2n\varphi (n)$.

By the power-series representation in [F9], $e^{n^2\mathrm{/}2}\ge (n^2\mathrm{/}2{)}^2\mathrm{/}2!=n^4\mathrm{/}8$, so $0\le n\varphi (n)\le 8\mathrm{/}(\sqrt{2\pi }{n}^3)\to 0$. Also $\varphi {\mathbf{1}}_{[-n,n]}\uparrow \varphi$ and $z^2\varphi {\mathbf{1}}_{[-n,n]}\uparrow z^2\varphi$. Monotone convergence [F11], normalization [F2], and the compact calculation in step 2.1 give $\mathbb{E}{Z}^2=\int z^2\varphi d\lambda ={\lim }_n{\int }_{[-n,n]}{z}^2\varphi d\lambda =1<\mathrm{\infty }$, also the second moment supplied in [F2].

Differentiating the displayed normal density gives ${\mathrm{\partial }}_{\eta }{p}_{\eta }=(x-\eta )p_{\eta }\mathrm{/}{\sigma }^2$, ${\mathrm{\partial }}_{\eta \eta }{p}_{\eta }=((x-\eta {)}^2\mathrm{/}{\sigma }^4-1\mathrm{/}{\sigma }^2)p_{\eta }$, $S_{\mu }=(X-\mu )\mathrm{/}{\sigma }^2$, and ${\mathrm{\ell }}_{\mu }^{\mathrm{\prime }\mathrm{\prime }}=-1\mathrm{/}{\sigma }^2$. The support is all of $\mathbb{R}$, and step 3.1 with $X=\mu +\sigma Z$ gives ${\mathbb{E}}_{\mu }{S}_{\mu }^2=1\mathrm{/}{\sigma }^2<\mathrm{\infty }$.

Fix ${\mu }_0\in \mathbb{R}$ and $\delta >0$, put $y=x-{\mu }_0$, and take $\mathrm{\mid }\eta -{\mu }_0\mathrm{\mid }<\delta$. Then $\mathrm{\mid }x-\eta \mathrm{\mid }\le \mathrm{\mid }y\mathrm{\mid }+\delta$ and $(x-\eta {)}^2\ge y^2\mathrm{/}2-{\delta }^2$, so both first and second density derivatives are bounded by $C(1+y^2)e^{-y^2\mathrm{/}(4{\sigma }^2)}$ for a finite $C$ independent of $\eta ,x$. First, write the displayed density as $p_{\mu }(x)={\sigma }^{-1}\varphi ((x-\mu )\mathrm{/}\sigma )$. For every Borel set $A$, the scaling and translation identities [F12] with $a=\sigma$ and $b=\mu$ give ${\int }_A{p}_{\mu }(x)dx=\int {\mathbf{1}}_A(\mu +\sigma z)\varphi (z)dz=P(\mu +\sigma Z\in A)$, so $p_{\mu }$ is indeed a density of the specified law $N(\mu ,{\sigma }^2)$. To justify the envelope transformation, put $a=\sqrt{2}\sigma >0$ and $q(y)=(4\pi {\sigma }^2{)}^{-1\mathrm{/}2}{e}^{-y^2\mathrm{/}(4{\sigma }^2)}$. For every Borel set $A$, [F12] applied to $f={\mathbf{1}}_{A}q$ gives ${\int }_{A}q(y)dy=a\int {\mathbf{1}}_A(az)q(az)dz=\int {\mathbf{1}}_A(az)\varphi (z)dz$, since $aq(az)=\varphi (z)$. By [F1] and [F3], the last integral equals $P(aZ\in A)$; thus $q$ is indeed a density of the affine law $Y=aZ\sim N(0,2{\sigma }^2)$. The envelope is a constant multiple of $(1+y^2)q(y)$. By [F3] and step 3.1, its integral in $y$ is finite, since $E(1+Y^2)=1+a^{2}E{Z}^2=1+2{\sigma }^2$. Translation invariance in [F12] transfers this finite integral from $y$ to $x=y+{\mu }_0$, proving integrability of the displayed envelope on the original data space. Thus the fixed-support, continuous-derivative, envelope, and score-moment hypotheses of [F5] hold, so ${\mathbb{E}}_{\mu }{S}_{\mu }=0$ and $I(\mu )=-{\mathbb{E}}_{\mu }{\mathrm{\ell }}_{\mu }^{\mathrm{\prime }\mathrm{\prime }}=1\mathrm{/}{\sigma }^2$.

From $X-\mu ={\sigma }^2{S}_{\mu }$ and step 5.1, ${\mathbb{E}}_{\mu }X=\mu$. Also ${\mathbb{E}}_{\mu }[(X-\mu {)}^2]={\sigma }^2\mathbb{E}{Z}^2={\sigma }^2$, so $T=X$ has variance ${\sigma }^2$.

The pointwise identity $X-\mu ={\sigma }^2{S}_{\mu }$ and [F6] give ${\mathrm{Cov}}_{\mu }(X,S_{\mu })={\sigma }^{2}I(\mu )=1$. On $\mathbb{R}$, $\psi (\mu )=\mu$ has difference quotient $1$ at every $x\mathrm{\ne }\mu$, so [F8] gives ${\psi }^{\mathrm{\prime }}(\mu )=1$; the score-covariance identity required by [F7] holds.

The bound [F7] is ${\mathrm{Var}}_{\mu }(X)\ge 1\mathrm{/}I(\mu )={\sigma }^2$, and step 6.1 gives equality. Also $X-\mu ={\sigma }^2{S}_{\mu }=({\psi }^{\mathrm{\prime }}(\mu )\mathrm{/}I(\mu ))S_{\mu }$, so the stated equality condition holds.

By [F3], the endpoint singletons have measure zero and $\lambda ((0,\theta ))=\theta$, so this density version gives the law $\mathrm{U}\mathrm{n}\mathrm{i}\mathrm{f}[0,\theta ]$ and has total mass one. Using [F2] for the nonnegative functions $2x$ and $(2x-\theta {)}^2$, then [F4] to evaluate their integrals, gives ${\mathbb{E}}_{\theta }T={\theta }^{-1}{\int }_0^{\theta }2xdx=\theta$ and ${\mathrm{Var}}_{\theta }(T)={\mathbb{E}}_{\theta }[(2X-\theta {)}^2]={\theta }^{-1}{\int }_0^{\theta }(2x-\theta {)}^{2}dx={\theta }^{-1}{[\frac{4}{3}{x}^3-2\theta x^2+{\theta }^{2}x]}_0^{\theta }=\frac{{\theta }^2}{3}$. Thus $T$ is unbiased for $\psi (\theta )=\theta$ and has variance ${\theta }^2\mathrm{/}3$.

For $0<x<\theta$, ${\mathrm{\ell }}_{\theta }(x)=-\log \theta$, so $S_{\theta }(x)=-1\mathrm{/}\theta$. Therefore $I(\theta )={\mathbb{E}}_{\theta }{S}_{\theta }^2=1\mathrm{/}{\theta }^2$ but ${\mathbb{E}}_{\theta }{S}_{\theta }=-1\mathrm{/}\theta \mathrm{\ne }0$. Since the score is constant $P_{\theta }$-almost surely, ${\mathrm{Cov}}_{\theta }(T,S_{\theta })=0$. On $\mathrm{\Theta }=(0,\mathrm{\infty })$, $\psi (\theta )=\theta$ has difference quotient $1$ at every $x\mathrm{\ne }\theta$, so [F8] gives ${\psi }^{\mathrm{\prime }}(\theta )=1$. Both score identities needed for [F7] fail here.

The formula obtained by ignoring those failures would give ${\psi }^{\mathrm{\prime }}(\theta {)}^2\mathrm{/}I(\theta )={\theta }^2$, contradicting the actual ${\mathrm{Var}}_{\theta }(T)={\theta }^2\mathrm{/}3$ from step 1.1. This does not contradict [F7]: the moving support violates its regularity route, and the score-mean and covariance-derivative hypotheses fail explicitly in step 2.1.