Nonregular and Misspecified Limits — Examples

For a sample in the support, independence makes the joint likelihood $L_n(\vartheta ;x)={\prod }_{i=1}^n{p}_{\vartheta }(x_i)={\vartheta }^{-n}\mathbf{1}\{x_{(n)}\le \vartheta \}$ for $\vartheta >0$. If $x_{(n)}>0$, this likelihood is zero below $x_{(n)}$ and strictly decreases as ${\vartheta }^{-n}$ above it; its maximum is at ${\widehat{\theta }}_n=x_{(n)}$.

Since each uniform observation has no atom at zero, $P_{\theta }(X_{(n)}=0)\le P_{\theta }(X_1=0)=0$. Thus the endpoint formula is the MLE almost surely. On the null event $X_{(n)}=0$, the likelihood is ${\vartheta }^{-n}$ and is unbounded as $\vartheta \downarrow 0$, so no maximizer exists there; a fixed positive fallback is only a measurable extension, not an MLE on that event. Applying [F2] identifies the exact rate and limit law.

Put $q_{\mu }(x)=(2\pi {\sigma }^2{)}^{-1\mathrm{/}4}{e}^{-(x-\mu {)}^2\mathrm{/}(4{\sigma }^2)}$. Direct differentiation gives $q_{\mu }^{\mathrm{\prime }}=(x-\mu )q_{\mu }\mathrm{/}(2{\sigma }^2)$ and $q_{\mu }^{\mathrm{\prime }\mathrm{\prime }}=((x-\mu {)}^2\mathrm{/}(4{\sigma }^4)-1\mathrm{/}(2{\sigma }^2))q_{\mu }$. Its square is a polynomial times $p_{\mu }=q_{\mu }^2$. By [F8] and [F9], $\int (x-\mu {)}^2{p}_{\mu }dx={\sigma }^2$ and $\int (x-\mu {)}^4{p}_{\mu }dx=3{\sigma }^4$, so $\mathrm{\parallel }{q}_{\mu }^{\mathrm{\prime }\mathrm{\prime }}{\mathrm{\parallel }}_2^2=3\mathrm{/}(16{\sigma }^4)<\mathrm{\infty }$. Also $s_{\mu }=2q_{\mu }^{\mathrm{\prime }}\mathrm{/}{q}_{\mu }=(x-\mu )\mathrm{/}{\sigma }^2$ has second moment $1\mathrm{/}{\sigma }^2$. The identity $q_a^{\mathrm{\prime }\mathrm{\prime }}(x)=q_0^{\mathrm{\prime }\mathrm{\prime }}(x-a)$ and [F12] give the same norm for every real shift $a$. With $\sigma =1$, Cauchy–Schwarz [F13] gives $E\mathrm{\mid }X-\mu \mathrm{\mid }\le 1$, so the even normal density [F1] gives $E(X-\mu )=0$ and $\mathrm{Var}(X)=1$. The two-variable functions $(x,v)\mapsto q_{\mu +v}^{\mathrm{\prime }\mathrm{\prime }}(x)$ and $(x,u)\mapsto q_{\mu +u}^{\mathrm{\prime }}(x)-q_{\mu }^{\mathrm{\prime }}(x)$ are continuous. To verify the needed product measurability locally, the rectangles $(a,b)\times (c,d)$ with rational endpoints form a countable base of ${\mathbb{R}}^2$: inside any open ball about a point, density of the rationals gives a rational rectangle containing that point and contained in the ball. Every open subset is therefore the union of the subfamily of these rectangles contained in it, hence a countable union of measurable product rectangles. Thus preimages of open subsets under these continuous functions belong to $\mathcal{B}(\mathbb{R})\otimes \mathcal{B}(\mathbb{R})$, and also to the product of the Lebesgue sigma-algebras. Their squares and restrictions to compact parameter intervals are consequently product measurable, as required by Tonelli below.

For fixed $x$, the functions $v\mapsto q_{\mu +v}^{\mathrm{\prime }\mathrm{\prime }}(x)$ and $u\mapsto q_{\mu +u}^{\mathrm{\prime }}(x)-q_{\mu }^{\mathrm{\prime }}(x)$ are continuous on each compact interval. By [F10] they are Riemann integrable, by [F15] their integrals are respectively $q_{\mu +u}^{\mathrm{\prime }}(x)-q_{\mu }^{\mathrm{\prime }}(x)$ and $q_{\mu +t}(x)-q_{\mu }(x)-t{q}_{\mu }^{\mathrm{\prime }}(x)$, and [F11] identifies these with their Lebesgue integrals under the assumed AC. Reversed endpoints give the same identities for negative $u,t$.

Cauchy–Schwarz [F13] on the interval between $0$ and $u$, then Tonelli [F14] and step 1.1, gives $\mathrm{\parallel }{q}_{\mu +u}^{\mathrm{\prime }}-q_{\mu }^{\mathrm{\prime }}{\mathrm{\parallel }}_2^2\le u^2\mathrm{\parallel }{q}_0^{\mathrm{\prime }\mathrm{\prime }}{\mathrm{\parallel }}_2^2$. The second identity of step 2.1 and another Cauchy–Schwarz/Tonelli application give $\mathrm{\parallel }{q}_{\mu +t}-q_{\mu }-t{q}_{\mu }^{\mathrm{\prime }}{\mathrm{\parallel }}_2^2\le \mathrm{\mid }t\mathrm{\mid }{\int }_0^{\mathrm{\mid }t\mathrm{\mid }}{u}^2\mathrm{\parallel }{q}_0^{\mathrm{\prime }\mathrm{\prime }}{\mathrm{\parallel }}_2^{2}du=\mathrm{\mid }t{\mathrm{\mid }}^4\mathrm{\parallel }{q}_0^{\mathrm{\prime }\mathrm{\prime }}{\mathrm{\parallel }}_2^2\mathrm{/}3=o(t^2)$. For negative $t$ use $q_{\mu -u}^{\mathrm{\prime }}$ on the length-$\mathrm{\mid }t\mathrm{\mid }$ interval; the same first bound holds. This proves QMD with the explicitly supplied normal densities, score $s_{\mu }$, and positive information $1\mathrm{/}{\sigma }^2$ by [F7].

Under $\mu =0$, the local sets are $C_n=\mathbb{R}$ and $C_{0n}=(-\mathrm{\infty },0]$ for every $n$, so they are already their limiting closed convex cones. Direct expansion gives ${\mathrm{\ell }}_n(h\mathrm{/}\sqrt{n})-{\mathrm{\ell }}_n(0)=h{Z}_n-h^2\mathrm{/}2$ with $Z_n=\sqrt{n}{\bar{X}}_n$ and zero remainder. The full and null local maximizers are $Z_n$ and $\min (0,Z_n)$. By [F4] and the mean and variance from step 1.1, $Z_n\Rightarrow G\sim N(0,1)$. Choose a large $R$ with $P(\mathrm{\mid }G\mathrm{\mid }\ge R)$ small and apply [F5] to the closed tails to see that both maximizer sequences are tight. Thus all hypotheses of [F3] hold with $I=1$.

Completing the square in ${\mathrm{\ell }}_n(\mu )$ gives the full MLE ${\bar{X}}_n$ and the constrained MLE $\min (0,{\bar{X}}_n)$. Hence the finite-sample likelihood-ratio statistic is $T_n=2[{\mathrm{\ell }}_n({\bar{X}}_n)-{\mathrm{\ell }}_n(\min (0,{\bar{X}}_n))]=n({\bar{X}}_n\vee 0{)}^2=(Z_n\vee 0{)}^2$. By [F5], $T_n\Rightarrow (G\vee 0{)}^2$. Its CDF is $0$ for $t<0$ and $P(G\le \sqrt{t})=\mathrm{\Phi }(\sqrt{t})$ for $t\ge 0$, so it has mass $1\mathrm{/}2$ at zero. By symmetry, $P(G^2\le t)=2\mathrm{\Phi }(\sqrt{t})-1$ for $t\ge 0$, while [F1] gives $P(G=0)=0$; hence $(G\vee 0{)}^2$ has the mixture law $\frac{1}{2}{\delta }_0+\frac{1}{2}\mathcal{L}(G^2)=\frac{1}{2}{\chi }_0^2+\frac{1}{2}{\chi }_1^2$, not ${\chi }_1^2$.

Let $Z\sim N(0,1)$, so . Direct integration gives $E\mathrm{\mid }Z\mathrm{\mid }=2(2\pi {)}^{-1\mathrm{/}2}{\int }_0^{\mathrm{\infty }}z{e}^{-z^2\mathrm{/}2}dz=\sqrt{2\mathrm{/}\pi }<\mathrm{\infty }$, so symmetry gives $EZ=0$. Since ${\varphi }^{\mathrm{\prime }}(z)=-z\varphi (z)$ and $z^k\varphi (z)\to 0$ at both tails, integration by parts gives $E{Z}^2=\int z^2\varphi (z)dz=1$ and $E{Z}^4=\int z^4\varphi (z)dz=3\int z^2\varphi (z)dz=3$. Hence $E_{P}X=0$, $E_P{X}^2={\tau }^2$, and $E_P{X}^4=3{\tau }^4$. The fitted log-density is ${\mathrm{\ell }}_{\theta }(x)=-\frac{1}{2}\log (2\pi )-\frac{1}{2}(x-\theta {)}^2$. The functions $1$, $X$, and $X^2$ are integrable by the finite absolute first and second moments just established. Thus the expanded log-density is a finite linear combination of integrable functions, and [F9] gives $E_P(X-\theta {)}^2=E_P{X}^2-2\theta E_{P}X+{\theta }^2$. Therefore $M(\theta )=E_P{\mathrm{\ell }}_{\theta }(X)=-\frac{1}{2}\log (2\pi )-\frac{1}{2}({\tau }^2+{\theta }^2)$, so ${\theta }^{\ast }=0$ is the unique separated maximizer, and $H=E_P{\mathrm{\partial }}_{\theta }{S}_{{\theta }^{\ast }}(X)=-1$.

The uniform criterion error is bounded by ${\sup }_{\theta \in [-1,1]}\mathrm{\mid }{P}_n{\mathrm{\ell }}_{\theta }-M(\theta )\mathrm{\mid }\le \frac{1}{2}\mathrm{\mid }{P}_n{X}^2-{\tau }^2\mathrm{\mid }+\mathrm{\mid }{P}_{n}X\mathrm{\mid }$. Put $Y_i=X_i$ and $W_i=X_i^2-{\tau }^2$. All terms of the following single-variable expansions are integrable by the finite first, second, and fourth moments in step 1.1. Linearity [F9] therefore gives ${\mathbb{E}}_P{Y}_i={\mathbb{E}}_P{W}_i=0$, ${\mathbb{E}}_P{Y}_i^2={\tau }^2$, and ${\mathbb{E}}_P{W}_i^2=3{\tau }^4-2{\tau }^2{\mathbb{E}}_P{X}_i^2+{\tau }^4=2{\tau }^4$. Moreover ${\mathbb{E}}_P\mathrm{\mid }{Y}_i\mathrm{\mid }=\tau \mathbb{E}\mathrm{\mid }Z\mathrm{\mid }<\mathrm{\infty }$ and ${\mathbb{E}}_P\mathrm{\mid }{W}_i\mathrm{\mid }\le {\mathbb{E}}_P{X}_i^2+{\tau }^2=2{\tau }^2$, so [F8] applies to these variables. By independence and [F8], all cross expectations ${\mathbb{E}}_P[Y_i{Y}_j]$ and ${\mathbb{E}}_P[W_i{W}_j]$ vanish for $i\mathrm{\ne }j$. For either centered family $V_i=Y_i$ or $V_i=W_i$, $\mathrm{\mid }{V}_i{V}_j\mathrm{\mid }\le (V_i^2+V_j^2)\mathrm{/}2$ makes every mixed product integrable. Thus [F9] permits termwise expectation of each finite square expansion and gives ${\mathbb{E}}_P[(P_{n}X{)}^2]={\tau }^2\mathrm{/}n$ and ${\mathbb{E}}_P[(P_n{X}^2-{\tau }^2{)}^2]=2{\tau }^4\mathrm{/}n$. Applying [F5] to these squared deviations gives $P_{n}X{\to }_{P}0$ and $P_n{X}^2{\to }_P{\tau }^2$ by [F7], proving the required uniform law.

The exact constrained MLE is ${\widehat{\theta }}_n={\mathrm{\Pi }}_{[-1,1]}({\bar{X}}_n)$, the projection of the sample mean onto $[-1,1]$. step 2.1 and [F7] give ${\bar{X}}_n{\to }_{P}0$, hence ${\widehat{\theta }}_n{\to }_{P}0$ and the separated-criterion consistency condition holds. On $\{\mathrm{\mid }{\bar{X}}_n\mathrm{\mid }\le 1\}$ the score equation is exact: $P_n{S}_{{\widehat{\theta }}_n}={\bar{X}}_n-{\widehat{\theta }}_n=0$. The complementary event has probability tending to zero, so $\sqrt{n}{P}_n{S}_{{\widehat{\theta }}_n}{\to }_{P}0$. The score is $S_{\theta }(x)=x-\theta$, its second moment at ${\theta }^{\ast }=0$ under $P$ is $J={\tau }^2$, and the sample Hessian is identically $-1$, so the Hessian law holds with $H=-1$.

All hypotheses of [F4] now hold. It gives $\sqrt{n}{\widehat{\theta }}_n\Rightarrow N(0,H^{-1}J{H}^{-T})=N(0,{\tau }^2)$. The fitted model's score second moment at $\theta =0$ is $E_{N(0,1)}[(X-0{)}^2]=1$, so the sandwich variance ${\tau }^2$ differs from the inverse fitted-model information $1$ because ${\tau }^2\mathrm{\ne }1$.