Likelihood Theory and Classical Calibration — Examples

Put $q_{\mu }(x)=(2\pi {\sigma }^2{)}^{-1\mathrm{/}4}{e}^{-(x-\mu {)}^2\mathrm{/}(4{\sigma }^2)}$. Direct differentiation gives $q_{\mu }^{\mathrm{\prime }}=(x-\mu )q_{\mu }\mathrm{/}(2{\sigma }^2)$ and $q_{\mu }^{\mathrm{\prime }\mathrm{\prime }}=((x-\mu {)}^2\mathrm{/}(4{\sigma }^4)-1\mathrm{/}(2{\sigma }^2))q_{\mu }$. Its square is a polynomial times $p_{\mu }=q_{\mu }^2$. By [F11] and [F12], $\int (x-\mu {)}^2{p}_{\mu }dx={\sigma }^2$ and $\int (x-\mu {)}^4{p}_{\mu }dx=3{\sigma }^4$, so $\mathrm{\parallel }{q}_{\mu }^{\mathrm{\prime }\mathrm{\prime }}{\mathrm{\parallel }}_2^2=3\mathrm{/}(16{\sigma }^4)<\mathrm{\infty }$. Also $s_{\mu }=2q_{\mu }^{\mathrm{\prime }}\mathrm{/}{q}_{\mu }=(x-\mu )\mathrm{/}{\sigma }^2$ has second moment $1\mathrm{/}{\sigma }^2$. The identity $q_a^{\mathrm{\prime }\mathrm{\prime }}(x)=q_0^{\mathrm{\prime }\mathrm{\prime }}(x-a)$ and [F15] give the same norm for every real shift $a$. The two-variable functions $(x,v)\mapsto q_{\mu +v}^{\mathrm{\prime }\mathrm{\prime }}(x)$ and $(x,u)\mapsto q_{\mu +u}^{\mathrm{\prime }}(x)-q_{\mu }^{\mathrm{\prime }}(x)$ are continuous. To verify the needed product measurability locally, the rectangles $(a,b)\times (c,d)$ with rational endpoints form a countable base of ${\mathbb{R}}^2$: inside any open ball about a point, density of the rationals gives a rational rectangle containing that point and contained in the ball. Every open subset is therefore the union of the subfamily of these rectangles contained in it, hence a countable union of measurable product rectangles. Thus preimages of open subsets under these continuous functions belong to $\mathcal{B}(\mathbb{R})\otimes \mathcal{B}(\mathbb{R})$, and also to the product of the Lebesgue sigma-algebras. Their squares and restrictions to compact parameter intervals are consequently product measurable, as required by Tonelli below.

For fixed $x$, the functions $v\mapsto q_{\mu +v}^{\mathrm{\prime }\mathrm{\prime }}(x)$ and $u\mapsto q_{\mu +u}^{\mathrm{\prime }}(x)-q_{\mu }^{\mathrm{\prime }}(x)$ are continuous on each compact interval. By [F13] they are Riemann integrable, by [F9] their integrals are respectively $q_{\mu +u}^{\mathrm{\prime }}(x)-q_{\mu }^{\mathrm{\prime }}(x)$ and $q_{\mu +t}(x)-q_{\mu }(x)-t{q}_{\mu }^{\mathrm{\prime }}(x)$, and [F14] identifies these with their Lebesgue integrals under the assumed AC. Reversed endpoints give the same identities for negative $u,t$.

Cauchy–Schwarz [F7] on the interval between $0$ and $u$, then Tonelli [F8] and step 1.1, gives $\mathrm{\parallel }{q}_{\mu +u}^{\mathrm{\prime }}-q_{\mu }^{\mathrm{\prime }}{\mathrm{\parallel }}_2^2\le u^2\mathrm{\parallel }{q}_0^{\mathrm{\prime }\mathrm{\prime }}{\mathrm{\parallel }}_2^2$. The second identity of step 2.1 and another Cauchy–Schwarz/Tonelli application give $\mathrm{\parallel }{q}_{\mu +t}-q_{\mu }-t{q}_{\mu }^{\mathrm{\prime }}{\mathrm{\parallel }}_2^2\le \mathrm{\mid }t\mathrm{\mid }{\int }_0^{\mathrm{\mid }t\mathrm{\mid }}{u}^2\mathrm{\parallel }{q}_0^{\mathrm{\prime }\mathrm{\prime }}{\mathrm{\parallel }}_2^{2}du=\mathrm{\mid }t{\mathrm{\mid }}^4\mathrm{\parallel }{q}_0^{\mathrm{\prime }\mathrm{\prime }}{\mathrm{\parallel }}_2^2\mathrm{/}3=o(t^2)$. For negative $t$ use $q_{\mu -u}^{\mathrm{\prime }}$ on the length-$\mathrm{\mid }t\mathrm{\mid }$ interval; the same first bound holds. This proves QMD with the explicitly supplied normal densities, score $s_{\mu }$, and positive information $1\mathrm{/}{\sigma }^2$ by [F5].

Use the global chart $V=\mathbb{R}$ with tested coordinate $\psi =\mu -{\mu }_0$, so both maximizers always lie in the chart and the off-chart conventions in [F6] never apply. The log likelihood is a constant minus $(2{\sigma }^2{)}^{-1}{\sum }_i(X_i-\mu {)}^2$. The identity ${\sum }_i(X_i-\mu {)}^2={\sum }_i(X_i-\bar{X}{)}^2+n(\mu -\bar{X}{)}^2$ proves $\widehat{\mu }=\bar{X}$. Since ${\sigma }^2$ is known, take the exact covariance and information estimates $\widehat{V}={\sigma }^2$ and ${\widehat{I}}_{\mathrm{e}\mathrm{f}\mathrm{f}}=1\mathrm{/}{\sigma }^2$; the Wald statistic is $W_n=n(\bar{X}-{\mu }_0{)}^2\mathrm{/}{\sigma }^2$. At the point null the efficient score is $n^{-1\mathrm{/}2}{\sum }_i(X_i-{\mu }_0)\mathrm{/}{\sigma }^2$, with information $1\mathrm{/}{\sigma }^2$; [F6] gives $S_n=n(\bar{X}-{\mu }_0{)}^2\mathrm{/}{\sigma }^2$. The constrained MLE is ${\mu }_0$, so twice the log-likelihood difference gives $L_n=n(\bar{X}-{\mu }_0{)}^2\mathrm{/}{\sigma }^2$.

Under $H_0$, put $Z_n=\sqrt{n}(\bar{X}-{\mu }_0)\mathrm{/}\sigma$. The affine and independent-sum characteristic-function identities [F3], together with [F2], give $\mathbb{E}{e}^{it{Z}_n}={\prod }_i\mathbb{E}{e}^{it(X_i-{\mu }_0)\mathrm{/}(\sigma \sqrt{n})}=[e^{-t^2\mathrm{/}(2n)}{]}^n=e^{-t^2\mathrm{/}2}$. By [F4], $Z_n\sim N(0,1)$, so $W_n=S_n=L_n=Z_n^2$ has the ${\chi }_1^2$ law by [F10]. The use of AC is exactly that recorded in [A1].

Put $q_{\mu }(x)=(2\pi {\sigma }^2{)}^{-1\mathrm{/}4}{e}^{-(x-\mu {)}^2\mathrm{/}(4{\sigma }^2)}$. Direct differentiation gives $q_{\mu }^{\mathrm{\prime }}=(x-\mu )q_{\mu }\mathrm{/}(2{\sigma }^2)$ and $q_{\mu }^{\mathrm{\prime }\mathrm{\prime }}=((x-\mu {)}^2\mathrm{/}(4{\sigma }^4)-1\mathrm{/}(2{\sigma }^2))q_{\mu }$. Its square is a polynomial times $p_{\mu }=q_{\mu }^2$. By [F11] and [F12], $\int (x-\mu {)}^2{p}_{\mu }dx={\sigma }^2$ and $\int (x-\mu {)}^4{p}_{\mu }dx=3{\sigma }^4$, so $\mathrm{\parallel }{q}_{\mu }^{\mathrm{\prime }\mathrm{\prime }}{\mathrm{\parallel }}_2^2=3\mathrm{/}(16{\sigma }^4)<\mathrm{\infty }$. Also $s_{\mu }=2q_{\mu }^{\mathrm{\prime }}\mathrm{/}{q}_{\mu }=(x-\mu )\mathrm{/}{\sigma }^2$ has second moment $1\mathrm{/}{\sigma }^2$. The identity $q_a^{\mathrm{\prime }\mathrm{\prime }}(x)=q_0^{\mathrm{\prime }\mathrm{\prime }}(x-a)$ and [F15] give the same norm for every real shift $a$. The two-variable functions $(x,v)\mapsto q_{\mu +v}^{\mathrm{\prime }\mathrm{\prime }}(x)$ and $(x,u)\mapsto q_{\mu +u}^{\mathrm{\prime }}(x)-q_{\mu }^{\mathrm{\prime }}(x)$ are continuous. To verify the needed product measurability locally, the rectangles $(a,b)\times (c,d)$ with rational endpoints form a countable base of ${\mathbb{R}}^2$: inside any open ball about a point, density of the rationals gives a rational rectangle containing that point and contained in the ball. Every open subset is therefore the union of the subfamily of these rectangles contained in it, hence a countable union of measurable product rectangles. Thus preimages of open subsets under these continuous functions belong to $\mathcal{B}(\mathbb{R})\otimes \mathcal{B}(\mathbb{R})$, and also to the product of the Lebesgue sigma-algebras. Their squares and restrictions to compact parameter intervals are consequently product measurable, as required by Tonelli below.

The log likelihood is a constant minus $(2{\sigma }^2{)}^{-1}{\sum }_i(X_i-\mu {)}^2$. Completing the square shows that the unrestricted MLE is $\widehat{\mu }=\bar{X}$ and the constrained MLE is $\tilde{\mu }=\min (0,\bar{X})$: when $\bar{X}\le 0$ the unrestricted maximum is feasible, and when $\bar{X}>0$ the closest feasible point is $0$.

For fixed $x$, the functions $v\mapsto q_{\mu +v}^{\mathrm{\prime }\mathrm{\prime }}(x)$ and $u\mapsto q_{\mu +u}^{\mathrm{\prime }}(x)-q_{\mu }^{\mathrm{\prime }}(x)$ are continuous on each compact interval. By [F13] they are Riemann integrable, by [F10] their integrals are respectively $q_{\mu +u}^{\mathrm{\prime }}(x)-q_{\mu }^{\mathrm{\prime }}(x)$ and $q_{\mu +t}(x)-q_{\mu }(x)-t{q}_{\mu }^{\mathrm{\prime }}(x)$, and [F14] identifies these with their Lebesgue integrals under the assumed AC. Reversed endpoints give the same identities for negative $u,t$.

Substituting these maximizers into [F1] gives $L_n=2\{{\mathrm{\ell }}_n(\widehat{\mu })-{\mathrm{\ell }}_n(\tilde{\mu })\}=\frac{n({\bar{X}}_{+}{)}^2}{{\sigma }^2},{\bar{X}}_{+}=\max (\bar{X},0)\mathrm{.}$ This follows from ${\sum }_i(X_i-\mu {)}^2={\sum }_i(X_i-\bar{X}{)}^2+n(\mu -\bar{X}{)}^2$: the difference is zero for $\bar{X}\le 0$, and otherwise it is $n{\bar{X}}^2\mathrm{/}{\sigma }^2$ after multiplication by $2$.

Cauchy–Schwarz [F8] on the interval between $0$ and $u$, then Tonelli [F9] and step 1.1, gives $\mathrm{\parallel }{q}_{\mu +u}^{\mathrm{\prime }}-q_{\mu }^{\mathrm{\prime }}{\mathrm{\parallel }}_2^2\le u^2\mathrm{\parallel }{q}_0^{\mathrm{\prime }\mathrm{\prime }}{\mathrm{\parallel }}_2^2$. The second identity of step 2.1 and another Cauchy–Schwarz/Tonelli application give $\mathrm{\parallel }{q}_{\mu +t}-q_{\mu }-t{q}_{\mu }^{\mathrm{\prime }}{\mathrm{\parallel }}_2^2\le \mathrm{\mid }t\mathrm{\mid }{\int }_0^{\mathrm{\mid }t\mathrm{\mid }}{u}^2\mathrm{\parallel }{q}_0^{\mathrm{\prime }\mathrm{\prime }}{\mathrm{\parallel }}_2^{2}du=\mathrm{\mid }t{\mathrm{\mid }}^4\mathrm{\parallel }{q}_0^{\mathrm{\prime }\mathrm{\prime }}{\mathrm{\parallel }}_2^2\mathrm{/}3=o(t^2)$. For negative $t$ use $q_{\mu -u}^{\mathrm{\prime }}$ on the length-$\mathrm{\mid }t\mathrm{\mid }$ interval; the same first bound holds. This proves QMD with the explicitly supplied normal densities, score $s_{\mu }$, and positive information $1\mathrm{/}{\sigma }^2$ by [F2].

Under $\mu =0$, put $Z=\sqrt{n}\bar{X}\mathrm{/}\sigma$. By [F4] and [F5], $\mathbb{E}{e}^{itZ}={\prod }_{i=1}^n{e}^{-t^2\mathrm{/}(2n)}=e^{-t^2\mathrm{/}2}$; uniqueness [F6] gives $Z\sim N(0,1)$ exactly. Its even density [F3] gives $\mathbb{P}(Z\le 0)=1\mathrm{/}2$, and for $t\ge 0$ the distribution function of $L_n=(Z_{+}{)}^2$ is $\mathbb{P}(Z\le \sqrt{t})=\mathrm{\Phi }(\sqrt{t})$, where $\mathrm{\Phi }(x):=\mathbb{P}(Z\le x)$. Thus its law is $\frac{1}{2}{\delta }_0+\frac{1}{2}{\chi }_1^2$, by [F7]. Since [F3] gives $\mathbb{P}(Z=0)=0$, ${\chi }_1^2$ has no atom at zero, so this mixture is not the ordinary ${\chi }_1^2$ law. The failure comes from the null's half-line local tangent, not a boundary of the full parameter space.