Normalized product likelihood expansion under QMD

Write $p_u=p_{{\theta }_0+u}$, $S=\{p_0>0\}$, and $\alpha (u)={\int }_{S^c}{p}_{u}d\mu$. For small $u$, [F3] gives $\alpha (u)<1\mathrm{/}2$. Define $q_u={\mathbf{1}}_S{p}_u\mathrm{/}((1-\alpha (u))p_0)$ on $S$ and $q_u=0$ on $S^c$. It is nonnegative and measurable, and $\int q_{u}d{P}_0=(1-\alpha (u){)}^{-1}{\int }_S{p}_{u}d\mu =1,$ so it is a probability density. At $u=0$, $\alpha (0)=0$ and $q_0=1$ $P_0$-almost surely, since $P_0(S^c)={\int }_{S^c}{p}_{0}d\mu =0$. On $S$, $\sqrt{q_u}=(1-\alpha (u){)}^{-1\mathrm{/}2}(1+\frac{1}{2}{u}^{T}s+\frac{r_u}{\sqrt{p_0}})\mathrm{.}$ The residual term has $L^2(P_0)$ norm $\mathrm{\parallel }{r}_u{\mathbf{1}}_S{\mathrm{\parallel }}_{L^2(\mu )}=o(\mathrm{\parallel }u{\mathrm{\parallel }}_2)$. Also $\mathrm{\mid }(1-\alpha {)}^{-1\mathrm{/}2}-1\mathrm{\mid }=\frac{\alpha }{\sqrt{1-\alpha }(1+\sqrt{1-\alpha })}=O(\alpha ),$ and [F2], [F8] give $\mathrm{\parallel }{u}^{T}s{\mathrm{\parallel }}_{L^2(P_0)}=O(\mathrm{\parallel }u{\mathrm{\parallel }}_2)$. Therefore $Y_u:=2(\sqrt{q_u}-1)=u^{T}s+R_u,\mathrm{\parallel }{R}_u{\mathrm{\parallel }}_{L^2(P_0)}=o(\mathrm{\parallel }u{\mathrm{\parallel }}_2)\mathrm{.}$ This is the required square-root expansion.

The product function $L_n={\prod }_i{q}_{u_n}(X_i)$ is a density of ${\bar{P}}_n^{\otimes n}$ relative to $P_0^{\otimes n}$. For a measurable rectangle $A_1\times \cdots \times A_n$, Tonelli gives ${\int }_{A_1\times \cdots \times A_n}{\prod }_i{q}_{u_n}(x_i)d{P}_0^{\otimes n}={\prod }_i{\int }_{A_i}{q}_{u_n}d{P}_0={\prod }_i{\bar{P}}_n(A_i)\mathrm{.}$ By [F6], agreement on rectangles identifies the product measures. Taking the whole space gives $E_{P_0^{\otimes n}}{L}_n=1$.

Since $E_0{q}_u=1$, $E_0{Y}_u=-\frac{1}{4}{E}_0{Y}_u^2\mathrm{.}$ By step 1.1, [F2], and [F8], $E_0{Y}_u^2=u^{T}Iu+o(\mathrm{\parallel }u{\mathrm{\parallel }}_2^2),E_0{Y}_u=-\frac{1}{4}{u}^{T}Iu+o(\mathrm{\parallel }u{\mathrm{\parallel }}_2^2)\mathrm{.}$ Indeed, the difference between $Y_u$ and $u^{T}s$ has $L^2(P_0)$ norm $o(\mathrm{\parallel }u{\mathrm{\parallel }}_2)$, so Cauchy–Schwarz makes the difference of their squared norms $o(\mathrm{\parallel }u{\mathrm{\parallel }}_2^2)$.

Apply [F4] to $Z_i=\mathrm{\parallel }s(X_i){\mathrm{\parallel }}_2^2$, whose mean is finite by [F2]. Then $\frac{1}{n}{\sum }_{i=1}^n(h_n^{T}s(X_i){)}^2\to h^{T}Ih$ in probability: for fixed $h$ this is [F4] applied to $(h^{T}s{)}^2$, and the difference for $h_n$ is bounded by $\mathrm{\parallel }{h}_n-h{\mathrm{\parallel }}_2\mathrm{\parallel }{h}_n+h{\mathrm{\parallel }}_2\frac{1}{n}{\sum }_{i=1}^n\mathrm{\parallel }s(X_i){\mathrm{\parallel }}_2^2\to 0.$ Step 1.1 and [F8] also give ${\sum }_{i=1}^n{Y}_{u_n}(X_i{)}^2-\frac{1}{n}{\sum }_{i=1}^n(h_n^{T}s(X_i){)}^2\to 0$ in probability, because its expected absolute value is at most $n\mathrm{\parallel }{R}_{u_n}{\mathrm{\parallel }}_2(\mathrm{\parallel }{Y}_{u_n}{\mathrm{\parallel }}_2+\mathrm{\parallel }{u}_n^{T}s{\mathrm{\parallel }}_2)=o(n\mathrm{\parallel }{u}_n{\mathrm{\parallel }}_2^2)=o(1)$. Consequently ${\sum }_i{Y}_{u_n}(X_i{)}^2\to h^{T}Ih$.

Put $V_n=\sqrt{n}{Y}_{u_n}(X_1)$. Step 1.1 implies $V_n-h_n^{T}s(X_1)\to 0$ in $L^2(P_0)$, so $V_n\to h^{T}s(X_1)$ in $L^2$. By [F8], $V_n^2\to (h^{T}s(X_1){)}^2$ in $L^1$; [F9] makes $(V_n^2)$ uniformly integrable. For every $\epsilon >0$, [F10] and the union bound give $P_0^{\otimes n}\text{\hspace{0.17em}⁣}({\max }_{i\le n}\mathrm{\mid }{Y}_{u_n}(X_i)\mathrm{\mid }>\epsilon )\le {\epsilon }^{-2}{E}_0[V_n^2{\mathbf{1}}_{\{\mathrm{\mid }{V}_n\mathrm{\mid }>\epsilon \sqrt{n}\}}]\to 0.$ Also $P_0^{\otimes n}(L_n=0)\le n\beta (u_n)\to 0$ by [F3] and the union bound.

Let $R_{u_n}=Y_{u_n}-u_n^{T}s$. Its centered IID sum has variance at most $n{E}_0{R}_{u_n}^2=o(n\mathrm{\parallel }{u}_n{\mathrm{\parallel }}_2^2)=o(1)$. By step 2.1, $n{E}_0{R}_{u_n}=n{E}_0{Y}_{u_n}=-\frac{1}{4}{h}_n^{T}I{h}_n+o(1)\to -\frac{1}{4}{h}^{T}Ih\mathrm{.}$ Markov's inequality [F10], applied to the square of the centered sum, gives ${\sum }_i{Y}_{u_n}(X_i)=h_n^T{\mathrm{\Delta }}_n-\frac{1}{4}{h}^{T}Ih+o_P(1)\mathrm{.}$ This is the required expansion of the sum.

On the event that every $q_{u_n}(X_i)>0$ and ${\max }_i\mathrm{\mid }{Y}_{u_n}(X_i)\mathrm{\mid }\le 1$, Taylor's formula gives $\log q_{u_n}(X_i)=Y_{u_n}(X_i)-\frac{1}{4}{Y}_{u_n}(X_i{)}^2+O(\mathrm{\mid }{Y}_{u_n}(X_i){\mathrm{\mid }}^3),$ with one constant valid for all $i,n$ by [F11], since the third derivative of $y\mapsto 2\log (1+y\mathrm{/}2)$ is bounded on $[-1,1]$. The total cubic remainder is bounded by a constant times ${\max }_i\mathrm{\mid }{Y}_{u_n}(X_i)\mathrm{\mid }{\sum }_i{Y}_{u_n}(X_i{)}^2$, which tends to zero by the square-sum limit in step 2.2 and the maximum bound in step 2.3. Combining the sum expansion in step 3.1 with the square-sum limit gives $\log L_n={\sum }_i\log q_{u_n}(X_i)=h_n^T{\mathrm{\Delta }}_n-\frac{1}{2}{h}^{T}Ih+o_P(1)\mathrm{.}$ This is the normalized log-likelihood expansion.

The full one-observation law decomposes as $P_{u_n}=(1-{\alpha }_n){\bar{P}}_n+{\alpha }_n{S}_n$, where $S_n$ is the normalized restriction of $P_{u_n}$ to $S^c$ when ${\alpha }_n>0$. In the product mixture, the probability that at least one coordinate is drawn from $S_n$ is at most $n{\alpha }_n$. Conditioning on the all-absolutely-continuous component shows, for every event, its probability under the full product differs from that under ${\bar{P}}_n^{\otimes n}$ by at most $n{\alpha }_n$. Since $n{\alpha }_n=o(n\mathrm{\parallel }{u}_n{\mathrm{\parallel }}_2^2)=o(1)$, [F7] proves the total-variation claim. If ${\alpha }_n=0$, the singular component is absent and the same bound is zero.