Fixed-order autoregressive OLS martingale limit

Fix $a\in {\mathbb{R}}^d$ and $t\ge 1$. For any $A\in {\mathcal{F}}_{t-1}$, the variables $a^T{Z}_t{1}_A$ and $(a^T{Z}_t{)}^2{1}_A$ are ${\mathcal{F}}_{t-1}$-measurable. Past-independence, [F5], and $\mathbb{E}[{ϵ}_t]=0$ give $\mathbb{E}[a^T{Z}_t{ϵ}_t{1}_A]=0$; past-independence, [F5], and $\mathbb{E}[{ϵ}_t^2]={\sigma }^2$ give $\mathbb{E}[(a^T{Z}_t{)}^2{ϵ}_t^2{1}_A]={\sigma }^2\mathbb{E}[(a^T{Z}_t{)}^2{1}_A]$. By [F4] these event-integral identities say $\mathbb{E}[a^T{Z}_t{ϵ}_t\mid {\mathcal{F}}_{t-1}]=0,\mathbb{E}[(a^T{Z}_t{ϵ}_t{)}^2\mid {\mathcal{F}}_{t-1}]={\sigma }^2(a^T{Z}_t{)}^2\mathrm{.}$ Factorization also gives $\mathbb{E}[(a^T{Z}_t{ϵ}_t{)}^2]={\sigma }^2{a}^T\mathrm{\Gamma }a<\mathrm{\infty }$, so each score increment is square-integrable.

For each coordinate pair $(i,j)$, the sequence $V_k^{ij}:=Z_{k,i}{Z}_{k,j}$, $k\ge 1$, is an integrable measurable function of the finite block $(X_{k-p},\dots ,X_{k-1})$. Cauchy–Schwarz gives $\mathbb{E}\mathrm{\mid }{V}_k^{ij}\mathrm{\mid }\le (\mathbb{E}{Z}_{k,i}^2{)}^{1\mathrm{/}2}(\mathbb{E}{Z}_{k,j}^2{)}^{1\mathrm{/}2}<\mathrm{\infty }$. The one-sided process $W_m:=X_{m+1-p}$, $m\ge 0$, has the same canonical path law as the one-sided restriction of $X$, by strict stationarity. Each lag block $(X_{k-p},\dots ,X_{k-1})$ is a fixed finite-block function of $W_{k-1},\dots ,W_{k+p-2}$. A shift-invariant event for this factor pulls back to a shift-invariant event for $W$, so the block process is ergodic. Applying [F2] componentwise gives $G_n\to \mathrm{\Gamma }$ almost surely, since there are finitely many entries.

By [F15], the positive eigenvalues of $\mathrm{\Gamma }$ have positive product, so $\det \mathrm{\Gamma }>0$. The determinant and cofactors are polynomials in the matrix entries, hence $\det G_n\to \det \mathrm{\Gamma }$ almost surely. Therefore $\mathbb{P}(G_n\text{ is singular})\to 0$. On the invertible event, the adjugate formula $G_n^{-1}=\mathrm{adj}(G_n)\mathrm{/}\det G_n$ gives inverse convergence on the invertible event. Define $H_n=G_n^{-1}$ there and $H_n={\mathrm{\Gamma }}^{-1}$ otherwise; then $H_n\to {\mathrm{\Gamma }}^{-1}$ in probability. The OLS fallback is measurable because $G_n$ and $\det G_n$ are measurable, and inversion is continuous on the invertible matrices.

Fix $a\mathrm{\ne }0$ and put ${\lambda }_a:={\sigma }^2{a}^T\mathrm{\Gamma }a>0$. For each $n$, define $D_{n,k}:=\frac{a^T{Z}_k{ϵ}_k}{\sqrt{n{\lambda }_a}}(1\le k\le n),D_{n,k}:=0(k>n),$ with filtration ${\mathcal{F}}_k$. By step 1.1 the nonzero terms are square-integrable martingale differences; the padded terms preserve that property.

By step 1.1, the terminal predictable variance of this row is ${\mathrm{\Gamma }}_{n,\mathrm{\infty }}=\frac{1}{n{a}^T\mathrm{\Gamma }a}{\sum }_{k=1}^n(a^T{Z}_k{)}^2\mathrm{.}$ The squared projection is integrable, stationary and ergodic as a measurable function of the lag block by step 1.2. Birkhoff [F2] makes this average converge almost surely to $\mathbb{E}[(a^T{Z}_t{)}^2]\mathrm{/}(a^T\mathrm{\Gamma }a)=1$.

Let $W_k:=a^T{Z}_k{ϵ}_k$. The AR equation makes $W_k$ the same finite-block measurable function of the strictly stationary process $X$ at each time, so the $W_k$ are identically distributed; step 1.1 gives $\mathbb{E}[W_1^2]={\lambda }_a<\mathrm{\infty }$. For every $\delta >0$, the unconditional Lindeberg sum is ${\sum }_{k=1}^n\mathbb{E}[D_{n,k}^2{1}_{\{\mathrm{\mid }{D}_{n,k}\mathrm{\mid }>\delta \}}]=\frac{1}{{\lambda }_a}\mathbb{E}\text{\hspace{0.17em}⁣}\left[W_1^2{1}_{\{\mathrm{\mid }{W}_1\mathrm{\mid }>\delta \sqrt{n{\lambda }_a}\}}\right]⟶0$ by dominated convergence [F12], since the indicators decrease pointwise to zero and $W_1^2$ is integrable.

Steps 2.2, 2.3, and 2.4 verify the square-integrable array, variance-clock, and unconditional Lindeberg hypotheses of [F7]. Thus $a^T{S}_n\mathrm{/}\sqrt{{\lambda }_a}\Rightarrow N(0,1)$ for $S_n:=n^{-1\mathrm{/}2}{\sum }_{k=1}^n{Z}_k{ϵ}_k$. This holds for every nonzero $a$; $a=0$ is immediate. Applying [F8] to the specified normal law gives $S_n\Rightarrow N_d(0,{\sigma }^2\mathrm{\Gamma })$.

On the event that $G_n$ is invertible, the normal equations give $\sqrt{n}({\widehat{\beta }}_n-\beta )=G_n^{-1}{S}_n=H_n{S}_n\mathrm{.}$ By step 2.1, $H_n{\to }_{\mathbb{P}}{\mathrm{\Gamma }}^{-1}$; by step 3.1 the score converges in distribution. The bilinear map $(A,x)\mapsto Ax$ is continuous, so [F9] gives the limit ${\mathrm{\Gamma }}^{-1}{N}_d(0,{\sigma }^2\mathrm{\Gamma })=N_d(0,{\sigma }^2{\mathrm{\Gamma }}^{-1})$. The OLS fallback differs from $H_n{S}_n$ only on an event whose probability tends to zero by step 2.1, so bounded-continuous test expectations have the same limit.

By [F9] and step 3.1, $S_n=O_{\mathbb{P}}(1)$; step 2.1 also gives $H_n=O_{\mathbb{P}}(1)$. Consequently ${\widehat{\beta }}_n-\beta =O_{\mathbb{P}}(n^{-1\mathrm{/}2})$. On the invertible event, expanding the residual square and using the normal equations gives ${\widehat{\sigma }}_n^2=\frac{1}{n}{\sum }_{k=1}^n{ϵ}_k^2-\frac{1}{n}{S}_n^T{G}_n^{-1}{S}_n\mathrm{.}$ The second term is $O_{\mathbb{P}}(n^{-1})$. By [F10], the first term converges almost surely to ${\sigma }^2$, because ${ϵ}_t^2$ are iid and integrable. The singular-event probability tends to zero by step 2.1, so ${\widehat{\sigma }}_n^2{\to }_{\mathbb{P}}{\sigma }^2$.

The event that $G_n$ is singular or ${\widehat{\sigma }}_n^2\le {\sigma }^2\mathrm{/}2$ has probability tending to zero by steps 2.1 and 4.2. The map $h(x,A,s):=x^{T}Ax\mathrm{/}\max (s,{\sigma }^2\mathrm{/}2)$ is continuous everywhere and agrees with the Wald formula on the complement of that event. By steps 1.2, 4.1, and 4.2, the joint limit of the vector, matrix, and scalar arguments of $h$ is $(Z,\mathrm{\Gamma },{\sigma }^2)$ with $Z\sim N_d(0,{\sigma }^2{\mathrm{\Gamma }}^{-1})$. [F9] gives the limit ${\sigma }^{-2}{Z}^T\mathrm{\Gamma }Z$. By [F15], let ${\mathrm{\Gamma }}^{1\mathrm{/}2}$ be the positive square root. The definition of the multivariate normal law in [F11] gives $U={\mathrm{\Gamma }}^{1\mathrm{/}2}Z\mathrm{/}\sigma \sim N_d(0,I_d)$, so [F11] identifies ${\sigma }^{-2}{Z}^T\mathrm{\Gamma }Z=\mathrm{\parallel }U{\mathrm{\parallel }}^2\sim {\chi }_d^2$. The statistic's arbitrary zero convention on the vanishing-probability exceptional event does not change this limit.