Finite-history linear prediction

Write the mean and covariance of $(U,V)$ in block form as $(m_U,m_V)$ and $\mathrm{\Sigma }=\mathrm{diag}({\mathrm{\Sigma }}_U,{\mathrm{\Sigma }}_V)$; the off-diagonal blocks vanish by hypothesis and each diagonal block is positive semidefinite as a principal block of $\mathrm{\Sigma }$. Let $A_U,A_V$ be their positive-semidefinite square roots and put $A=\mathrm{diag}(A_U,A_V)$. Then $A$ is positive semidefinite, $A^2=\mathrm{\Sigma }$, and [F2] identifies it with the unique positive-semidefinite square root of $\mathrm{\Sigma }$.

By [F1], the joint law has a realization $(m_U+A_U{W}_U,m_V+A_V{W}_V)$ with independent standard-normal coordinates in $W_U,W_V$. By [F3] the full coordinate vector has product law, so rectangles in the two coordinate blocks factor into their block marginals; [F6] gives block independence. Applying [F4] to the separate block-affine maps preserves independence.

The realization in step 2.1 and the original vector have the same multivariate-normal law by [F5]. Their joint and marginal rectangle probabilities therefore agree; the factorization established in step 2.1 transfers to the original pair, and [F6] gives independence. AC is used only through the Gaussian-law construction and uniqueness suppliers.

Fix $n,h\ge 1$ and take target $X_{n+h}$ with regressors $X_1,\dots ,X_n$. By [F1] these variables are square-integrable, so [F2] supplies a unique minimizing almost-sure class in their affine span.

By [F3], the affine span used in [F2] is exactly $M_n$, so the unique minimizing class from step 1.1 is the finite-history forecast. The uniqueness asserted is of the $L^2$ class, not necessarily of its coefficients; this remains true when the history covariance is singular by [F2].

Since weak stationarity gives $X_{n+h},Y_1,\dots ,Y_n\in L^2$, [F1] supplies a unique minimizing class and [F2] an affine representative with coefficients satisfying the normal equations. Every variable has mean $\mu$ by [F3], so the intercept is $a_0=\mu (1-{\sum }_i{a}_i)$ and the representative is $\mu +{\sum }_i{a}_i(Y_i-\mu )$.

For each $i,j$, the lag difference between $Y_i$ and $Y_j$ is $j-i$; covariance symmetry in [F5] gives $\gamma (j-i)=\gamma (i-j)$, so $\mathrm{Cov}(Y_i,Y_j)={\mathrm{\Gamma }}_{ij}$. The lag from $Y_i=X_{n+1-i}$ to $X_{n+h}$ is $h+i-1$, so $\mathrm{Cov}(X_{n+h},Y_i)=\gamma (h+i-1)$. Rewriting step 1.1's equations gives $\mathrm{\Gamma }a={\gamma }_h$.

Under the statement's nonsingularity hypothesis, $\mathrm{\Gamma }a={\gamma }_h$ has the unique solution $a={\mathrm{\Gamma }}^{-1}{\gamma }_h$.

Put $U=X_{n+h}-\mu$ and $Z_i=Y_i-\mu$; the intercept equation makes the error mean zero, and stationarity gives $\mathbb{E}[U{Z}_i]=\gamma (h+i-1)$ and $\mathbb{E}[Z_i{Z}_j]={\mathrm{\Gamma }}_{ij}$. Expanding gives $\mathbb{E}[(U-a^{T}Z{)}^2]=\gamma (0)-2a^T{\gamma }_h+a^T\mathrm{\Gamma }a=\gamma (0)-a^T{\gamma }_h$ because $\mathrm{\Gamma }a={\gamma }_h$. By [F6] this is the error variance.

Let $Y=(X_n,\dots ,X_1)$ and choose an affine representative ${\widehat{X}}_{n+h}$ of the unique class in [F1]. Put $R=X_{n+h}-{\widehat{X}}_{n+h}$. By [F2], its mean is zero by the intercept equation, and each normal equation gives $\mathrm{Cov}(X_{n+h},Y_i)=\mathrm{Cov}({\widehat{X}}_{n+h},Y_i)$, hence $\mathrm{Cov}(R,Y_i)=0$.

The vector $(R,Y)$ is an affine linear image of $(X_{n+h},Y)$, so every scalar linear projection is normal by [F3] and $(R,Y)$ is jointly multivariate normal. Step 1.1 gives zero cross-covariance between $R$ and each coordinate of $Y$, so [F4] makes $R$ independent of $Y$ and therefore of $\mathcal{G}:=\sigma (Y)$.

The forecast ${\widehat{X}}_{n+h}$ is an affine function of $Y$, hence $\mathcal{G}$-measurable and integrable. By [F5], $\mathbb{E}[R\mid \mathcal{G}]=\mathbb{E}[R]=0$ and $\mathbb{E}[{\widehat{X}}_{n+h}\mid \mathcal{G}]={\widehat{X}}_{n+h}$. Since $X_{n+h}=R+{\widehat{X}}_{n+h}$, [F6] makes the forecast a version of $\mathbb{E}[X_{n+h}\mid \mathcal{G}]$ under [F7]; uniqueness in [F6] gives the almost-sure equality. AC is used only through the Gaussian and conditional-expectation branches.

For $k=1$, positive definiteness of ${\mathrm{\Gamma }}_1=[\gamma (0)]$ gives $\gamma (0)>0$. The algorithm sets ${\varphi }_{1,1}=\gamma (1)\mathrm{/}\gamma (0)$, so ${\mathrm{\Gamma }}_1{\varphi }_1=r_1$, and [F2] gives error variance $\gamma (0)-\gamma (1{)}^2\mathrm{/}\gamma (0)={\nu }_1$, including the zero-error case.

For the induction step fix $2\le k\le m$ and assume the claim at order $k-1$. Write $r=(\gamma (1),\dots ,\gamma (k-1){)}^T$; covariance symmetry gives ${\mathrm{\Gamma }}_k=\left(\begin{array}{cc}\gamma (0)& r^T\\ r& {\mathrm{\Gamma }}_{k-1}\end{array}\right)$. The induction hypothesis gives ${\varphi }_{k-1}={\mathrm{\Gamma }}_{k-1}^{-1}r$ and ${\nu }_{k-1}=\gamma (0)-r^T{\mathrm{\Gamma }}_{k-1}^{-1}r$. The positive quadratic form of ${\mathrm{\Gamma }}_k$ at $(1,-{\mathrm{\Gamma }}_{k-1}^{-1}r{)}^T$ equals this value, so ${\nu }_{k-1}>0$ and the order-$k$ division is valid.

Set $T={\sum }_{j=1}^{k-1}{\varphi }_{k-1,j}\gamma (k-j)$ and ${\kappa }_k=(\gamma (k)-T)\mathrm{/}{\nu }_{k-1}$. For $i<k$, the new coefficient rule gives $({\mathrm{\Gamma }}_k{\varphi }_k{)}_i={\sum }_{j=1}^{k-1}\gamma (i-j){\varphi }_{k-1,j}-{\kappa }_k{\sum }_{j=1}^{k-1}\gamma (i-j){\varphi }_{k-1,k-j}+{\kappa }_k\gamma (i-k)$. The first sum is $\gamma (i)$ by the old system; reindexing the second by $\mathrm{\ell }=k-j$ gives its row $k-i$, namely $\gamma (k-i)$. Since $\gamma (i-k)=\gamma (k-i)$, these last terms cancel and $({\mathrm{\Gamma }}_k{\varphi }_k{)}_i=\gamma (i)$.

Let $S=r^T{\varphi }_{k-1}=\gamma (0)-{\nu }_{k-1}$. Reindexing the last row gives ${\sum }_{j=1}^{k-1}\gamma (k-j){\varphi }_{k-1,j}-{\kappa }_k{\sum }_{j=1}^{k-1}\gamma (k-j){\varphi }_{k-1,k-j}+{\kappa }_k\gamma (0)=T+{\kappa }_k(\gamma (0)-S)=T+{\kappa }_k{\nu }_{k-1}=\gamma (k)$. Thus ${\mathrm{\Gamma }}_k{\varphi }_k=r_k$, and positive definiteness makes the solution unique.

Reindexing the new vector gives $r_k^T{\varphi }_k=S-{\kappa }_{k}T+{\kappa }_k\gamma (k)=S+{\kappa }_k^2{\nu }_{k-1}$ by the definition of ${\kappa }_k$. Therefore $\gamma (0)-r_k^T{\varphi }_k={\nu }_{k-1}(1-{\kappa }_k^2)={\nu }_k$. The forecast theorem [F2] identifies this with the one-step projection-error variance, completing the induction through $k=m$.