Finite-history linear prediction — Examples

Reindexing the convergent causal series gives $X_t-\mu =\varphi (X_{t-1}-\mu )+{ϵ}_t$; repeating this identity gives $X_{n+h}-\mu ={\varphi }^h(X_n-\mu )+R_h$, where $R_h={\sum }_{r=0}^{h-1}{\varphi }^r{ϵ}_{n+h-r}$. Every innovation in $R_h$ has index greater than $n$, while the causal series for each $X_j-\mu$, $1\le j\le n$, uses only indices at most $j$.

For each $j\le n$, every finite partial sum for $X_j-\mu$ uses innovation indices at most $j$, so its covariance with each future term of $R_h$ is zero by the uncorrelated-noise assumption; [F4] passes this covariance identity to the $L^2$ limit. Thus $\mathrm{Cov}(R_h,X_j)=0$ and $\mathbb{E}[R_h]=0$, making the residual orthogonal to the constant and every history variable.

Let $H=\mu +{\varphi }^h(X_n-\mu )$ and let any competing affine forecast $Z$ be in the history span; then $D:=H-Z$ is also in that span. Step 2.1 gives $\mathbb{E}[R_h]=0$ and $\mathrm{Cov}(R_h,X_j)=0$ for every $j$; bilinearity [F3] gives $\mathrm{Cov}(R_h,D)=0$, hence $\mathbb{E}[R_{h}D]=0$. Since $X_{n+h}-Z=R_h+D$, expansion yields $\mathbb{E}[(X_{n+h}-Z{)}^2]=\mathbb{E}[R_h^2]+\mathbb{E}[D^2]\ge \mathbb{E}[R_h^2]$, with equality exactly when $D=0$ as an $L^2$ class. Thus $H$ is the unique best class by [F2]. The innovations in $R_h$ are pairwise uncorrelated, so [F3] gives $\mathrm{Var}(R_h)={\sigma }^2{\sum }_{r=0}^{h-1}{\varphi }^{2r}$. This proves both formulas without Gaussian or independent innovations.

Each outcome has weight $1\mathrm{/}2$, so [F1] gives $\mathbb{E}[Z]=0$ and $\mathbb{E}[Z^2]=1$. Every $X_t=Z$ is square-integrable with mean zero, and every finite-dimensional law is shift-invariant because each coordinate is the same $Z$; hence $X$ is weakly stationary.

The reversed history is $(Y_1,Y_2)=(X_2,X_1)=(Z,Z)$ and the target is $X_3=Z$. Direct covariance calculation gives $\mathrm{\Gamma }=\left(\begin{array}{cc}1& 1\\ 1& 1\end{array}\right)$ and ${\gamma }_h=(1,1{)}^T$; by [F4] the normal equations reduce to $a_1+a_2=1$, so the matrix is singular and has solutions $(1,0{)}^T$ and $(0,1{)}^T$.

The intercept is zero because every variable has mean zero; either displayed solution gives forecast $a_{1}Z+a_{2}Z=Z$ and zero error, so they represent the same unique minimizing class by [F4].

A common shift $s$ changes phase $\omega$ to $(\omega +s)<mrow><mi\; mathvariant="normal">m</mi><mi\; mathvariant="normal">o</mi><mi\; mathvariant="normal">d</mi></mrow>3$, a permutation of the equally weighted outcomes, so every shifted finite coordinate vector has the original law. One-coordinate laws give a common mean, pair laws at each lag give covariance independent of $t$, and boundedness gives finite moments; hence the process is weakly stationary by [F3].

The values of $(X_1,X_2)$ at $\omega =0,1,2$ are $(-1,0),(0,1),(1,-1)$; hence both means are zero, $\mathrm{Var}(X_1)=2\mathrm{/}3$, and $\mathrm{Cov}(X_2,X_1)=-1\mathrm{/}3$. By [F4], the unique best affine forecast from this one-variable history is $(-1\mathrm{/}3)\mathrm{/}(2\mathrm{/}3)X_1=-X_1\mathrm{/}2$.

Since $X_1:\mathrm{\Omega }\to \{-1,0,1\}$ is bijective, $\sigma (X_1)$ is the full event sigma-algebra and $X_2$ is measurable with respect to it. The conditional-expectation definition and known-variable property give $\mathbb{E}[X_2\mid \sigma (X_1)]=X_2$, with values $(0,1,-1)$ at $\omega =0,1,2$. Every outcome has positive probability; if $X_2=a+b{X}_1$ almost surely, the outcomes with $X_1=0$ and $X_1=-1$ force $a=b=1$, which would give $X_2=2$ when $X_1=1$, contrary to its value $-1$. Thus the conditional mean is not affine in $X_1$, while step 2.1 gives the best affine forecast $-X_1\mathrm{/}2$.