Linear processes, white noise, and ARMA models

Let $X=(X_t{)}_{t\in \mathbb{Z}}$ be a real process on one probability space. The backshift is $(BX{)}_t:=X_{t-1}$, and for a polynomial $a(z)={\sum }_{j=0}^m{a}_j{z}^j$ set $a(B)X_t:={\sum }_{j=0}^m{a}_j{X}_{t-j}$. Weak stationarity and the covariance convention are those of Strict and weak stationarity for integer-indexed time series and Moments, variance, and covariance on a probability space.

A real process $ϵ=({ϵ}_t{)}_{t\in \mathbb{Z}}$ is white noise with variance ${\sigma }^2$ when $\mathbb{E}[{ϵ}_t]=0$, $0<{\sigma }^2=\mathbb{E}[{ϵ}_t^2]<\mathrm{\infty }$, and $\mathrm{Cov}({ϵ}_t,{ϵ}_s)=0$ for all $s\mathrm{\ne }t$. This definition requires only zero covariance at distinct times; it does not require independence, identical distributions, or Gaussian laws.

For a real sequence $\psi =({\psi }_j{)}_{j\ge 0}$ with ${\sum }_{j\ge 0}\mathrm{\mid }{\psi }_j\mathrm{\mid }<\mathrm{\infty }$, a causal linear process driven by white noise is $X_t-\mu :={\sum }_{j=0}^{\mathrm{\infty }}{\psi }_j{ϵ}_{t-j},t\in \mathbb{Z},$ where the series is its almost-sure and $L^2$ limit, established by Absolutely summable linear filters preserve weak stationarity ↗.

An ARMA$(p,q)$ model, for nonnegative integers $p,q$, is an equation $\varphi (B)(X_t-\mu )=\theta (B){ϵ}_t,t\in \mathbb{Z},$ where $\varphi (z)=1-{\sum }_{j=1}^p{\varphi }_j{z}^j,\theta (z)=1+{\sum }_{j=1}^q{\theta }_j{z}^j$ have real coefficients and constant term one. If $p=0$ or $q=0$, the corresponding polynomial is $1$. The displayed equation is imposed with a coprime normalized pair, meaning that the two polynomials have no common complex root and both have constant term one. Given an unreduced pair, cancel every common nonconstant factor in $\mathbb{C}[z]$ and normalize both remaining polynomials to have constant term one to define its associated reduced model. This is a model convention: a solution of the unreduced equation need not satisfy the reduced equation. Root criteria for the reduced pair apply only to solutions of that reduced equation.

For an AR$(p)$ model ($q=0$), expansion gives the regression form $X_t=c+{\sum }_{j=1}^p{\varphi }_j{X}_{t-j}+{ϵ}_t$ with $c=\mu (1-{\sum }_{j=1}^p{\varphi }_j)$. Thus a fixed intercept in the AR regression is the original mean parameter written in observation coordinates.

A causal representation of this model is a representation $X_t-\mu ={\sum }_{j\ge 0}{\psi }_j{ϵ}_{t-j}$ with ${\sum }_j\mathrm{\mid }{\psi }_j\mathrm{\mid }<\mathrm{\infty }$. An invertible representation is ${ϵ}_t={\sum }_{j\ge 0}{\pi }_j(X_{t-j}-\mu )$ with ${\sum }_j\mathrm{\mid }{\pi }_j\mathrm{\mid }<\mathrm{\infty }$. These are properties of the displayed relation between $X$ and $ϵ$, not claims that white noise is iid.