An atomic sinusoid and an AR(1) process have different spectra

On $\mathrm{\Omega }=\{-1,1{\}}^2$ with weight $1\mathrm{/}4$ at each outcome, let $A(a,b)=\sqrt{v}a$, $B(a,b)=\sqrt{v}b$, and define the sinusoid model $X_t^{(s)}=A\cos (t{\lambda }_s)+B\sin (t{\lambda }_s)$. Direct finite sums and [F15] give $\mathbb{E}[A]=\mathbb{E}[B]=\mathbb{E}[AB]=0$ and $\mathbb{E}[A^2]=\mathbb{E}[B^2]=v$. Thus $\mathbb{E}[X_t^{(s)}]=0$, and [F16] gives ${\gamma }_s(h)=\mathbb{E}[X_{t+h}^{(s)}{X}_t^{(s)}]=v\cos (h{\lambda }_s)$, independent of $t$; this model is weakly stationary by [F1]–[F3], [F10].

Let $S_{t,N}:={\sum }_{j=0}^N{\rho }^j{\epsilon }_{t-j}$. For $M>N$, independence and centering make all cross terms vanish, so $\mathrm{\parallel }{S}_{t,M}-S_{t,N}{\mathrm{\parallel }}_2^2={\sigma }^2{\sum }_{j=N+1}^M{\rho }^{2j}⟶0$ by convergence of the real geometric series [F17]. Thus $(S_{t,N}{)}_N$ is Cauchy in $L^2$ for every $t$.

For a complex $z$ with $\mathrm{\mid }z\mathrm{\mid }<1$, put $C_N(z)={\sum }_{j=0}^N{z}^j$. The field and power laws in [F28] give $z{C}_N(z)={\sum }_{j=1}^{N+1}{z}^j$, so subtracting the two finite sums yields $(1-z)C_N(z)=1-z^{N+1}$. Since $\mathrm{\mid }1\mathrm{\mid }=1$, $z\mathrm{\ne }1$ and this factor is invertible. Multiplicativity of modulus in [F18] gives $\mathrm{\mid }{z}^{N+1}\mathrm{\mid }=\mathrm{\mid }z{\mathrm{\mid }}^{N+1}$ by finite induction, hence $\mid C_N(z)-\frac{1}{1-z}\mid =\frac{\mathrm{\mid }z{\mathrm{\mid }}^{N+1}}{\mathrm{\mid }1-z\mathrm{\mid }}⟶0$ by [F17] applied to the real number $\mathrm{\mid }z\mathrm{\mid }<1$. The metric and partial-sum definition in [F28] therefore give ${\sum }_{j=0}^{\mathrm{\infty }}{z}^j=(1-z{)}^{-1}$, and subtracting its first term gives ${\sum }_{j=1}^{\mathrm{\infty }}{z}^j=z\mathrm{/}(1-z)$. This proof includes $z=0$ with $z^0=1$.

Put $a={\lambda }_s$, $b=-{\lambda }_s$, and $F_s=(v\mathrm{/}2)({\delta }_a+{\delta }_b)$. By [F20] this is a finite positive Borel measure. It satisfies the Radon regularity clauses [F21]: for a Borel $E\subseteq \mathbb{T}$, the open set $U=\mathbb{T}\setminus \{x\in \{a,b\}:x\mathrm{\notin }E\}$ contains $E$ and has $F_s(U)=F_s(E)$; for an open $O\subseteq \mathbb{T}$, the finite set $K=O\cap \{a,b\}$ is compact and has $F_s(K)=F_s(O)$; finiteness on compact sets follows from $F_s(\mathbb{T})=v$. For a nonnegative simple $q={\sum }_j{c}_j{\mathbf{1}}_{E_j}$ with pairwise disjoint measurable $E_j$, its integral is $\int qd{F}_s={\sum }_j{c}_j(v\mathrm{/}2)({\mathbf{1}}_{a\in E_j}+{\mathbf{1}}_{b\in E_j})=(v\mathrm{/}2)(q(a)+q(b))$ by [F20] and [F24]. Increasing simple approximation [F24] and monotone convergence [F14] extend this formula to nonnegative measurable $q$, and decomposition into positive and negative real and imaginary parts extends it to bounded complex Borel $q$, which is integrable because $F_s$ is finite by [F25]. By [F26] the character $\lambda \mapsto e^{ih\lambda }$ is Borel, and its modulus is one by [F18], so it is integrable. The evaluation formula now gives ${\int }_{\mathbb{T}}{e}^{ih\lambda }{F}_s(d\lambda )=\frac{v}{2}(e^{ih{\lambda }_s}+e^{-ih{\lambda }_s})=v\cos (h{\lambda }_s)={\gamma }_s(h)$. Thus $F_s$ is a spectral measure. Its two atoms are distinct because $0<{\lambda }_s<\pi$, and each has mass $v\mathrm{/}2>0$. By [F22]–[F23], any density relative to $d\lambda$ assigns zero mass to $\{a\}$, whereas $F_s(\{a\})=v\mathrm{/}2$; hence $F_s$ has no spectral density.

By [F6] and AC, for each $t$ the Cauchy sequence $S_{t,N}$ has an $L^2$ limit class represented by a real random variable $X_t^{(\rho )}$. Cauchy–Schwarz [F7] gives $\mathbb{E}\mathrm{\mid }{\epsilon }_0\mathrm{\mid }\le \sqrt{\mathbb{E}{\epsilon }_0^2}=\sqrt{{\sigma }^2}$. Monotone convergence [F14] and the real geometric series [F17] therefore give $\mathbb{E}[{\sum }_{j=0}^{\mathrm{\infty }}\mathrm{\mid }\rho {\mathrm{\mid }}^j\mathrm{\mid }{\epsilon }_{t-j}\mathrm{\mid }]={\sum }_{j=0}^{\mathrm{\infty }}\mathrm{\mid }\rho {\mathrm{\mid }}^j\mathbb{E}\mathrm{\mid }{\epsilon }_0\mathrm{\mid }\le \sqrt{{\sigma }^2}\mathrm{/}(1-\mathrm{\mid }\rho \mathrm{\mid })<\mathrm{\infty }$. Let $A_{t,N}:={\sum }_{j=0}^N\mathrm{\mid }\rho {\mathrm{\mid }}^j\mathrm{\mid }{\epsilon }_{t-j}\mathrm{\mid }$ and $A_t:={\lim }_N{A}_{t,N}$. For each $M>0$, $M{\mathbf{1}}_{\{A_t=\mathrm{\infty }\}}\le A_t$, so monotonicity [F25] gives $M\mathbb{P}(A_t=\mathrm{\infty })\le \mathbb{E}{A}_t<\mathrm{\infty }$; letting $M$ increase shows the absolute series is finite almost surely. Each $A_{t,N}$ and $S_{t,N}$ is measurable with respect to ${\mathcal{F}}_t:=\sigma ({\epsilon }_s:s\le t)$, so [F19] makes $A_t$, $C_t:=\{A_t<\mathrm{\infty }\}$, and ${\mathrm{lim\hspace{0.17em}sup}}_N{S}_{t,N}$ measurable. On $C_t$ the absolute convergence implies $S_{t,N}$ has a finite limit. Define $Y_t$ to be ${\mathrm{lim\hspace{0.17em}sup}}_N{S}_{t,N}$ on $C_t$ and $0$ off $C_t$; it is ${\mathcal{F}}_t$-measurable and equals the signed series with value $0$ on the null divergence set. Its absolute tail satisfies $\mathbb{E}\mathrm{\mid }{Y}_t-S_{t,N}\mathrm{\mid }\le \sqrt{{\sigma }^2}\mathrm{\mid }\rho {\mathrm{\mid }}^{N+1}\mathrm{/}(1-\mathrm{\mid }\rho \mathrm{\mid })\to 0$. Since $S_{t,N}\to X_t^{(\rho )}$ in $L^2$, [F7] applied with the constant $1$ gives $\mathbb{E}\mathrm{\mid }{S}_{t,N}-X_t^{(\rho )}\mathrm{\mid }\to 0$, and hence $\mathbb{E}\mathrm{\mid }{Y}_t-X_t^{(\rho )}\mathrm{\mid }\le \mathbb{E}\mathrm{\mid }{Y}_t-S_{t,N}\mathrm{\mid }+\mathbb{E}\mathrm{\mid }{S}_{t,N}-X_t^{(\rho )}\mathrm{\mid }\to 0$. Hence $\mathbb{E}\mathrm{\mid }{Y}_t-X_t^{(\rho )}\mathrm{\mid }=0$, and [F27] implies $Y_t=X_t^{(\rho )}$ almost surely. Choose the causal version $X_t^{(\rho )}:=Y_t$. For $N\ge 1$, $S_{t,N}={\epsilon }_t+\rho S_{t-1,N-1}$, and $\mathrm{\parallel }\rho (S_{t-1,N-1}-X_{t-1}^{(\rho )}){\mathrm{\parallel }}_2=\mathrm{\mid }\rho \mathrm{\mid }\mathrm{\parallel }{S}_{t-1,N-1}-X_{t-1}^{(\rho )}{\mathrm{\parallel }}_2\to 0$, so uniqueness of $L^2$ limits gives $X_t^{(\rho )}={\epsilon }_t+\rho X_{t-1}^{(\rho )}$ in $L^2$. Finite partial sums have mean zero by [F8], and $\mathrm{\mid }\mathbb{E}{X}_t^{(\rho )}\mathrm{\mid }\le \mathrm{\parallel }{X}_t^{(\rho )}-S_{t,N}{\mathrm{\parallel }}_2\to 0$ by [F7]; hence $\mathbb{E}{X}_t^{(\rho )}=0$.

Finite second moments make every ${\epsilon }_t$ integrable by [F7], and its variance is as specified in [F9]. For $h\ge 0$, the only nonzero terms in $\mathbb{E}[S_{t+h,N}{S}_{t,N}]$ pair the identical innovation indices; by [F4]–[F5], for $N\ge h$ this expectation equals ${\sigma }^2{\sum }_{k=0}^{N-h}{\rho }^{h+2k}⟶\frac{{\sigma }^2{\rho }^h}{1-{\rho }^2}$ by [F17]. For the passage to the limit, Cauchy–Schwarz [F7] gives $\mathrm{\mid }\mathbb{E}[S_{a,N}{S}_{b,N}]-\mathbb{E}[X_a^{(\rho )}{X}_b^{(\rho )}]\mathrm{\mid }\le \mathrm{\parallel }{S}_{a,N}-X_a^{(\rho )}{\mathrm{\parallel }}_2\mathrm{\parallel }{S}_{b,N}{\mathrm{\parallel }}_2+\mathrm{\parallel }{X}_a^{(\rho )}{\mathrm{\parallel }}_2\mathrm{\parallel }{S}_{b,N}-X_b^{(\rho )}{\mathrm{\parallel }}_2\to 0,$ because $\mathrm{\parallel }{S}_{b,N}{\mathrm{\parallel }}_2\le \mathrm{\parallel }{X}_b^{(\rho )}{\mathrm{\parallel }}_2+\mathrm{\parallel }{S}_{b,N}-X_b^{(\rho )}{\mathrm{\parallel }}_2$ is bounded by the $L^2$ convergence. Covariance symmetry for these mean-zero variables follows from commutativity of real products, so the formula for $h\ge 0$ gives ${\gamma }_{\rho }(h)={\sigma }^2{\rho }^{\mathrm{\mid }h\mathrm{\mid }}\mathrm{/}(1-{\rho }^2)$ for every integer $h$, with ${\rho }^0:=1$. It is independent of $t$, so $X^{(\rho )}$ is weakly stationary by [F10].

Since $\mathrm{\mid }\rho \mathrm{\mid }<1$, the autocovariance is absolutely summable by its formula in step 3.1 and the real geometric series [F17]. Put $z_{-}=\rho e^{-i\lambda }$ and $z_{+}=\rho e^{i\lambda }$. By [F18], both have modulus $\mathrm{\mid }\rho \mathrm{\mid }<1$, so step 1.3 sums their series from index $1$ as $z_{-}\mathrm{/}(1-z_{-})$ and $z_{+}\mathrm{/}(1-z_{+})$. Applying [F12], these two sums and [F18] yield $f_{\rho }(\lambda )=\frac{{\sigma }^2}{2\pi (1-{\rho }^2)}(1+{\sum }_{h=1}^{\mathrm{\infty }}{\rho }^h(e^{-ih\lambda }+e^{ih\lambda }))=\frac{{\sigma }^2}{2\pi (1-2\rho \cos \lambda +{\rho }^2)}=\frac{{\sigma }^2}{2\pi \mathrm{\mid }1-\rho e^{-i\lambda }{\mathrm{\mid }}^2}\mathrm{.}$ The denominator is at least $(1-\mathrm{\mid }\rho \mathrm{\mid }{)}^2>0$, so this is a continuous nonnegative density.

Applying [F13] to the two spectral representations gives $\mathbb{E}[I_n^{(s)}(\lambda )]=\frac{v}{2}\{{\mathrm{\Phi }}_n(\lambda -{\lambda }_s)+{\mathrm{\Phi }}_n(\lambda +{\lambda }_s)\},\mathbb{E}[I_n^{(\rho )}(\lambda )]={\int }_{\mathbb{T}}{\mathrm{\Phi }}_n(\lambda -\theta )f_{\rho }(\theta )d\theta \mathrm{.}$ The first expectation is a sum of two shifted kernels because the spectrum is atomic; the second averages a continuous density. These are exact finite-sample bias formulas and make no distributional or coverage claim.