Weighted overlapping-block moment and covariance limits

For $h\ge 1$, $X_{h,h}$ is independent of $X_0$: its retained original innovation indices are $1,\dots ,h$, and its replacement array is independent of all inputs to $X_0$. Its mean is zero. The tensor $X_0\otimes X_{h,h}$ is Bochner integrable by [F1], [F3]–[F4], and the rank-one norm identity. For arbitrary $f,g\in H$, [F6] gives $⟨{\mathbb{E}}_B[X_0\otimes X_{h,h}]f,g⟩=E[⟨f,X_{h,h}⟩⟨X_0,g⟩]=0$, hence ${\mathbb{E}}_B[X_0\otimes X_{h,h}]=0$. Thus $\mathrm{\parallel }E[X_0\otimes X_h]{\mathrm{\parallel }}_1\le \sigma {\delta }_h$ by [F3]–[F4]. For every finite real coefficient sequence $(u_t)$, expansion and $2\mathrm{\mid }{u}_t{u}_{t+h}\mathrm{\mid }\le u_t^2+u_{t+h}^2$ therefore give $E\mathrm{\parallel }{\sum }_t{u}_t{X}_t{\mathrm{\parallel }}^2\le K{\sum }_t{u}_t^2$. This proves the prefix bound. The same argument after applying $R=I-P$ gives $\mathrm{\mid }E⟨R{X}_0,R{X}_h⟩\mathrm{\mid }\le {\alpha }_P{\delta }_h$, since $\mathrm{\parallel }R\mathrm{\parallel }\le 1$ and projection preserves independence and centering. It proves the asserted projected prefix bound. Along complete basis projections, ${\alpha }_{P_d}\to 0$ by norm convergence and dominated convergence, with dominating squared norm $\mathrm{\parallel }{X}_0{\mathrm{\parallel }}^2$.

Put $D_{t,m}=X_t-X_{t,m}$. For $h\ge 1$, $X_{h,h}$ is independent of $D_{0,m}$, so [F6] and Cauchy–Schwarz bound $\mathrm{\mid }E⟨D_{0,m},X_h⟩\mathrm{\mid }\le {\delta }_m{\delta }_h$. If $h\ge m$, $X_{h,m}$ is independent of $D_{0,m}$, while for $1\le h<m$, Cauchy–Schwarz bounds $\mathrm{\mid }E⟨D_{0,m},X_{h,m}⟩\mathrm{\mid }$ by ${\delta }_m\sigma$. Consequently the individual lag bounds are $\mathrm{\mid }E⟨D_{0,m},D_{h,m}⟩\mathrm{\mid }\le {\delta }_m({\delta }_h+\sigma )$ for $h<m$ and $\le {\delta }_m{\delta }_h$ for $h\ge m$. Negative-lag scalar covariances equal their positive counterparts by stationarity. Expanding any bounded-weight block and bounding each $\mathrm{\mid }{c}_{b,\mathrm{\ell }}{c}_{b,\mathrm{\ell }+h}\mathrm{\mid }$ by $A^2$ gives ${\sup }_{j,b}E\mathrm{\parallel }{b}^{-1\mathrm{/}2}{\sum }_{\mathrm{\ell }=1}^b{c}_{b,\mathrm{\ell }}{D}_{j+\mathrm{\ell }-1,m}{\mathrm{\parallel }}^2\le A^2{\epsilon }_m^2$, where ${\epsilon }_m^2={\delta }_m^2+2{\delta }_m{\sum }_{h\ge 1}{\delta }_h+2\sigma (m-1){\delta }_m\to 0$. The convergence uses precisely the assumed $m{\delta }_m\to 0$ and summability.

Minkowski's inequality in $L^2(H)$ and the finite-coefficient bound in step 1.1 give $\mathrm{\parallel }{N}^{-1}{\sum }_{j=1}^N{X}_{j+\mathrm{\ell }-1}{\mathrm{\parallel }}_{L^2(H)}\le \sqrt{K\mathrm{/}N}$ for each $\mathrm{\ell }$. Hence $\mathrm{\parallel }{\bar{Z}}_{n,b}{\mathrm{\parallel }}_{L^2(H)}\le b^{-1\mathrm{/}2}{\sum }_{\mathrm{\ell }=1}^b\mathrm{\mid }{c}_{b,\mathrm{\ell }}\mathrm{\mid }\sqrt{K\mathrm{/}N}\le A\sqrt{Kb\mathrm{/}N}$, as claimed. Minkowski here follows by expanding a finite squared norm and applying Cauchy–Schwarz to each cross term.

Fix $m$. Write $Y_t=X_{t,m}$ and define $V_t=Y_t{\mathbf{1}}_{\{\mathrm{\parallel }{Y}_t\mathrm{\parallel }\le M\}}-E[Y_0{\mathbf{1}}_{\{\mathrm{\parallel }{Y}_0\mathrm{\parallel }\le M\}}]$. This is centered, stationary and $m$-dependent, with $\mathrm{\parallel }{V}_t\mathrm{\parallel }\le 2M$. Put ${\eta }_{m,M}=\mathrm{\parallel }{Y}_0-V_0{\mathrm{\parallel }}_{L^2(H)}$. Since $E{Y}_0=0$, ${\eta }_{m,M}\le 2\mathrm{\parallel }{Y}_0{\mathbf{1}}_{\{\mathrm{\parallel }{Y}_0\mathrm{\parallel }>M\}}{\mathrm{\parallel }}_{L^2(H)}\to 0$ by dominated convergence. The centered residual $Y-V$ is $m$-dependent, so its covariances vanish at distances greater than $m$ by [F6], and all remaining scalar covariances have absolute value at most ${\eta }_{m,M}^2$. Hence its normalized weighted block has second moment at most $A^2(2m+1){\eta }_{m,M}^2$. With $Z_{j,b}^V=b^{-1\mathrm{/}2}{\sum }_{\mathrm{\ell }=1}^b{c}_{b,\mathrm{\ell }}{V}_{j+\mathrm{\ell }-1}$, step 1.2 and Minkowski give ${\sup }_{j,b}\mathrm{\parallel }{Z}_{j,b}-Z_{j,b}^V{\mathrm{\parallel }}_{L^2(H)}\le d_{m,M}:=A{\epsilon }_m+A\sqrt{2m+1}{\eta }_{m,M}$. Choose first $m$ and then $M$ to make this arbitrarily small.

For independent centered Hilbert elements $U_1,\dots ,U_q$ bounded by $L$, write $\mathrm{\parallel }{\sum }_i{U}_i{\mathrm{\parallel }}^2={\sum }_i\mathrm{\parallel }{U}_i{\mathrm{\parallel }}^2+2{\sum }_{i<j}⟨U_i,U_j⟩$. On squaring, every term containing an index exactly once has expectation zero, by integrating that independent centered element first using [F6]. In particular all mixed terms between the two sums vanish; among products of two inner products only identical pairs survive. Cauchy–Schwarz then gives $E\mathrm{\parallel }{\sum }_i{U}_i{\mathrm{\parallel }}^4\le {\sum }_{i}E\mathrm{\parallel }{U}_i{\mathrm{\parallel }}^4+6{\sum }_{i<j}E\mathrm{\parallel }{U}_i{\mathrm{\parallel }}^{2}E\mathrm{\parallel }{U}_j{\mathrm{\parallel }}^2\le 3q^2{L}^4$; the empty sum is zero. Partition the $b$ summands defining $\sqrt{b}{Z}_{j,b}^V$ into $p=m+1$ residue classes. Inside each class their input innovation sets are disjoint, hence they are jointly independent by [F2]. Applying the proved bound to the class sizes $q_s$ and using $\mathrm{\parallel }{\sum }_{s=1}^p{u}_s{\mathrm{\parallel }}^4\le p^3{\sum }_s\mathrm{\parallel }{u}_s{\mathrm{\parallel }}^4$ and ${\sum }_s{q}_s^2\le b^2$ gives ${\sup }_{j,b}E\mathrm{\parallel }{Z}_{j,b}^V{\mathrm{\parallel }}^4\le 3p^3{A}^4(2M{)}^4=:C_{m,M}$. Thus fourth moments are used only for bounded truncations.

The rank-one identity and Cauchy–Schwarz imply both ${\sup }_{j,b}E\mathrm{\mid }\mathrm{\parallel }{Z}_{j,b}{\mathrm{\parallel }}^2-\mathrm{\parallel }{Z}_{j,b}^V{\mathrm{\parallel }}^2\mathrm{\mid }\le d_{m,M}(2A\sqrt{K}+d_{m,M})$ and ${\sup }_{j,b}E\mathrm{\parallel }{Z}_{j,b}\otimes Z_{j,b}-Z_{j,b}^V\otimes Z_{j,b}^V{\mathrm{\parallel }}_1\le d_{m,M}(2A\sqrt{K}+d_{m,M})$. Here use $\mathrm{\parallel }{Z}_{j,b}{\mathrm{\parallel }}_{L^2}\le A\sqrt{K}$ and $\mathrm{\parallel }u\otimes u-v\otimes v{\mathrm{\parallel }}_1\le \mathrm{\parallel }u-v\mathrm{\parallel }(\mathrm{\parallel }u\mathrm{\parallel }+\mathrm{\parallel }v\mathrm{\parallel })$. The squared truncated blocks are uniformly integrable by step 3.1 and Markov's inequality. For nonnegative $F,G$, splitting at $G>L\mathrm{/}2$ proves $E[F{\mathbf{1}}_{\{F>L\}}]\le 2E\mathrm{\mid }F-G\mathrm{\mid }+E[G{\mathbf{1}}_{\{G>L\mathrm{/}2\}}]$. Apply this with the two squared block norms, then let $L\to \mathrm{\infty }$ and $d_{m,M}\to 0$. This proves the asserted uniform integrability for $Z$.

Fix $m,M$ and set $Q_n^V=N^{-1}{\sum }_{j=1}^N{Z}_{j,b}^V\otimes Z_{j,b}^V$ and $B_n=E{Q}_n^V$. For fixed unit vectors $e,f$, the real variables $⟨Z_{j,b}^V,e⟩⟨Z_{j,b}^V,f⟩$ have second moments at most $C_{m,M}$. When two starts differ by more than $b+m$, the blocks use disjoint original innovations and disjoint replacement arrays, so these variables are independent by [F2]. Expanding the variance of their empirical mean and factoring the independent cross terms by [F6] therefore gives $E\mathrm{\mid }⟨(Q_n^V-B_n)f,e⟩{\mathrm{\mid }}^2\le C_{m,M}(2(b+m)+1)\mathrm{/}N\to 0$, since $b\mathrm{/}n\to 0$. For any fixed finite-rank orthogonal projection $P$ of rank $d$, expansion in an orthonormal basis of its range and the rank-one norm bound show $\mathrm{\parallel }P(Q_n^V-B_n)P{\mathrm{\parallel }}_1\le {\sum }_{u,v\le d}\mathrm{\mid }⟨(Q_n^V-B_n)e_v,e_u⟩\mathrm{\mid }$. Thus its expectation tends to zero.

Choose complete basis projections $P_d$ and put $R_d=I-P_d$. For any square-integrable $H$-valued $U$, the operator $T=E[U\otimes U]$ satisfies $\mathrm{\parallel }T-PTP{\mathrm{\parallel }}_1\le E\mathrm{\parallel }RU{\mathrm{\parallel }}^2+2\sqrt{E\mathrm{\parallel }PU{\mathrm{\parallel }}^{2}E\mathrm{\parallel }RU{\mathrm{\parallel }}^2}$: expand its four $P,R$ corners, use the rank-one norm inequality on each off-diagonal corner, and use positivity for the $R$ corner. The same inequality holds for an empirical average, using its finite empirical expectation. For the $V$ blocks, finite dependence and covariance expansion give $E\mathrm{\parallel }{R}_d{Z}_{j,b}^V{\mathrm{\parallel }}^2\le A^2(2m+1)E\mathrm{\parallel }{R}_d{V}_0{\mathrm{\parallel }}^2=:t_d\to 0$ by dominated convergence. Also ${\sup }_{j,b}E\mathrm{\parallel }{Z}_{j,b}^V{\mathrm{\parallel }}^2\le (A\sqrt{K}+d_{m,M}{)}^2=:C$. Applying the corner inequality to $Q_n^V$ and $B_n$, and then Cauchy–Schwarz to expectations, yields $E\mathrm{\parallel }{Q}_n^V-B_n{\mathrm{\parallel }}_1\le E\mathrm{\parallel }{P}_d(Q_n^V-B_n)P_d{\mathrm{\parallel }}_1+2t_d+4\sqrt{C{t}_d}$. Send $n\to \mathrm{\infty }$ by step 4.2, then $d\to \mathrm{\infty }$. This proves trace-norm concentration locally, without inferring trace-norm convergence from coordinate convergence alone.

Define $Q_n=N^{-1}{\sum }_{j=1}^N{Z}_{j,b}\otimes Z_{j,b}$. By step 4.1, $E\mathrm{\parallel }{Q}_n-Q_n^V{\mathrm{\parallel }}_1$ and $\mathrm{\parallel }E{Q}_n-B_n{\mathrm{\parallel }}_1$ are each at most $d_{m,M}(2A\sqrt{K}+d_{m,M})$. The triangle inequality and step 5.1 therefore give ${\mathrm{lim\hspace{0.17em}sup}}_{n}E\mathrm{\parallel }{Q}_n-E{Q}_n{\mathrm{\parallel }}_1\le 2d_{m,M}(2A\sqrt{K}+d_{m,M})$. First send $M\to \mathrm{\infty }$ for each fixed $m$, then send $m\to \mathrm{\infty }$. It follows that $E\mathrm{\parallel }{Q}_n-E{Q}_n{\mathrm{\parallel }}_1\to 0$. This concentration conclusion required only the uniform bound on weights.

Now impose the squared normalization and fixed-lag autocorrelation hypotheses. Extend each weight vector by zero. Cauchy–Schwarz gives $\mathrm{\mid }{q}_b(h)\mathrm{\mid }\le 1$. With $C_h=E[X_0\otimes X_h]$, stationarity gives $E{Q}_n=C_0+{\sum }_{h=1}^{b-1}{q}_b(h)(C_h+C_h^{\ast })$. The series is absolutely convergent in trace norm by step 1.1. On each finite set of lags $q_b(h)\to 1$, and the remaining trace-norm tail is uniformly bounded by its summable covariance tail. Thus $E{Q}_n\to \mathrm{\Gamma }$ in trace norm. Step 6.1 gives $E\mathrm{\parallel }{Q}_n-\mathrm{\Gamma }{\mathrm{\parallel }}_1\to 0$. The empirically centered covariance equals $Q_n-{\bar{Z}}_{n,b}\otimes {\bar{Z}}_{n,b}$, whose difference from $Q_n$ has expected trace norm $E\mathrm{\parallel }{\bar{Z}}_{n,b}{\mathrm{\parallel }}^2\le A^{2}Kb\mathrm{/}N\to 0$ by step 2.1. This proves the asserted $L^1$ limit and its probability consequence by Markov's inequality.

MBB has $q_b(h)=(1-h\mathrm{/}b{)}_{+}$ and squared normalization exactly. For TBB the squared normalization is also exact and the coefficients are bounded eventually by [F5]. Extend its nonnegative unimodal coefficients by zero to all integers; the total variation is at most $2A$. Telescoping gives ${\sum }_{\mathrm{\ell }\in \mathbb{Z}}\mathrm{\mid }{c}_{b,\mathrm{\ell }+h}-c_{b,\mathrm{\ell }}\mathrm{\mid }\le 2Ah$, so $\mathrm{\mid }{q}_b(h)-q_b(0)\mathrm{\mid }\le 2A^{2}h\mathrm{/}b\to 0$. This uses exactly the midpoint coefficients of the algorithm and requires no continuity of the taper. All proved limits and uniform-integrability conclusions apply after any finite initial set of block sizes is discarded. The moment and projection conclusions, covariance limit, and claimed specializations are now established.