Moving and tapered block bootstrap for dependent data

Let $E$ be a separable complete metric space, let ${\mathcal{D}}_n$ be the sigma-algebra generated by the observed data at sample size $n$, and let $Z_n^{\ast }$ be a bootstrap random element whose remaining randomness is the resampling randomization. Write ${\mathcal{L}}^{\ast }(Z_n^{\ast })$ for its conditional Borel probability law given ${\mathcal{D}}_n$. Supply as part of the data a metric $d_w$ on Borel probability measures on $E$ such that $d_w({\mu }_n,\mu )\to 0$ exactly when ${\mu }_n\Rightarrow \mu$ in the sense of Weak convergence of borel probability measures. Assume the displayed metric distance is a measurable real random variable. For a fixed Borel probability measure $\mu$ on $E$, we say that $Z_n^{\ast }$ converges conditionally weakly in probability to $\mu$ when

$d_w\text{\hspace{0.17em}⁣}({\mathcal{L}}^{\ast }(Z_n^{\ast }),\mu )⟶0\text{in probability}\mathrm{.}$

The metric is supplied as a hypothesis; this definition makes no metric-existence assertion. It applies to separable complete metric spaces, including separable Banach spaces, their closed linear subspaces and finite products.

The conditioning fixes the observations and averages only over bootstrap randomization. A conditional limit in probability is not an assertion of almost-sure conditional convergence.

Let $E,F$ be normed spaces, let $D\subseteq E$, let $\theta \in D$, and let $E_0\subseteq E$ be a linear subspace. A map $\mathrm{\Psi }:D\to F$ is Hadamard differentiable at $\theta$, tangentially to $E_0$ if there is a continuous linear map ${\mathrm{\Psi }}_{\theta }^{\mathrm{\prime }}:E_0\to F$ such that, for every sequence $t_m>0$ with $t_m\to 0$, and every sequence $h_m\to h\in E_0$ for which $\theta +t_m{h}_m\in D$,

$\frac{\mathrm{\Psi }(\theta +t_m{h}_m)-\mathrm{\Psi }(\theta )}{t_m}⟶{\mathrm{\Psi }}_{\theta }^{\mathrm{\prime }}(h)\text{in }F\mathrm{.}$

The subspace is part of the hypothesis: directions outside $E_0$ are not covered by this definition.

Input: a finite sequence $x_1,\dots ,x_n$ in a measurable space, with $n\ge 1$, and an integer block length $b$ satisfying $1\le b\le n$. Set $N=n-b+1$ and $k=\lceil n\mathrm{/}b\rceil$.

Randomization: draw $J_1,\dots ,J_k$ independently and uniformly from $\{1,\dots ,N\}$.

Output: in order $s=1,\dots ,k$, append the block $x_{J_s},x_{J_s+1},\dots ,x_{J_s+b-1}$, then retain the first $n$ entries of the concatenation. Thus the final selected block is truncated to its first $r=n-b\lfloor n\mathrm{/}b\rfloor$ entries when $r>0$. Conditional expectation given the input sequence means expectation over the indices $J_s$ alone.

Input: observations $x_1,\dots ,x_n$ in a real Hilbert space $H$, and an integer block length $b$ with $1\le b\le n$.

Output: a pseudo-series $X_1^{\ast },\dots ,X_n^{\ast }$, generated by Overlapping block-index resampling: within each selected block retain the original order and concatenate blocks in their draw order, truncating only the last block when $b\nmid n$. The conditional bootstrap sample mean is ${\bar{X}}_n^{\ast }=n^{-1}{\sum }_{t=1}^n{X}_t^{\ast }$. Its conditional center is $E^{\ast }{\bar{X}}_n^{\ast }$, where $E^{\ast }$ averages over the independent uniform block starts while the observations remain fixed.

This algorithm specifies the resampling law only. It does not assert that the conditional law approximates the sampling law for any data-generating process or block-length regime.

Under Sampling units and the resampling-unit contract, take each observation $x_t$ to be a complete time-indexed unit record. Each draw selects an ordered group of $b$ consecutive records; the output retains every record of each complete block and only the initial $r=n-b\lfloor n\mathrm{/}b\rfloor$ records of the final block when $r>0$. Thus selected length-$b$ groups are not declared indivisible sampling units. Unit compatibility for a stated target requires that block boundaries and final truncation do not split any bundle declared indivisible by that target's sampling or assignment design. This is the sampling-unit interface, not a bootstrap-validity result.

Input: observations $Y_1,\dots ,Y_n$ in a real Hilbert space $H$, an integer $1\le b\le n$, and a window $w:\mathbb{R}\to [0,1]$ that vanishes off $[0,1]$, is positive on a neighbourhood of $1\mathrm{/}2$, satisfies $w(1-u)=w(u)$, and is nondecreasing on $[0,1\mathrm{/}2]$. Put $w_b(r)=w((r-\frac{1}{2})\mathrm{/}b)$ for $1\le r\le b$, $W_b=({\sum }_{r=1}^b{w}_b(r{)}^2{)}^{1\mathrm{/}2}$, and $a_b=\sqrt{b}\mathrm{/}{W}_b$. The procedure is admissible only when $W_b>0$.

Center and taper: set ${\bar{Y}}_n=n^{-1}{\sum }_{t=1}^n{Y}_t$ and $R_t=Y_t-{\bar{Y}}_n$. For each start $j\in \{1,\dots ,n-b+1\}$, form the block

$(a_b{w}_b(1)R_j,\dots ,a_b{w}_b(b)R_{j+b-1})\mathrm{.}$

Resample: use the independent uniform block starts from Overlapping block-index resampling, multiply each selected centered observation by its within-block taper and the factor $a_b$, concatenate in draw order, and retain the first $n$ entries. The conditional bootstrap mean is centered by its own conditional expectation when used to approximate the sampling law of $\sqrt{n}{\bar{Y}}_n$.

The factor $a_b$ compensates for the window's squared-weight sum in the block-sum variance calculation; this definition alone gives no bootstrap validity result.

For a dependent time series, the selected length-$b$ consecutive run is the resampling unit under Sampling units and the resampling-unit contract. The taper changes each unit record's within-block weight but the whole block is selected and resampled together; validity remains a separate theorem.

Fix a positive integer $n$, observations $x_1,\dots ,x_n\in H$, and an integer block length $b$ with $1\le b\le n$. Put $N=n-b+1$, $k=\lfloor n\mathrm{/}b\rfloor$, and $r=n-kb$. For $q\in \{b\}$, and also $q=r$ when $r>0$, define

$S_j^{(q)}={\sum }_{\mathrm{\ell }=1}^q{x}_{j+\mathrm{\ell }-1},{\bar{S}}_q=N^{-1}{\sum }_{j=1}^N{S}_j^{(q)},C_q=N^{-1}{\sum }_{j=1}^N(S_j^{(q)}-{\bar{S}}_q)\otimes (S_j^{(q)}-{\bar{S}}_q)\mathrm{.}$

Set $C_0=0$. If ${\bar{X}}_n^{\ast }$ is the mean produced by Moving block bootstrap, then its conditional covariance operator is

$n{\mathrm{Cov}}^{\ast }({\bar{X}}_n^{\ast })=\frac{k{C}_b+C_r}{n},$

where the $C_r$ term is zero when $r=0$. Here ${\mathrm{Cov}}^{\ast }(Z)=E^{\ast }[(Z-E^{\ast }Z)\otimes (Z-E^{\ast }Z)]$.

When $r=0$, this reduces to $n{\mathrm{Cov}}^{\ast }({\bar{X}}_n^{\ast })=C_b\mathrm{/}b$. The identity is finite-sample and makes no stationarity or moment assumption on an underlying process.

Fix a positive integer $n$, observations $Y_1,\dots ,Y_n\in H$, an integer block length $b$ with $1\le b\le n$ and $W_b>0$, and the taper and scale $w_b,a_b$ from Tapered block bootstrap. Set $R_t=Y_t-{\bar{Y}}_n$, $N=n-b+1$, $k=\lfloor n\mathrm{/}b\rfloor$, and $r=n-kb$. For $q=b$, and also $q=r$ when $r>0$, define the tapered partial block sum

$T_j^{(q)}=a_b{\sum }_{\mathrm{\ell }=1}^q{w}_b(\mathrm{\ell })R_{j+\mathrm{\ell }-1},{\bar{T}}_q=N^{-1}{\sum }_{j=1}^N{T}_j^{(q)},V_q=N^{-1}{\sum }_{j=1}^N(T_j^{(q)}-{\bar{T}}_q)\otimes (T_j^{(q)}-{\bar{T}}_q)\mathrm{.}$

Set $V_0=0$. If ${\bar{X}}_n^{\ast }$ is the tapered bootstrap mean, then

$n{\mathrm{Cov}}^{\ast }({\bar{X}}_n^{\ast })=\frac{k{V}_b+V_r}{n},$

with the $V_r$ term zero when $r=0$.

Let $w$ satisfy the taper conditions in Tapered block bootstrap. As the positive integer $b\to \mathrm{\infty }$, $w^2$ is Riemann integrable on $[0,1]$, and

$\frac{1}{b}{\sum }_{j=1}^{b}w\text{\hspace{0.17em}⁣}{\left(\frac{j-\frac{1}{2}}{b}\right)}^2⟶I_w:={\int }_0^{1}w(u{)}^{2}du>0.$

Consequently, $W_b>0$ for all sufficiently large $b$ and $a_b=\sqrt{b}\mathrm{/}{W}_b\to I_w^{-1\mathrm{/}2}$.

Input: a fixed integer $K\ge 2$ of observed series $Y_{i,1},\dots ,Y_{i,n_i}$ in a real Hilbert space $H$, positive integer sample sizes $n_i$, total size $M={\sum }_{i=1}^K{n}_i$, integer block lengths $1\le b_i\le n_i$, and a mode, moving or tapered. For tapered mode also supply an admissible window with positive block norm at each $b_i$. Put

${\bar{Y}}_i=\frac{1}{n_i}{\sum }_{t=1}^{n_i}{Y}_{i,t},{\bar{Y}}_P=\frac{1}{M}{\sum }_{i=1}^K{n}_i{\bar{Y}}_i,R_{i,t}=Y_{i,t}-{\bar{Y}}_i\mathrm{.}$

For each series $i=1,\dots ,K$, let $N_i=n_i-b_i+1$ and $k_i=\lceil n_i\mathrm{/}{b}_i\rceil$. For every candidate start $j=1,\dots ,N_i$ and position $r=1,\dots ,b_i$, define the following arrays. In moving mode set $A_{i,j,r}=R_{i,j+r-1}$. In tapered mode set $A_{i,j,r}=a_{b_i}{w}_{b_i}(r)R_{i,j+r-1}$, with $a_{b_i}$ and $w_{b_i}$ as in Tapered block bootstrap. For each $i=1,\dots ,K$ and $s=1,\dots ,k_i$, draw independent uniform starts $J_{i,s}\in \{1,\dots ,N_i\}$, using the index law of Overlapping block-index resampling. For each within-block position define

$c_{i,r}=\frac{1}{N_i}{\sum }_{j=1}^{N_i}{A}_{i,j,r}\mathrm{.}$

Output: for each $i$, a pseudo-series $Y_{i,1}^{\ast },\dots ,Y_{i,n_i}^{\ast }$. In the order $s=1,\dots ,k_i$, retain the positions $r=1,\dots ,q_{i,s}$, where $q_{i,s}=\min \{b_i,n_i-(s-1)b_i\}$, and set

$Y_{i,(s-1)b_i+r}^{\ast }={\bar{Y}}_P+A_{i,J_{i,s},r}-c_{i,r}\mathrm{.}$

Thus a divisible sample uses $k_i$ complete blocks; otherwise only the first $n_i-b_i\lfloor n_i\mathrm{/}{b}_i\rfloor$ positions of the final block are retained. This centers each resampled residual position by its own conditional average, then gives every pseudo-series the same pooled location ${\bar{Y}}_P$. All block starts are conditionally independent across blocks and across series.

The algorithm is a finite-sample null imposition. Its validity for a statistic requires additional process and block-length hypotheses.

For fixed data and either mode of Null-constrained block bootstrap for independent series, every pseudo-observation in every series has conditional expectation ${\bar{Y}}_P$. Consequently, for each $i$,

$E^{\ast }{\bar{Y}}_i^{\ast }={\bar{Y}}_P,$

so the conditional mean functions of the $K$ pseudo-series are equal.

Let $E,F$ be separable Banach spaces, $E_0\subseteq E$ a closed linear subspace, $\theta \in E$, and $D\subseteq E$ a neighbourhood of $\theta$. Let $t_n>0$ decrease to zero, let $Z_n\in E_0$ be data-measurable and tight, and let $Z_n^{\ast }\in E_0$ be bootstrap random elements such that $\theta +t_n{Z}_n$ and $\theta +t_n(Z_n+Z_n^{\ast })$ lie in $D$ with probability tending to one. Supply weak-convergence metrics on probability laws on $E_0$ and $F$ as in Conditional weak convergence in probability, with measurable distances for the conditional laws below. Assume:

1. ${\mathcal{L}}^{\ast }(Z_n^{\ast })$ converges conditionally weakly in probability to the fixed law of $G\in E_0$, as in Conditional weak convergence in probability;
2. $Z_n^{\ast }$ is conditionally tight in probability: for each $ϵ,\eta >0$, there is a deterministic compact $K\subset E_0$ such that ${\mathrm{lim\hspace{0.17em}sup}}_n\mathrm{Pr}\{{\mathrm{Pr}}^{\ast }(Z_n^{\ast }\mathrm{\notin }K)>ϵ\}<\eta$;
3. $D$ is Borel and $\mathrm{\Psi }:D\to F$ is Borel measurable and Hadamard differentiable at $\theta$, tangentially to $E_0$, with continuous linear derivative ${\mathrm{\Psi }}_{\theta }^{\mathrm{\prime }}$ as in Hadamard differentiability tangentially to a subspace.

Then

${\mathcal{L}}^{\ast }\text{\hspace{0.17em}⁣}(t_n^{-1}\{\mathrm{\Psi }(\theta +t_n(Z_n+Z_n^{\ast }))-\mathrm{\Psi }(\theta +t_n{Z}_n)\})$

converges conditionally weakly in probability to the law of ${\mathrm{\Psi }}_{\theta }^{\mathrm{\prime }}(G)$. Define the displayed bootstrap difference to be $0\in F$ whenever either argument lies outside $D$; the assumed vanishing probability of this event makes this convention asymptotically immaterial.

This theorem is conditional on the bootstrap limit and tightness hypotheses. It does not prove those hypotheses for a particular resampling algorithm.

Assume the Axiom of Choice. Let $(X_t{)}_{t\in \mathbb{Z}}$ be a centered $L^2$-$m$-approximable causal Bernoulli shift in $H=L^2([0,1];\mathbb{R})$ under L2-m-approximable causal Bernoulli shifts, and suppose additionally that $m{\delta }_m\to 0$. Put $\sigma =\mathrm{\parallel }{X}_0{\mathrm{\parallel }}_{L^2(H)}$, let $\mathrm{\Gamma }$ be its long-run covariance operator, and put

$K={\sigma }^2+2\sigma {\sum }_{h\ge 1}{\delta }_h\mathrm{.}$

For every integer $b\ge 1$, let $c_{b,1},\dots ,c_{b,b}$ be deterministic real weights, uniformly bounded in absolute value by $A<\mathrm{\infty }$. Define

$Z_{j,b}=b^{-1\mathrm{/}2}{\sum }_{\mathrm{\ell }=1}^b{c}_{b,\mathrm{\ell }}{X}_{j+\mathrm{\ell }-1}\mathrm{.}$

Then $\{\mathrm{\parallel }{Z}_{j,b}{\mathrm{\parallel }}_H^2:j\in \mathbb{Z},b\ge 1\}$ is uniformly integrable, and for $0\le q\le b$,

$E{\parallel {\sum }_{\mathrm{\ell }=1}^q{c}_{b,\mathrm{\ell }}{X}_{j+\mathrm{\ell }-1}\parallel }_H^2\le A^{2}Kq\mathrm{.}$

If $P$ is any finite-rank orthogonal projection, $R=I-P$ and ${\alpha }_P=\mathrm{\parallel }R{X}_0{\mathrm{\parallel }}_{L^2(H)}$, then also

$E{\parallel {\sum }_{\mathrm{\ell }=1}^q{c}_{b,\mathrm{\ell }}R{X}_{j+\mathrm{\ell }-1}\parallel }_H^2\le A^{2}q({\alpha }_P^2+2{\alpha }_P{\sum }_{h\ge 1}{\delta }_h)\mathrm{.}$

For $n\ge b$, set $N=n-b+1$ and ${\bar{Z}}_{n,b}=N^{-1}{\sum }_{j=1}^N{Z}_{j,b}$. Then

$E\mathrm{\parallel }{\bar{Z}}_{n,b}{\mathrm{\parallel }}_H^2\le A^{2}Kb\mathrm{/}N\mathrm{.}$

Suppose further that $b=b_n\to \mathrm{\infty }$, $b_n\mathrm{/}n\to 0$, and the weights satisfy

$b^{-1}{\sum }_{\mathrm{\ell }=1}^b{c}_{b,\mathrm{\ell }}^2=1,q_b(h):=b^{-1}{\sum }_{\mathrm{\ell }=1}^{b-h}{c}_{b,\mathrm{\ell }}{c}_{b,\mathrm{\ell }+h}⟶1\text{for every fixed }h\ge 0.$

The centered overlapping-block covariance then satisfies the stronger $L^1$ trace-norm limit

$E{\parallel N^{-1}{\sum }_{j=1}^N(Z_{j,b}-{\bar{Z}}_{n,b})\otimes (Z_{j,b}-{\bar{Z}}_{n,b})-\mathrm{\Gamma }\parallel }_1⟶0.$

In particular it converges in probability in trace norm. Both the MBB weights $c_{b,\mathrm{\ell }}=1$ and the actual TBB weights $c_{b,\mathrm{\ell }}=a_{b}w((\mathrm{\ell }-\frac{1}{2})\mathrm{/}b)$ satisfy these weight hypotheses for all sufficiently large $b$. For any complete finite or countable orthonormal basis and its initial-coordinate projections $P_d$, the projection-tail coefficient ${\alpha }_{P_d}^2+2{\alpha }_{P_d}{\sum }_{h\ge 1}{\delta }_h$ tends to zero as $d\to \mathrm{\infty }$.