Consistency and Limiting Distributions

For each positive integer $n$, let $({\mathrm{\Omega }}_n,{\mathcal{A}}_n,P_n)$ be a probability space and let $X_n:{\mathrm{\Omega }}_n\to {\mathbb{R}}^d$ be measurable into the Borel sigma-algebra generated by the Euclidean open sets, where $d\ge 1$ is fixed and ${\mathbb{R}}^d$ has the Euclidean norm $\mathrm{\parallel }\cdot {\mathrm{\parallel }}_2$ (Euclidean spheres and closed balls as subspaces of ${\mathbb{R}}^n$, Random elements and real random variables, Probability measures and probability spaces). The spaces and measures may depend on $n$; no common probability space is required.

Write $X_n=o_P(1)$ if for every $\epsilon >0$, $P_n(\mathrm{\parallel }{X}_n{\mathrm{\parallel }}_2>\epsilon )⟶0.$ Write $X_n=O_P(1)$, or say that $(X_n)$ is bounded in probability, if for every $\eta >0$ there is $M>0$ such that ${\mathrm{lim\hspace{0.17em}sup}}_{n\to \mathrm{\infty }}{P}_n(\mathrm{\parallel }{X}_n{\mathrm{\parallel }}_2>M)\le \eta \mathrm{.}$ For real-valued variables on one common probability space, $o_P(1)$ is convergence in probability to zero as defined in Convergence in probability.

The norms and tail probabilities are part of the definition. Independence is not implicit in either notation.

Let $d\ge 1$ be fixed, and equip ${\mathbb{R}}^d$ with the Borel sigma-algebra generated by its Euclidean open sets. Let $Z_n$ be measurable random elements of this Borel space, not necessarily defined on a common probability space, and let $\nu$ be a Borel probability measure on ${\mathbb{R}}^d$. The sequence $(Z_n)$ has asymptotic distribution $\nu$ if its laws converge weakly to $\nu$; equivalently, $Z_n\Rightarrow Z$ for any random vector $Z$ with law $\nu$ (Convergence in distribution of random elements, Weak convergence of borel probability measures).

For an estimator, any centering or scaling, such as $r_n(T_n-\psi (\theta ))$, is part of the sequence whose asymptotic distribution is being specified; it is not implicit in the term. Weak convergence alone does not assert convergence of moments.

Let all random vectors with index $n$ be defined on a common probability space for that $n$; the spaces may vary with $n$. In any fixed finite-dimensional Euclidean space, the following rules hold:

1. If $U_n=o_P(1)$ and $V_n=o_P(1)$, then $U_n+V_n=o_P(1)$. If $U_n=O_P(1)$ and $V_n=O_P(1)$, then $U_n+V_n=O_P(1)$.
2. For positive integers $p,q,r$, let $B:{\mathbb{R}}^p\times {\mathbb{R}}^q\to {\mathbb{R}}^r$ be a fixed map for which there is a constant $C\ge 0$ with $\mathrm{\parallel }B(u,v){\mathrm{\parallel }}_2\le C\mathrm{\parallel }u{\mathrm{\parallel }}_2\mathrm{\parallel }v{\mathrm{\parallel }}_2$ for all $u,v$. Assume in each application that $B(U_n,V_n)$ is a random vector on the same probability space as $U_n,V_n$, for every $n$. If $U_n=o_P(1)$ and $V_n=O_P(1)$, or conversely, then $B(U_n,V_n)=o_P(1)$. If both are $O_P(1)$, then $B(U_n,V_n)=O_P(1)$.
3. If $X_n\Rightarrow X$ in ${\mathbb{R}}^d$, then $X_n=O_P(1)$.

The last conclusion is finite-dimensional boundedness in probability. No independence or Axiom of Choice is assumed.

The composite random-vector hypothesis in part 2 is necessary. A norm bound on an arbitrary map does not establish measurability of its composites. For an explicit counterexample to the original claim without that premise, suppose a non-Borel set $A\subset (0,1)$ is supplied, and let $Z(\omega )=\omega$ on the supplied uniform probability space $(0,1)$ with its Borel sigma-algebra. Set $B(u,v)=\mathrm{\mid }u\mathrm{\mid }\mathrm{\mid }v\mathrm{\mid }{\mathbf{1}}_A(v)$ on ${\mathbb{R}}^2$, where ${\mathbf{1}}_A$ is zero outside $A$. This map satisfies $\mathrm{\mid }B(u,v)\mathrm{\mid }\le \mathrm{\mid }u\mathrm{\mid }\mathrm{\mid }v\mathrm{\mid }$ for every pair. Taking $U_n\equiv 1$ and $V_n=Z$ gives bounded-in-probability inputs, but $\{B(U_n,V_n)>0\}=A$ is not a measurable event, so the output is not a random variable and has no $O_P(1)$ classification under the definition. Taking instead $U_n\equiv 1\mathrm{/}n$ gives an $o_P(1)$ input and the same nonmeasurable positive-set inverse image, despite the pointwise bound by $1\mathrm{/}n$. This argument assumes the stated non-Borel set and uniform Borel probability space; it does not supply an existence proof for that set. Requiring only the actually used composites to be random vectors retains arbitrary norm-bounded maps, without imposing Borel measurability, bilinearity, or continuity on $B$ itself.

Let $d,k,m\ge 1$ be fixed. For each $n$, let $X_n\in {\mathbb{R}}^d$ and $Y_n\in {\mathbb{R}}^k$ be random vectors on one probability space; the probability space may vary with $n$. Let $X\in {\mathbb{R}}^d$ be a random vector on any probability space, and let $c\in {\mathbb{R}}^k$ be constant. If $X_n\Rightarrow X$ and $Y_n-c=o_P(1)$, then $(X_n,Y_n)\Rightarrow (X,c)\mathrm{.}$ Consequently, for every continuous $h:{\mathbb{R}}^{d+k}\to {\mathbb{R}}^m$, $h(X_n,Y_n)\Rightarrow h(X,c)$. No independence between $X_n$ and $Y_n$ is required.

Let $\mathsf{P}:\mathrm{\Theta }\to \mathcal{P}(\mathcal{X},\mathcal{A})$ be a parametric model on a statistical experiment, and let $\psi :\mathrm{\Theta }\to \mathcal{T}$ be an estimand whose target set $\mathcal{T}$ is equipped with a metric $d$ (Statistical experiments, Parametric statistical models, Estimands on a statistical model, Metric space: $d(x,y)=0$ iff $x=y$, symmetry, and the triangle inequality; pseudometric and ultrametric). For $n\ge 1$, the IID sample-size-$n$ experiment has sample space ${\mathcal{X}}^n$, product sigma-algebra ${\mathcal{A}}^{\otimes n}$, and sampling law $P_{\theta }^{\otimes n}$ at parameter $\theta$ (The product sigma-algebra and its finite iterates). The product law is defined recursively from $P_{\theta }$; each factor is a probability measure and hence sigma-finite, so the finite product theorem supplies its existence, rectangle formula, and uniqueness (For sigma-finite factors, the product measure exists, has the rectangle formula, is sigma-finite, and is unique).

A nonrandomized estimator sequence of $\psi$ is a sequence of measurable maps, in the sense of A measurable function between measurable spaces, $T_n:({\mathcal{X}}^n,{\mathcal{A}}^{\otimes n})\to (\mathcal{T},\mathcal{B}(\mathcal{T})),n\ge 1,$ where $\mathcal{B}(\mathcal{T})$ is the sigma-algebra generated by the open sets of the metric $d$, that is, the Borel sigma-algebra of its metric topology (The Borel sigma-algebra of a topological space). Its sampling distribution at $\theta$ and sample size $n$ is the pushforward of $P_{\theta }^{\otimes n}$ by $T_n$.

This definition fixes IID repeated sampling and data-measurable estimators. It imposes no measurability on the parameter map $\theta \mapsto P_{\theta }$ or on $\psi$.

Let $(T_n)$ be an estimator sequence of an estimand $\psi$ with metric target $(\mathcal{T},d)$, and fix $\theta \in \mathrm{\Theta }$. The sequence is consistent in probability at $\theta$ if for every $\epsilon >0$, $P_{\theta }^{\otimes n}(\{x\in {\mathcal{X}}^n:d(T_n(x),\psi (\theta ))>\epsilon \})⟶0.$

The error event is measurable: by the triangle inequality, $t\mapsto d(t,\psi (\theta ))$ is continuous, so its strict superlevel set at $\epsilon$ is open in the target metric topology and hence Borel; its preimage under $T_n$ is measurable.

The consistency is pointwise because $\theta$ is fixed before the limit. It does not assert uniform convergence over $\mathrm{\Theta }$ (Estimator sequences in IID experiments).

Let $\mathcal{X}=\mathbb{R}$ with its Borel sigma-algebra, and let $\mathsf{P}$ be a parametric statistical experiment on $\mathcal{X}$. Define the mean estimand by $\mu (\theta )=\int x{P}_{\theta }(dx)$ and suppose it is finite for every $\theta$. For a fixed $\theta$, suppose an IID sequence $(X_i{)}_{i\ge 1}$ with one-observation law $P_{\theta }$ is given and ${\mathbb{E}}_{\theta }\mathrm{\mid }{X}_1\mathrm{\mid }<\mathrm{\infty }$. The sample-mean estimator $T_n(x_1,\dots ,x_n)=n^{-1}{\sum }_{i=1}^n{x}_i$ is consistent in probability for $\mu$ at $\theta$ in the IID sample-size experiments.

The assertion is pointwise in $\theta$. The existence of a common probability space carrying the IID sequence is part of the hypothesis.

Let $\mathsf{P}:\mathrm{\Theta }\to \mathcal{P}(\mathcal{X},\mathcal{A})$ be a parametric statistical model, let $\psi :\mathrm{\Theta }\to (\mathcal{T},d)$ be a metric-valued estimand, and let $(T_n)$ be a nonrandomized estimator sequence for IID samples from this model. If $\psi$ is not identifiable, then there is a pair of observationally equivalent parameters with distinct target values, and no estimator sequence is consistent in probability at both parameters.