Consistency and Limiting Distributions — Examples

Set ${\mathrm{\Omega }}_n=\{0,1\}$, ${\mathcal{A}}_n=\mathcal{P}({\mathrm{\Omega }}_n)$, $P_n(\{1\})=1\mathrm{/}n$, and $P_n(\{0\})=1-1\mathrm{/}n$. These masses are nonnegative and sum to one. Since the space has two points, every disjoint sequence of subsets has at most two nonempty members; the mass formula is countably additive and gives zero to the empty set by [F3]. Thus [F1], [F2], and [F4] make this a probability space for each $n$. The map $X_n(0)=0$, $X_n(1)=n$ is measurable by [F5], hence a real random variable by [F6].

For any fixed $\epsilon >0$, once $n>\epsilon$ the event $\{\mathrm{\mid }{X}_n\mathrm{\mid }>\epsilon \}$ is exactly $\{1\}$, so $P_n(\mathrm{\mid }{X}_n\mathrm{\mid }>\epsilon )=1\mathrm{/}n\to 0$. Therefore $X_n=o_P(1)$ by [F7].

Fix bounded continuous $f:\mathbb{R}\to \mathbb{R}$, set $M=\mathrm{\parallel }f{\mathrm{\parallel }}_{\mathrm{\infty }}$, and put $g=f+M\ge 0$. By [F13], the law $P_{X_n}$ assigns mass $1-1\mathrm{/}n$ to $0$ and $1\mathrm{/}n$ to $n$. Its pushforward formula makes it countably additive because preimages of disjoint Borel sets are disjoint, and $P_{X_n}(\mathrm{\varnothing })=P_n(\mathrm{\varnothing })=0$; its total mass is $P_n({\mathrm{\Omega }}_n)=1$, so it is a probability measure by [F3]–[F4]. The point mass ${\delta }_0$ is also a Borel probability measure: it assigns zero to $\mathrm{\varnothing }$, and in a disjoint family of Borel sets at most one set contains $0$, while ${\delta }_0(\mathbb{R})=1$. Since $\mathrm{\mid }f\mathrm{\mid }\le M$ and $0\le g\le 2M$, every nonnegative simple minorant of $\mathrm{\mid }f\mathrm{\mid }$ or $g$ has simple integral at most the corresponding constant times $\mu (\mathbb{R})$, by finite additivity and [F12]. Taking the suprema in the nonnegative integral defining expectation [F11] gives $\int \mathrm{\mid }f\mathrm{\mid }d\mu \le M$ and $\int gd\mu \le 2M$ for either probability law $\mu \in \{P_{X_n},{\delta }_0\}$, so both functions are integrable. Thus [F15] gives $\int fd\mu =\int (g-M)d\mu =\int gd\mu -M$. The nonnegative simple function $g(X_n)$ has expectation $\int gd{P}_{X_n}=(1-1\mathrm{/}n)g(0)+(1\mathrm{/}n)g(n)$ by [F11]–[F12] and [F14]; the same formula gives $\int gd{\delta }_0=g(0)$. Thus $\mathrm{\mid }\int fd{P}_{X_n}-\int fd{\delta }_0\mathrm{\mid }=\mathrm{\mid }\mathbb{E}g(X_n)-g(0)\mathrm{\mid }=\mathrm{\mid }g(n)-g(0)\mathrm{\mid }\mathrm{/}n\le 2M\mathrm{/}n\to 0$. By [F8], the laws converge weakly to ${\delta }_0$; by [F9]–[F10], the asymptotic distribution is ${\delta }_0$.

The nonnegative simple-integral formula [F11]–[F12] gives $\mathbb{E}{X}_n=(1\mathrm{/}n)n+(1-1\mathrm{/}n)0=1$ for every $n$, while the limit variable $0$ has expectation $0$. For the finite check $n=2$, $P_2(X_2=2)=1\mathrm{/}2$ and $\mathbb{E}{X}_2=1$, confirming the stated mass and expectation formulas.

The power set on $\mathrm{\Omega }$ is a sigma-algebra by [F1], hence gives a measurable space by [F2]. Assigning mass $1\mathrm{/}2$ to each singleton defines a measure: any pairwise disjoint sequence contains at most two nonempty sets, and the mass of their union is the sum of their masses. Its total mass is one, so it is a probability measure by [F3]–[F4]. The identity map $Z$ is measurable because every preimage of a Borel set is a subset of $\mathrm{\Omega }$; thus it is a real random variable by [F5]–[F6].

For every $n$, $\mathrm{\mid }{Z}_n\mathrm{\mid }=1$ everywhere, so choosing $M=1$ makes $P(\mathrm{\mid }{Z}_n\mathrm{\mid }>M)=0$ and proves $Z_n=O_P(1)$ by [F7]. For each $\epsilon >0$, $n^{-1\mathrm{/}2}\le \epsilon$ for all sufficiently large $n$, so $P(\mathrm{\mid }{a}_n\mathrm{\mid }>\epsilon )=0$ eventually; hence $a_n=o_P(1)$ by [F7].

The multiplication map $B(u,v)=uv$ satisfies $\mathrm{\mid }B(u,v)\mathrm{\mid }=\mathrm{\mid }u\mathrm{\mid }\mathrm{\mid }v\mathrm{\mid }$ for all real $u,v$, so it has the required bilinear norm bound with constant one. Each $R_n=a_n{Z}_n$ is measurable because the power-set sigma-algebra on $\mathrm{\Omega }$ contains every preimage. Thus [F8] applies to $a_n=o_P(1)$ and $Z_n=O_P(1)$ and gives $R_n=o_P(1)$. The identity $\sqrt{n}{R}_n=Z_n$ then gives $\sqrt{n}{R}_n=O_P(1)$ from step 1.2. For the finite check $n=4$, $R_4$ takes the values $\pm 1\mathrm{/}2$ and $\sqrt{4}{R}_4$ takes the values $\pm 1$.

The power set of $\{0,1\}$ is a sigma-algebra by [F1], so it defines a measurable space by [F2]. Because $0\le p\le 1$, the two singleton masses are nonnegative and sum to one. For any pairwise disjoint sequence of subsets, each of $0$ and $1$ occurs in at most one set, so the mass formula is countably additive and gives zero to the empty set by [F3]. Thus $P_p$ is a probability measure by [F4], and the family is a statistical experiment with model map $p\mapsto P_p$ and target $\psi (p)=p$ by [F5, F6, F7].

For $X\sim P_p$, $X=1_{\{1\}}$ and $X^2=X$, so [F12] gives ${\mathbb{E}}_{p}X=p$ and ${\mathbb{E}}_p{X}^2=p$. By [F13], ${\mathrm{Var}}_p(X)=p-p^2=p(1-p)$, which is finite for all $p\in [0,1]$.

By [F17], each coordinate projection on ${\mathcal{X}}^n$ is measurable; [F18] makes their finite sum and scalar multiple $T_n$ measurable, as required by [F8]. The displayed map is the sample mean of [F9]. The given common sequence satisfies the IID hypothesis of [F14] by [F10], and [F11] identifies the law of its first $n$ observations with $P_p^{\otimes n}$. Applying [F14] with ${\sigma }^2=p(1-p)$ from step 1.2 and transferring through this joint-law identity gives $P_p^{\otimes n}(\mathrm{\mid }{T}_n-p\mathrm{\mid }\ge \epsilon )\le p(1-p)\mathrm{/}(n{\epsilon }^2)$. Since $p(1-p)=1\mathrm{/}4-(p-1\mathrm{/}2{)}^2\le 1\mathrm{/}4$, this is at most $1\mathrm{/}(4n{\epsilon }^2)$.

For fixed $\epsilon >0$, the bound in step 2.1 tends to zero, so [F16] gives consistency; this is also the special case of [F15]. By [F8] and [F11], at $p=0$ and $p=1$ the product law assigns probability one to the all-zero and all-one samples respectively, so $T_n=p$ almost surely at each endpoint. For the finite check $n=2,p=1\mathrm{/}2$, the same rectangle formula gives mass $1\mathrm{/}4$ to each ordered pair, hence $T_2$ has masses $1\mathrm{/}4,1\mathrm{/}2,1\mathrm{/}4$ at $0,1\mathrm{/}2,1$; the probability of error at least $1\mathrm{/}2$ is $1\mathrm{/}2$, matching the bound in step 2.1.

The power set on $\mathcal{X}$ is a sigma-algebra by [F1], so $(\mathcal{X},\mathcal{P}(\mathcal{X}))$ is a measurable space by [F2]. For every $u\in [0,1]$, the two singleton masses $u$ and $1-u$ are nonnegative and sum to one. Any disjoint sequence of subsets of $\mathcal{X}$ has at most two nonempty members, so the assignment is countably additive and gives zero to the empty set by [F3]; hence each $P_{(u,v)}$ is a probability measure by [F4]. This defines the experiment and model in [F5]–[F6], and $\psi$ is the target in [F7].

At ${\theta }_0=(1\mathrm{/}2,0)$ and ${\theta }_1=(1\mathrm{/}2,1)$, both laws assign probability $1\mathrm{/}2$ to each singleton and therefore are equal as measures on the power set. The target values are $0$ and $1$, whose distance is $1>0$. Thus [F8] makes the parameters observationally equivalent, while [F7] and [F9] show the target is not identifiable.

By [F11]–[F12], estimators are measurable maps on the IID sample spaces and consistency is pointwise under those sample laws. Apply [F13] to the pair from step 1.2: no estimator sequence is consistent in probability at both ${\theta }_0$ and ${\theta }_1$. For the finite check at sample size $n=2$, [F14] gives mass $(1\mathrm{/}2{)}^2=1\mathrm{/}4$ to each ordered pair under both product laws, confirming that the observation distributions remain identical.