Sufficiency, Completeness and Exponential Families — Examples

The endpoint laws are $P_0={\delta }_0$ and $P_1={\delta }_1$, so $Q=\frac{1}{2}{\delta }_0+\frac{1}{2}{\delta }_1$ assigns mass $1\mathrm{/}2$ to each point. This is a probability measure.

For every $A\subseteq \{0,1\}$, if $Q(A)=0$ then $A=\mathrm{\varnothing }$, so $P_{\theta }(A)=0$; hence $P_{\theta }\ll Q$. Define $q_{\theta }(0)=2(1-\theta )$ and $q_{\theta }(1)=2\theta$. For each subset $A$, ${\int }_A{q}_{\theta }dQ=1_{\{0\in A\}}(1-\theta )+1_{\{1\in A\}}\theta =P_{\theta }(A)$. Thus $q_{\theta }=d{P}_{\theta }\mathrm{/}dQ$, including its zero values at endpoints, and the displayed finite mixture directly dominates every member.

Define $K(t,A)=1_A((t,0))$ for $t\in \{0,1\}$. Each section is the Dirac set function [F5]: it vanishes on the empty set, has total mass one, and is countably additive because a disjoint family contains $(t,0)$ in at most one member. Thus each section is a probability measure [F4]. All evaluations are measurable on the finite full sigma-field, so [F6] makes $K$ a probability kernel independent of $\theta$, between the standard Borel spaces in [F2]. For every $A\subseteq \mathcal{X}$ and $C\subseteq \{0,1\}$, $P_{\theta }(A\cap T^{-1}(C))={\sum }_{t\in C}{1}_A((t,0))P_{\theta }(\{(t,0)\})={\int }_{C}K(t,A)P_{\theta }^T(dt)$. Hence $T$ is sufficient by [F1].

Every atom of $R$ has mass $1\mathrm{/}4$, so $P_{\theta }\ll R$. The density values are $d{P}_{\theta }\mathrm{/}dR(t,u)=4P_{\theta }(t,0)$ if $u=0$ and zero if $u=1$. For fixed $t$, the two points $(t,0)$ and $(t,1)$ have the same $T$ value but different derivative values. Every atom has positive $R$-mass, so no modification on an $R$-null set can change them; the derivative has no $\sigma (T)$-measurable version.

This refutes the implication with an arbitrary external reference. Taking the reference to be a member $P_{{\theta }_0}$ restores the common conditional law needed for the criterion: the conditional kernel at the reference is then the model’s shared kernel. The counterexample’s $R$ has conditional second coordinate uniform given $T$, while each model law has second coordinate zero, identifying the precise missing hypothesis.

The map $T$ is polynomial and hence a statistic by [F2]. Let ${\lambda }_B$ be one-dimensional Lebesgue measure restricted to $\mathcal{B}(\mathbb{R})$; the Borel half-open intervals $(-k,k]$ give it a sigma-finite cover of finite measure by [F5]. By [F4] and the affine change of variables [F10] for $z\mapsto \theta +\theta z$, each $X_i$ has Borel density $q_{\theta }(x_i)=(\sqrt{2\pi }\theta {)}^{-1}{e}^{-(x_i-\theta {)}^2\mathrm{/}(2{\theta }^2)}$ relative to ${\lambda }_B$. Independence [F5] gives the joint law $P_{\theta }^{X_1}\times P_{\theta }^{X_2}$. The measure with density $q_{\theta }(x_1)q_{\theta }(x_2)$ relative to ${\lambda }_B\times {\lambda }_B$ exists by [F9]; Tonelli [F6] shows that it has total mass one and agrees with this product law on every measurable rectangle. By [F11], these rectangles generate the Borel sigma-algebra of ${\mathbb{R}}^2$, so the product-measure uniqueness clause [F9] identifies this density measure with the joint law. Expanding its density gives $p_{\theta }(x_1,x_2)=(2\pi {)}^{-1}{\theta }^{-2}\exp (-S_2\mathrm{/}(2{\theta }^2)+S_1\mathrm{/}\theta -1)$. Put $u=1\mathrm{/}\theta$, $h(x_1,x_2)=(2\pi {)}^{-1}$, $\mathrm{\Omega }=\{(u,-u^2\mathrm{/}2):u>0\}$, and $A(u,-u^2\mathrm{/}2)=1-2\log u$. Then $p_{\theta }=h\exp \{u{S}_1-u^2{S}_2\mathrm{/}2-A(u,-u^2\mathrm{/}2)\}$, with normalizer $Z(u,-u^2\mathrm{/}2)=e^A=e\mathrm{/}{u}^2$; the normalization follows from the product normal density just derived. Thus the canonical statistic is $(S_1,S_2)$, and the natural parameter set lies on ${\eta }_2=-{\eta }_1^2\mathrm{/}2$, so it contains no open subset of ${\mathbb{R}}^2$. Both densities are strictly positive, so $P_{\theta }\ll P_{{\theta }_0}$ for any fixed ${\theta }_0>0$. Their ratio is a Borel function of $T$; since $P_{{\theta }_0}$ is a model member, [F7] proves sufficiency.

From $X_i=\theta +\theta Z_i$ and [F4], $E_{\theta }{X}_i=\theta$ and $E_{\theta }{X}_i^2=2{\theta }^2$. The bound $\mathrm{\mid }z\mathrm{\mid }\le 1+z^2$ gives $E\mathrm{\mid }{Z}_i\mathrm{\mid }\le 1+E{Z}_i^2=2$, hence $E_{\theta }\mathrm{\mid }{X}_i\mathrm{\mid }<\mathrm{\infty }$. Let $q_{\theta }$ be the one-dimensional normal density from step 1.1 and write $x^{+}=\max (x,0)$, $x^{-}=\max (-x,0)$. Independence [F5] identifies the joint law with the product of the two marginal laws, each having density $q_{\theta }$. Tonelli [F6] therefore gives, for every $a,b\in \{+,-\}$, $E_{\theta }[X_1^a{X}_2^b]=({\int }_{\mathbb{R}}{x}^a{q}_{\theta }(x)dx)({\int }_{\mathbb{R}}{x}^b{q}_{\theta }(x)dx)$, where all four factors are finite by $E_{\theta }\mathrm{\mid }{X}_i\mathrm{\mid }<\mathrm{\infty }$. Subtracting the four finite identities yields $E_{\theta }[X_1{X}_2]=(E_{\theta }{X}_1)(E_{\theta }{X}_2)$; linearity [F8] now gives $E_{\theta }{S}_1^2=E{X}_1^2+E{X}_2^2+2E{X}_{1}E{X}_2=6{\theta }^2$, while $E_{\theta }{S}_2=4{\theta }^2$. Hence $E_{\theta }g(T)=6{\theta }^2-\frac{3}{2}(4{\theta }^2)=0$ for every $\theta >0$. Also $\mathrm{\mid }g(T)\mathrm{\mid }\le S_1^2+\frac{3}{2}{S}_2$, whose expectation is finite, so $g(T)$ is integrable.

At $(x_1,x_2)=(1,0)$, $g(T)=-1\mathrm{/}2$. By continuity it is nonzero on a neighborhood. The density in step 1.1 is continuous and strictly positive at this point, so there is a smaller nondegenerate rectangle inside that neighborhood on which the density is bounded below by half its value at $(1,0)$. The rectangle has positive ${\lambda }_B\times {\lambda }_B$ measure by the one-dimensional box formula and the product rectangle formula [F5, F9], hence it has positive probability under every $\theta >0$. Therefore $g(T)$ is not zero almost surely, contradicting completeness [F3].

The mean map is continuous, hence Borel by [F2] and a statistic. Use $n$-dimensional Lebesgue measure $dx$ as base measure; it is sigma-finite by [F3]. By [F1] and the translation case of [F10], each coordinate has density $\varphi (x_i-\mu )$. Independence [F4] gives the product joint law. For $n=1$ this is already the joint density. For $n\ge 2$, repeated use of [F13] identifies ${\lambda }_n$ with the product of the one-dimensional Lebesgue measures. The measure with density $q(x)={\prod }_i\varphi (x_i-\mu )$ relative to ${\lambda }_n$ exists by [F11]. Repeated Tonelli [F4] and the rectangle formula [F12] show that its values on Borel rectangles are the products of the marginal probabilities; [F14] says these rectangles generate the Borel sigma-algebra of ${\mathbb{R}}^n$, so repeated product-measure uniqueness [F12] identifies it with the joint law. Thus $p_{\mu }(x)=(2\pi {)}^{-n\mathrm{/}2}\exp (-\frac{1}{2}{\sum }_i{x}_i^2+\mu {\sum }_i{x}_i-\frac{n}{2}{\mu }^2)$. Setting $T=\bar{X}$, $\eta =n\mu$, and $h(x)=(2\pi {)}^{-n\mathrm{/}2}{e}^{-{\sum }_i{x}_i^2\mathrm{/}2}$ writes it as $h(x)e^{\eta T(x)-A(\eta )}$ with $A(\eta )={\eta }^2\mathrm{/}(2n)$. Completing the square and using Tonelli gives $\int h{e}^{\eta T}dx=e^{{\eta }^2\mathrm{/}(2n)}$. The natural parameter set is all $\mathbb{R}$, hence open. For $b\mathrm{\ne }0$, the level set $b\bar{x}=c$ is a proper affine hyperplane, null by [F3], so $T$ has no affine constraint; it is full rank by [F5]. Thus [F6] and [F7] make $\bar{X}$ sufficient and complete.

The residual map $x\mapsto x-\bar{x}\mathbf{1}$ is continuous, hence a statistic by [F2]. Write $X_i=\mu +Z_i$ with iid $Z_i\sim N(0,1)$ by [F1]. Then $R_i=Z_i-\bar{Z}$, so the law of $R$ does not depend on $\mu$; it is ancillary by [F8]. Apply [F9] to conclude $\bar{X}$ and $R$ are independent under every $P_{\mu }$.

Every sample point has probability $m_p(x)=p^{t(x)}(1-p{)}^{n-t(x)}>0$, so the common support is the whole finite sample space. Counting measure assigns mass one to each point, so the densities equal these point masses; for $x,y$, their ratio is $m_p(x)\mathrm{/}{m}_p(y)=[p\mathrm{/}(1-p){]}^{t(x)-t(y)}$.

If $t(x)=t(y)$, this ratio is one for all $p$. If $d=t(x)-t(y)\mathrm{\ne }0$, its values at $p=1\mathrm{/}2$ and $p=1\mathrm{/}3$ are respectively $1$ and $(1\mathrm{/}2{)}^d$, which differ. Thus equivalence classes are exactly the level sets of $T$. By [F2], $T$ is minimal sufficient.

Since $Q(\{j\})=2^{-j}$ and $P_k(\{j\})=1$ exactly when $j=k$, the density is $f_k(j)=2^k{1}_{\{j=k\}}$. Therefore $T(j)$ has coordinate $j$ equal to $2^j$ and all other coordinates zero.

The discrete metric on ${\mathbb{N}}_{\ge 1}$ is complete and separable, and its Borel sigma-field is the full sigma-field. The weights $2^{-k}>0$ sum to one, and step 1.1 gives finite nonnegative measurable density versions. Since every $Q$-atom has positive mass, the common $Q$-null set on which these versions are reset is empty. Thus all hypotheses of [F1], including AC [A1], hold, and that theorem proves minimal sufficiency. The vectors in step 1.1 are distinct for distinct $j$, proving injectivity. Finally, a set null under every $P_j$ contains no $j$, so the family-null ideal is trivial and the minimality comparison in [F2] is exact in this model.