Statistical Experiments, Models and Identifiability

An experiment is a measurable space $(\mathcal{X},\mathcal{A})$, a set $\mathrm{\Theta }$ of parameter labels, and a family $(P_{\theta }{)}_{\theta \in \mathrm{\Theta }}$ of probability measures on $(\mathcal{X},\mathcal{A})$ (Measurable spaces and measurable sets, Probability measures and probability spaces). The measures $P_{\theta }$ are the sampling laws. If data under a parameter value are realized as a random element, its law is the pushforward measure $P_X(B)=P(X^{-1}(B))$ from Law or distribution of a random element; it is a probability measure by The law of a random element is a probability measure.

The definition specifies the family of laws only. It supplies no sigma-algebra on $\mathrm{\Theta }$, no measurability condition on $\theta \mapsto P_{\theta }$, and no common underlying probability space on which all observations must be realized.

For an experiment on $(\mathcal{X},\mathcal{A})$, let $\mathcal{P}(\mathcal{X},\mathcal{A})$ denote the set of all probability measures on that measurable space. Its parametric statistical model is the parameter map

regarded together with its parameter set and sample space (Statistical experiments). The map need not be injective. Unless a measurable structure on $\mathrm{\Theta }$ is supplied separately, no measurability of $\mathsf{P}$ is asserted.

Suppose an experiment on $(\mathcal{X},\mathcal{A})$ also has a measurable parameter space $(\mathrm{\Theta },\mathcal{T})$ (Measurable spaces and measurable sets). A Markov kernel (or probability kernel) representation is a function

such that, for every $\theta \in \mathrm{\Theta }$, the set function $A\mapsto K(\theta ,A)$ is a probability measure on $(\mathcal{X},\mathcal{A})$, and, for every $A\in \mathcal{A}$ and every Borel set $C\subseteq [0,1]$,

Here the Borel sigma-algebra on $[0,1]$ is generated by its relatively open subsets $U\cap [0,1]$, where $U$ is open in $\mathbb{R}$. Thus the second condition says that each event-probability map $\theta \mapsto K(\theta ,A)$ is measurable. The kernel represents the given experiment exactly when $K(\theta ,A)=P_{\theta }(A)$ for every $\theta$ and $A$ (Statistical experiments, Probability measures and probability spaces).

This is an optional parameter-measurable version of a model. An arbitrary experiment does not include a measurable structure on $\mathrm{\Theta }$ or imply this kernel condition.

In a parametric model $\mathsf{P}:\mathrm{\Theta }\to \mathcal{P}(\mathcal{X},\mathcal{A})$, two parameter values $\theta ,\eta \in \mathrm{\Theta }$ are observationally equivalent, written $\theta \sim \eta$, if and only if

that is, $P_{\theta }(A)=P_{\eta }(A)$ for every $A\in \mathcal{A}$ (Parametric statistical models). Equivalent parameter values therefore give exactly the same sampling law. No distinctness of $\theta$ and $\eta$ is required.

For a parametric model with parameter set $\mathrm{\Theta }$, an estimand or parameter target is a specified function

where ${\mathcal{T}}_0$ is a set of target values (Parametric statistical models). This definition imposes no measurable structure on $\mathrm{\Theta }$ or ${\mathcal{T}}_0$ and does not require $\psi$ to determine the full parameter value.

An estimand $\psi :\mathrm{\Theta }\to {\mathcal{T}}_0$ is identifiable in a parametric model $\mathsf{P}$ when

or equivalently when $\psi$ is constant on each set of parameter values with the same sampling law (Parametric statistical models, Observational equivalence, Estimands on a statistical model). The model is identifiable when its identity target ${\mathrm{id}}_{\mathrm{\Theta }}:\mathrm{\Theta }\to \mathrm{\Theta }$ is identifiable, equivalently when $\mathsf{P}$ is injective.

Let $\mathsf{P}:\mathrm{\Theta }\to \mathcal{P}(\mathcal{X},\mathcal{A})$ be a parametric model, let $\psi :\mathrm{\Theta }\to {\mathcal{T}}_0$ be an estimand, and set $\mathcal{M}:=\mathsf{P}(\mathrm{\Theta })$, the set of laws in the model. For $\theta \in \mathrm{\Theta }$, write

Then the following are equivalent: (i) $\psi$ is identifiable; (ii) $\psi$ is constant on every observational equivalence class $[\theta {]}_{\sim }$; and (iii) there is a unique function $\bar{\psi }:\mathcal{M}\to {\mathcal{T}}_0$ such that $\psi =\bar{\psi }\circ \mathsf{P}$.

The factorization is a statement about sets and functions. It asserts no measurable structure or measurable factor map.