Finite-Sample Point Estimation

Given: A measurable space, a point $x_0$ in it, its Dirac measure, and a nonnegative measurable function $f$.

${\delta }_{x_0}(A)=1$ if $x_0\in A$ and $0$ otherwise, and this set function is a probability measure on $(X,\mathcal{A})$. (The Dirac set function at a point, A Dirac set function is a probability measure)

A nonnegative simple function has a finite representation by disjoint measurable sets, and its simple integral is the sum of its values times the set measures. (Nonnegative simple measurable functions, The integral of a nonnegative simple function, The nonnegative integral agrees with the simple integral on simple functions)

Every nonnegative measurable function is the pointwise increasing limit of nonnegative simple functions. (Every nonnegative measurable function is the increasing limit of simple measurable functions)

Increasing limits pass through the nonnegative Lebesgue integral. (Monotone convergence for the integral)

Given: A measurable statistic $T$ from the data measurable space to the action measurable space.

A statistic is a measurable map; hence $T^{-1}(B)\in \mathcal{A}$ for every $B\in \mathcal{G}$. (Statistics and estimators)

For each point $d\in \mathsf{D}$, the Dirac set function ${\delta }_d$ is a probability measure on $(\mathsf{D},\mathcal{G})$. (The Dirac set function at a point, A Dirac set function is a probability measure)

A probability kernel has probability-measure sections and measurable event evaluations. (Measure kernel and probability kernel)

Given: A statistical experiment, a real estimand $\psi$, and an estimator $T$ that is square-integrable under each parameter law considered.

Bias is ${\mathbb{E}}_{\theta }[T]-\psi (\theta )$ and MSE is ${\mathbb{E}}_{\theta }[(T-\psi (\theta ){)}^2]$. (Bias and mean squared error)

For square-integrable real $T$, ${\mathrm{Var}}_{\theta }(T)={\mathbb{E}}_{\theta }[(T-{\mathbb{E}}_{\theta }T{)}^2]$. (Moments, variance, and covariance on a probability space)

The Lebesgue integral is linear on finite linear combinations of integrable functions. (The Lebesgue integral is linear on $L^1(\mu )$)

Cauchy–Schwarz gives $\int \mathrm{\mid }fg\mathrm{\mid }d\mu \le \mathrm{\parallel }f{\mathrm{\parallel }}_2\mathrm{\parallel }g{\mathrm{\parallel }}_2$ for square-integrable $f,g$. (Cauchy-Schwarz inequality for $L^2$)

Given: A standard-Borel experiment and statistic space, a sufficient statistic with the supplied common regular conditional kernel described in the Statement, AC, and an unbiased estimator $U$ with finite second moment under every $P_{\theta }$.

A finite-variance unbiased estimator is integrable, has expectation $\psi (\theta )$ at every parameter, and has the variance convention stated in the UMVU definition. (Unbiased estimators and UMVU estimators)

Sufficiency supplies one probability kernel $K$ from the standard-Borel statistic space to the standard-Borel data space such that, for every parameter $\theta$, data event $A$, and statistic event $B$, $P_{\theta }(A\cap S^{-1}(B))={\int }_{B}K(s,A)P_{\theta }^S(ds)$. The same kernel is fixed independently of $\theta$. (Sufficient statistics)

For a probability kernel, integrating a nonnegative measurable function gives a measurable function of the input, with the corresponding iterated-integral identity. (A randomized procedure has a measurable action law)

Positive/negative parts, absolute values, and the finite piecewise combinations used to define $g$ preserve measurability. (Closure properties of measurable functions used by the integral)

Nonnegative measurable functions admit increasing simple approximations. (Every nonnegative measurable function is the increasing limit of simple measurable functions)

The RCD identity for indicators extends to nonnegative increasing limits by monotone convergence and finite linearity on $L^1$. (Monotone convergence for the integral, The Lebesgue integral is linear on $L^1(\mu )$)

The nonnegative integral is monotone, so $\int u^{\pm }dK\le \int \mathrm{\mid }u\mathrm{\mid }dK$. (Monotonicity and nonnegative homogeneity of the nonnegative integral)

Every event in $\sigma (S)$ is $S^{-1}(B)$ for a Borel statistic-space event $B$. (The sigma-algebra generated by a function)

A conditional-expectation version is measurable, integrable, and has the same event integrals on the conditioning sigma-algebra. (Conditional expectation given a sigma algebra)

Under AC, versions with the same event integrals determine one almost-sure class. (Conditional expectation as an ae class)

Conditional expectation maps $L^2$ into $L^2$. (Conditional lp contraction)

Conditional expectation preserves ordinary expectation. (Basic algebra and order properties of conditional expectation)

Under AC, the conditional-expectation class of an $L^2$ input is its orthogonal projection onto the $L^2$ conditioning subspace. (Conditional expectation is the l2 orthogonal projection)

The Axiom of Choice is required by the cited conditional-expectation and $L^2$ projection interfaces. (The Axiom of Choice)

A nonnegative integrable function with zero integral is zero almost everywhere. (A nonnegative measurable function has integral $0$ exactly when it vanishes almost everywhere)

A randomized procedure is a parameter-independent probability kernel from an experiment's data space to its measurable action space. (Randomized procedures as probability kernels)

Given: AC, a standard-Borel experiment, a complete sufficient statistic $S$, and a finite-variance unbiased estimator $g(S)$ of a real estimand $\psi$.

UMVU compares the variances of unbiased finite-variance estimators at every parameter. (Unbiased estimators and UMVU estimators)

For any unbiased finite-variance estimator $U$, Rao–Blackwellization by $S$ is an unbiased finite-variance function of $S$, has no larger variance, and equality at $\theta$ holds exactly when $U$ is $\sigma (S)$-measurable $P_{\theta }$-almost surely. (Rao–Blackwell theorem for scalar finite-variance estimators)

Sufficiency supplies one probability kernel $K$ from the standard-Borel statistic space to the standard-Borel data space such that, for every parameter $\theta$, data event $A$, and statistic event $B$, $P_{\theta }(A\cap S^{-1}(B))={\int }_{B}K(s,A)P_{\theta }^S(ds)$. The same kernel is fixed independently of $\theta$. (Sufficient statistics)

Completeness says that for every measurable real-valued $h(S)$ integrable under every $P_{\theta }$, if ${\mathbb{E}}_{\theta }h(S)=0$ for every $\theta$, then $h(S)=0$ $P_{\theta }$-almost surely for every $\theta$. (Complete statistics)

The conditional-expectation results used by Rao–Blackwell assume AC, which is stated in that theorem and carried here. (The Axiom of Choice)

An integrable $\mathcal{G}$-measurable function is its own conditional-expectation version given $\mathcal{G}$. An integrable function equal to it almost surely has the same conditional-expectation class, and versions determine one almost-sure class. (Conditional expectation given a sigma algebra, Conditional expectation as an ae class)