Loss, Risk and Decision Theory — Examples

For each $\theta \in [0,1]$, define $P_{\theta }(A)=\theta 1_{\{1\in A\}}+(1-\theta )1_{\{0\in A\}}$ for $A\subseteq \{0,1\}$. Its singleton masses are nonnegative and sum to one; any disjoint sequence of subsets has at most two nonempty terms, and their masses add to the union mass. Thus $P_{\theta }$ is a probability measure by [F1]. The identity rule $d(x)=x$ is measurable on the finite full-power-set spaces and estimates the target $\psi (\theta )=\theta$ by [F2].

The finite expectation formula [F4] and risk definition [F3] give $R(\theta ,d)=(1-\theta )(0-\theta {)}^2+\theta (1-\theta {)}^2=\theta (1-\theta )$.

Since $\theta (1-\theta )=\frac{1}{4}-(\theta -\frac{1}{2}{)}^2$, the risk is at most $1\mathrm{/}4$, with equality at $\theta =1\mathrm{/}2$; substitution gives risk zero at $\theta =0$ and $\theta =1$.

The one-point data law assigns mass one to its only point under either parameter, so it is a probability measure by [F1]. For each $q\in [0,1]$, define ${\delta }_q(\ast ,B)=q{1}_{\{1\in B\}}+(1-q)1_{\{0\in B\}}$ for $B\subseteq \{0,1\}$; its two singleton masses are nonnegative and sum to one, hence it is a probability measure. Evaluations are measurable on the one-point data space, so ${\delta }_q$ is a probability kernel by [F2]. Deterministic rules are the cases $q=0$ and $q=1$.

By the finite expectation formula [F4] and the loss-risk definition [F3], the rule with probability $q$ of action $1$ has risks $R(0)=q$ and $R(1)=1-q$. Its worst-case risk is therefore $\max (q,1-q)\ge (q+1-q)\mathrm{/}2=1\mathrm{/}2$, with equality at $q=1\mathrm{/}2$.

The two deterministic rules have $q=0$ or $q=1$, so each has worst-case risk $1$. The fair randomized rule has worst-case risk $1\mathrm{/}2$, and step 1.2 shows no rule can do better. Thus the minimax value is $1\mathrm{/}2$ and randomization strictly improves worst-case risk over every deterministic rule.

For each $\theta \in \{0,1\}$, let $P_{\theta }$ be the point mass at $\theta$. In any disjoint sequence of events at most one event contains $\theta$, so this law is countably additive and has total mass one; it is a probability measure by [F1]. Both maps $d_0(x)=0$ and $d_1(x)=x$ are measurable on the finite full-power-set spaces by [F2].

Under squared-error loss, the only data value with positive $P_{\theta }$-mass is $x=\theta$. The finite-sum formula [F5] and risk definition [F3] therefore give $R(\theta ,d_0)={\theta }^2=\theta$ and $R(\theta ,d_1)=(\theta -\theta {)}^2=0$.

The risks satisfy $R(\theta ,d_1)\le R(\theta ,d_0)$ at both parameters, and the inequality is strict at $\theta =1$. Thus $d_1$ dominates the constant rule $d_0$ by [F4].

The prior and data law are point masses on singleton standard-Borel spaces by [F1]; their joint law is the point mass on the singleton pair. Define the posterior to be the point mass at $0$. The only conditioning events are empty or the whole singleton, and the conditional-law identity holds for both, so this is a regular conditional law by [F3]. Integrating $L(0,a)=a$ against it gives posterior risk $q(a)=a$ by [F2].

For every $a\in (0,1)$, $q(a)=a>0$, while the sequence $a_n=1\mathrm{/}(n+1)$ lies in $(0,1)$ and has $q(a_n)\to 0$. Thus ${\inf }_{a\in (0,1)}q(a)=0$ and the infimum is not attained. The claimed universal minimizer does not exist, which refutes the statement and illustrates the explicit existence premise in [F4].