Contamination and Robust Risk

For each $A\in \mathcal{A}$, both terms in $M_t(A)$ are nonnegative and finite. Also $M_t(\mathrm{\varnothing })=0$ and $M_t(X)=(1-t)+t=1$, using [F1].

If $(A_j{)}_{j\ge 1}$ are pairwise disjoint, countable additivity of $P,Q$ gives $M_t({\bigcup }_j{A}_j)=(1-t){\sum }_{j}P(A_j)+t{\sum }_{j}Q(A_j)={\sum }_j{M}_t(A_j)$. Together with step 1.1, this is a probability measure.

For every $z\in \mathcal{Z}$, apply the separation condition to $a=\widehat{\theta }(z)$. At least one of $\mathrm{\ell }({\theta }_0,\widehat{\theta }(z))$ and $\mathrm{\ell }({\theta }_1,\widehat{\theta }(z))$ is at least $\kappa \mathrm{\ell }({\theta }_0,{\theta }_1)$. The loss-event sets are measurable by [given] and [F2], so the two event indicators sum to at least one pointwise.

Integrating that indicator inequality under the probability measure $R$ gives $R(E_0)+R(E_1)\ge 1$, where $E_i=\{\mathrm{\ell }({\theta }_i,\widehat{\theta })\ge \kappa \mathrm{\ell }({\theta }_0,{\theta }_1)\}$. Hence at least one probability is at least $1\mathrm{/}2$.

If $ϵ=1$, set $Q_0=Q_1=P_0$; both displayed mixtures equal $P_0$. If $ϵ<1$ and $\mathrm{\parallel }{P}_0-P_1{\mathrm{\parallel }}_{TV}=0$, [F1] gives $P_0=P_1$, so again set $Q_0=Q_1=P_0$. It remains to consider $0<ϵ<1$ and $\delta :=\mathrm{\parallel }{P}_0-P_1{\mathrm{\parallel }}_{TV}>0$.

Put $\nu =P_0+P_1$. Countable additivity and nonnegativity of $P_0,P_1$ show that $\nu$ is a positive measure, and $\nu (X)=2$, so it is finite and sigma-finite. If $\nu (A)=0$, then $P_0(A)+P_1(A)=0$ with both terms nonnegative, so each is zero; hence $P_i\ll \nu$. By [A1], [F3], and [F4], choose nonnegative measurable densities $p_i$ with $P_i(A)={\int }_A{p}_{i}d\nu$ for every measurable $A$. In particular $\int p_{i}d\nu =1$, so $p_i\in L^1(\nu )$.

Let $g=p_0-p_1$, with measurable positive and negative parts $g^{+},g^{-}$ by [F6]. Since $\int gd\nu =0$, [F5] gives $\int g^{+}d\nu =\int g^{-}d\nu$. For every measurable $A$, $\mathrm{\mid }{\int }_{A}gd\nu \mathrm{\mid }\le \max (\int g^{+}d\nu ,\int g^{-}d\nu )$; the set $A=\{g>0\}$ is measurable by [F7] and attains ${\int }_{A}gd\nu =\int g^{+}d\nu$. Therefore [F1] gives $\delta =\int g^{+}d\nu =\int g^{-}d\nu$.

Define $Q_0(A)={\delta }^{-1}{\int }_A{g}^{-}d\nu$ and $Q_1(A)={\delta }^{-1}{\int }_A{g}^{+}d\nu$. By [F8] these are measures; step 2.1 gives total mass one, so they are probability measures.

Let ${ϵ}^{\mathrm{\prime }}=\delta \mathrm{/}(1+\delta )$. The assumed inequality implies ${ϵ}^{\mathrm{\prime }}\le ϵ$, and ${ϵ}^{\mathrm{\prime }}\mathrm{/}\delta =1-{ϵ}^{\mathrm{\prime }}$. The densities of $(1-{ϵ}^{\mathrm{\prime }})P_0+{ϵ}^{\mathrm{\prime }}{Q}_0$ and $(1-{ϵ}^{\mathrm{\prime }})P_1+{ϵ}^{\mathrm{\prime }}{Q}_1$ are respectively $(1-{ϵ}^{\mathrm{\prime }})(p_0+g^{-})=(1-{ϵ}^{\mathrm{\prime }})\max (p_0,p_1)$ and $(1-{ϵ}^{\mathrm{\prime }})(p_1+g^{+})=(1-{ϵ}^{\mathrm{\prime }})\max (p_0,p_1)$. Both mixtures are probability measures by [F9], and equal densities give equality of the measures.

By [F10], the target laws have the Huber mixture form. Since $ϵ>0$, set ${\tilde{Q}}_i=({ϵ}^{\mathrm{\prime }}\mathrm{/}ϵ)Q_i+(1-{ϵ}^{\mathrm{\prime }}\mathrm{/}ϵ)P_i$. By [F9] each ${\tilde{Q}}_i$ is a probability measure, and $(1-ϵ)P_i+ϵ{\tilde{Q}}_i=(1-{ϵ}^{\mathrm{\prime }})P_i+{ϵ}^{\mathrm{\prime }}{Q}_i$. Step 4.1 therefore gives the required equality at the original fraction $ϵ$.

Steps 1.1 and 5.1 cover all endpoint and positive-distance cases, respectively.

Suppose $\omega (ϵ,\mathrm{\Theta })\le M_0$. For any estimator $\widehat{\theta }$, the clean premise says its supremum over $\theta$ of the clean probability at threshold $M_0$ is at least $c_0$. For each $\theta$, the Huber supremum may choose $Q=P_{\theta }$, giving $P_{ϵ,\theta ,Q}=P_{\theta }$ by [F1]. Since $M_{ϵ}=(\kappa \mathrm{/}4)M_0\le M_0$, the event at $M_{ϵ}$ contains the clean event at $M_0$. Taking suprema therefore gives at least $c_0$ for every estimator.

Suppose $\omega (ϵ,\mathrm{\Theta })>M_0$ and is finite. Then $\omega >0$, so the definition of supremum supplies ${\theta }_0,{\theta }_1$ with $L({\theta }_0,{\theta }_1)>\omega \mathrm{/}2$; if $ϵ<1$ this pair satisfies $TV(P_{{\theta }_0},P_{{\theta }_1})\le ϵ\mathrm{/}(1-ϵ)$, while if $ϵ=1$ every pair is allowed.

If $\omega (ϵ,\mathrm{\Theta })\le M_0$, step 1.1 gives the asserted bound.

By [F4] choose $Q_0,Q_1$ making the two one-observation Huber laws equal to a common law $R_1$. By [F1], each sample law is the product of $N$ copies of $R_1$. The finite product sigma-algebra and measure are formed recursively; at each stage the existing partial product is sigma-finite, so [F6] supplies the next factor product. Both parameters therefore give the same recursively constructed sample law $R:=R_1^{\otimes N}$.

The loss-separation hypothesis gives ${\max }_{i}L({\theta }_i,a)\ge \kappa L({\theta }_0,{\theta }_1)>\kappa \omega \mathrm{/}2>M_{ϵ}$ for every $a$. Thus the events at threshold $\kappa L({\theta }_0,{\theta }_1)$ in [F5] are contained in those at threshold $M_{ϵ}$. Apply [F5] under $R$ to get ${\max }_{i}R\{L({\theta }_i,\widehat{\theta })\ge M_{ϵ}\}\ge 1\mathrm{/}2$. The supremum over $\theta ,Q$ is at least this maximum for each estimator.

If $\omega \le M_0$, step 2.1 gives $c_0$; if $\omega >M_0$, step 3.1 gives $1\mathrm{/}2$. Taking the infimum over estimators proves the stated $\min (c_0,1\mathrm{/}2)$ bound. The explicit formula for $M_{ϵ}$ gives comparability to $M_0\vee \omega$. If $c_0\ge c_{\ast }>0$ for a fixed $c_{\ast }\le 1\mathrm{/}2$, then $\min (c_0,1\mathrm{/}2)\ge c_{\ast }$, yielding a universal probability constant under that additional clean-bound normalization.

If $\omega =+\mathrm{\infty }$, fix any finite threshold $T$. The supremum definition gives a pair with $L({\theta }_0,{\theta }_1)>2T\mathrm{/}\kappa$ satisfying the TV constraint (or any pair when $ϵ=1$). Equalize their mixtures by [F4]. Pairwise separation makes the events at threshold $\kappa L({\theta }_0,{\theta }_1)$ in [F5] subsets of the corresponding events at threshold $T$, so [F5] gives a maximum tail probability at threshold $T$ of at least $1\mathrm{/}2$. This holds for every estimator and hence for the minimax supremum.