Cutoff risk and minimax RMS error over statistical ellipsoids

If $Q=0$, then ${\mathrm{\Theta }}_a(Q)=\{0\}$. The zero estimator has minimax risk zero, and the $D=0$ cutoff expression is also zero, so the asserted inequalities hold. Hence assume $Q>0$.

Put ${\tau }_j^2={ϵ}^2{s}_j^{-2}$ and let $\mathcal{C}$ be the set of sequences $r_j\ge 0$ with ${\sum }_j{a}_j^2{r}_j\le Q^2$. For $r\in \mathcal{C}$, define $J(r)={\sum }_j\min (r_j,{\tau }_j^2)$. Since $a_j\ge a_1>0$, one has ${\sum }_j{r}_j\le Q^2\mathrm{/}{a}_1^2$, so $J(r)$ is finite.

As $s_j\to 0$ and $ϵ>0$, ${\tau }_j^2\to \mathrm{\infty }$. Choose $N\ge 1$ so that ${\tau }_j^2>Q^2\mathrm{/}{a}_1^2$ for $j>N$. Every feasible $r$ then has $r_j<{\tau }_j^2$ on that tail. Replacing its tail by ${\tilde{r}}_{N+1}=({\sum }_{j>N}{a}_j^2{r}_j)\mathrm{/}{a}_{N+1}^2$ at coordinate $N+1$ and zero thereafter preserves the weighted budget and does not decrease $J$, because $a_j\ge a_{N+1}$ for $j>N$ and ${\sum }_{j>N}{r}_j\le {\tilde{r}}_{N+1}<{\tau }_{N+1}^2$. The finite-dimensional feasible set in coordinates $1,\dots ,N+1$ is nonempty (it contains zero), closed, and bounded (each coordinate is at most $Q^2\mathrm{/}{a}_j^2$). The finite sum $J$ is continuous there, so [F9] gives a finite-support maximizer $r^{\ast }$.

For $k\in \mathbb{N}$, put $c_k:=(k+3)!$. By induction, $c_k\ge 6\cdot 3^k$, with strict inequality for $k\ge 1$: the base case is $c_0=3!=6$, and if $c_k\ge 6\cdot 3^k$, then $c_{k+1}=(k+4)c_k\ge 4c_k\ge 24\cdot 3^k>18\cdot 3^k=6\cdot 3^{k+1}$, using $(k+4)\ge 4$ and $c_k>0$. Thus for every $N\ge 4$, ${\sum }_{n=0}^N\frac{1}{n!}\le \frac{5}{2}+\frac{1}{6}{\sum }_{k=0}^{N-3}{3}^{-k}-\frac{1}{72},$ because the $n=4$ term is strictly below its comparison term by $1\mathrm{/}18-1\mathrm{/}24=1\mathrm{/}72$ and all other terms from $n\ge 3$ are bounded by the induction. For $k=0$ one has $(1\mathrm{/}3{)}^0=3^0=1$. For $k\ge 1$, the product and reciprocal laws in [F7] give $3^k(1\mathrm{/}3{)}^k=(3(1\mathrm{/}3){)}^k=1$, hence $(1\mathrm{/}3{)}^k=(3^k{)}^{-1}=3^{-k}$. Applying the geometric-series formula in [F7] at ratio $1\mathrm{/}3$ therefore gives ${\sum }_{k\ge 0}{3}^{-k}=1\mathrm{/}(1-1\mathrm{/}3)=3\mathrm{/}2$. By [F7], the left partial sums tend to $e$ and these geometric sums tend to $3\mathrm{/}2$; [F7] therefore gives $e\le \frac{11}{4}-\frac{1}{72}<\frac{11}{4}$. Hence $2\sqrt{2e}<\sqrt{22}<\frac{121}{25}=(\frac{11}{5}{)}^2$.

The compact-SVD coordinate realization uses Countable Choice [A1]. For each $D$, the first $D$ coordinate evaluations are measurable in the product Borel data space [F1], and their finite-support embedding into ${\mathrm{\ell }}^2$ is continuous; thus the cutoff rule [F3] is measurable, including the constant rule at $D=0$. Its squared loss is measurable by [F10].

Among the finite-support maximizers choose one, $r^{\ast }$, with the longest initial saturated segment, and let $D$ be its length; this choice exists because step 1.3 bounds the support by $N+1$. If $P=\{j:r_j^{\ast }\ge {\tau }_j^2\}$ had a later saturated index $j>D+1$, then $i=D+1$ is the first gap before such an index and $r_i^{\ast }<{\tau }_i^2$. Set $\delta ={\tau }_i^2-r_i^{\ast }>0$. Since $s_j$ is nonincreasing, ${\tau }_i^2\le {\tau }_j^2$, so $\delta \le {\tau }_i^2\le {\tau }_j^2\le r_j^{\ast }$. Replacing $r_i^{\ast }$ by $r_i^{\ast }+\delta$ and $r_j^{\ast }$ by $r_j^{\ast }-\delta$ does not increase the budget, since $a_i\le a_j$, and does not decrease $J$, since the gain at $i$ is $\delta$ and the loss at $j$ is at most $\delta$. It therefore gives a maximizer whose initial saturated segment has length at least $D+1$, a contradiction. Thus $P=\{1,\dots ,D\}$ for some $D\ge 0$.

Let the prior choose independently, for each $j\le N+1$, the two values $\pm b_j$ with equal probability, where $b_j=\min \{\sqrt{r_j^{\ast }},{\tau }_j\}$, and set all later coordinates to zero. Every point in its finite support belongs to ${\mathrm{\Theta }}_a(Q)$ because ${\sum }_{j\le N+1}{a}_j^2{b}_j^2\le {\sum }_j{a}_j^2{r}_j^{\ast }\le Q^2$. For any such truth $\eta$, relative to the zero-truth product law $P_0$, the likelihood ratio is $L_{\eta }(z)={\prod }_{j=1}^{N+1}\exp ({\eta }_j{z}_j\mathrm{/}{\tau }_j^2-{\eta }_j^2\mathrm{/}(2{\tau }_j^2))$. For a cylinder determined by a finite coordinate set $F$, split $F$ into active coordinates $F\cap \{1,\dots ,N+1\}$ and inactive coordinates. On each active coordinate, multiplying the zero-mean Gaussian density by its factor in $L_{\eta }$ gives the shifted Gaussian density; inactive coordinates retain their zero-truth density, and active likelihood factors outside $F$ integrate to one. Finite-coordinate independence therefore gives equality of the two probability measures on every such cylinder, and the product density integrates to one. These cylinders form a generating pi-system, so [F8] gives equality on the full product Borel sigma-algebra and proves that $L_{\eta }$ is the likelihood ratio for the entire experiment. The finite product prior and this factorized likelihood give a coordinatewise posterior.

The admissible cutoff rule from step 1.5 has minimax risk no greater than its worst-case risk by [F6]. For each $D$ this risk is the exact expression in [F4]; taking the infimum over $D$ gives $R_{\ast }(ϵ)\le R_{\mathrm{c}\mathrm{u}\mathrm{t}}(ϵ)$.

Fix $r\in \mathcal{C}$ and $m\ge N+1$, and truncate $r$ after coordinate $m$; the truncation remains feasible. The segment from $r^{\ast }$ to this finite-support truncation is feasible, and its finite sum $J$ is maximized at its $r^{\ast }$ endpoint. Its right derivative there is therefore nonpositive. Since $P=\{1,\dots ,D\}$ by step 2.1, coordinates $j>D$ are strictly below their caps at $r^{\ast }$ and contribute their direction increments; coordinates $j\le D$ with $r_j^{\ast }>{\tau }_j^2$ contribute zero, while those with $r_j^{\ast }={\tau }_j^2$ contribute $\min (r_j-{\tau }_j^2,0)$. Consequently ${\sum }_{D<j\le m}(r_j-r_j^{\ast })+{\sum }_{\{j\le D:r_j^{\ast }={\tau }_j^2\}}\min (r_j-{\tau }_j^2,0)\le 0$. Letting $m\to \mathrm{\infty }$ is legitimate because $\sum r_j$ and $\sum r_j^{\ast }$ converge. Rearranging gives ${\sum }_{j>D}{r}_j\le {\sum }_{j>D}{r}_j^{\ast }+{\sum }_{\{j\le D:r_j^{\ast }={\tau }_j^2\}}({\tau }_j^2-r_j{)}_{+}\le {\sum }_{j>D}{r}_j^{\ast }+{\sum }_{j=1}^D{\tau }_j^2$.

For fixed observed data, the randomized rule draws its action from the parameter-independent kernel in [F6]; thus additionally conditioning on that action leaves the posterior from step 2.2 unchanged. For each active coordinate, conditional on the data and action, squared error averaged over this posterior is its posterior variance plus a nonnegative squared displacement from the posterior mean. Averaging over the action kernel and then over the data preserves the posterior-variance lower bound. Thus every rule's prior-averaged risk is at least the sum of the scalar Bayes risks. Each scalar prior is supported at $\pm \min \{\sqrt{r_j^{\ast }},{\tau }_j\}$, so [F5] bounds that sum below by ${\sum }_{j\le N+1}\min (r_j^{\ast },{\tau }_j^2)\mathrm{/}\sqrt{2e}=J_{\ast }\mathrm{/}\sqrt{2e}$. The supremum risk dominates the supported prior average; taking the minimax infimum gives $R_{\ast }(ϵ{)}^2\ge J_{\ast }\mathrm{/}\sqrt{2e}$.

For any $\theta \in {\mathrm{\Theta }}_a(Q)$, the sequence $r_j={\theta }_j^2$ belongs to $\mathcal{C}$. Step 3.1 bounds its omitted squared norm. The cutoff risk is that tail plus the retained variance by [F4], so at the cutoff $D$ from step 2.1 its worst-case risk is at most ${\sum }_{j>D}{r}_j^{\ast }+2{\sum }_{j\le D}{\tau }_j^2\le 2J_{\ast }$. Therefore $R_{\mathrm{c}\mathrm{u}\mathrm{t}}(ϵ{)}^2\le 2J_{\ast }$.

Combining steps 4.1 and 3.2 yields $R_{\mathrm{c}\mathrm{u}\mathrm{t}}(ϵ{)}^2\le 2\sqrt{2e}{R}_{\ast }(ϵ{)}^2$.

Since both risks are nonnegative, steps 5.1 and 1.4 imply $R_{\mathrm{c}\mathrm{u}\mathrm{t}}(ϵ)\le \frac{11}{5}{R}_{\ast }(ϵ)$.