Constrained least squares in a two-coordinate orthogonal sequence model

The three candidate supports are $\mathrm{\varnothing },\{1\},\{2\}$. Their fitted vectors are $0$, $({\epsilon }_1,0)$, and $(0,{\epsilon }_2)$, with residual squares ${\epsilon }_1^2+{\epsilon }_2^2$, ${\epsilon }_2^2$, and ${\epsilon }_1^2$, respectively. Hence the chosen fitted vector has squared norm $\max ({\epsilon }_1^2,{\epsilon }_2^2)$, including a tie.

Pointwise, ${\epsilon }_1^2\le \max ({\epsilon }_1^2,{\epsilon }_2^2)\le {\epsilon }_1^2+{\epsilon }_2^2$. The coordinate squares are integrable by the covariance assumption [F4], so their sum is integrable by [F6]. The nonnegative maximum is dominated by this sum, hence is integrable by nonnegative-integral monotonicity in [F6]. Taking expectations with [F6] and using [F4] gives ${\sigma }^2\le \mathbb{E}\mathrm{\parallel }{\widehat{\mu }}_s{\mathrm{\parallel }}_2^2\le 2{\sigma }^2$. Division by $n=2$ proves the stated interval for fitted-value error.

Here $s=1$, $d=2$, and $n=2$, and the noise satisfies [F3], so [F2] gives expected error at most $C{\sigma }^2\log (2e)\mathrm{/}2$. This is consistent with the direct calculation; the example makes no separate minimax lower-bound claim. By [F5], this expectation is the squared fitted-value risk.