Unrolled reconstruction and finite-iteration limits — Examples and Counterexamples

The scalar map $A:\mathbb{R}\to \mathbb{R}$, $x\mapsto ax$, is nonzero and compact because $a>0$ and the space is finite-dimensional. For every $x$, $\mathrm{\parallel }Ax{\mathrm{\parallel }}_2=a\mathrm{\mid }x\mathrm{\mid }$, so $\mathrm{\parallel }A\mathrm{\parallel }=a$. For every $u,v\in \mathbb{R}$, $⟨Au,v⟩=auv=⟨u,av⟩$, hence $A^{\ast }$ is also multiplication by $a$. With [A1], $0<\omega a^2<1$ implies the supplier's condition $0<\omega <2\mathrm{/}\mathrm{\parallel }A{\mathrm{\parallel }}^2$. The initialization and scalar update formulas are continuous, hence Borel by [F3], so they satisfy [F2]'s estimator-domain condition. Together with the supplied $\tau >1$ and noise bound $\delta =0$, these verify every input condition of [F1], which gives the exact recurrence and its finite-budget stopping rule for this instance.

Substituting $y=a{x}^{†}$ gives $x_{k+1}-x^{†}=(1-\omega a^2)(x_k-x^{†})$. Starting from $x_0=0$, induction gives $x_k=x^{†}[1-(1-\omega a^2{)}^k]$ for every $0\le k\le N$, in particular $x_N=x^{†}[1-(1-\omega a^2{)}^N]$. Hence $\mathrm{\mid }y-a{x}_k\mathrm{\mid }=a\mathrm{\mid }{x}^{†}\mathrm{\mid }(1-\omega a^2{)}^k>0$ for every such finite $k$. Since $\tau \delta =0$, the discrepancy test fails even at $k=N$, so no earlier discrepancy stop occurs and the procedure returns this value at budget exhaustion.

For the learned architecture, ${\tilde{x}}_0=0$ and its displayed recurrence give ${\tilde{x}}_{k+1}-x^{†}=(1-{\tilde{\omega }}_k{a}^2)({\tilde{x}}_k-x^{†})$. Starting with ${\tilde{x}}_0-x^{†}=-x^{†}$, repeated substitution yields ${\tilde{x}}_N-x^{†}=-x^{†}{\prod }_{k=0}^{N-1}(1-{\tilde{\omega }}_k{a}^2)$, or ${\tilde{x}}_N=x^{†}[1-{\prod }_{k=0}^{N-1}(1-{\tilde{\omega }}_k{a}^2)]$. The fixed-parameter updates are continuous and Borel by [F3], so this is an admissible finite point map. If all layer parameters equal $\omega$, the formula is the exact Landweber value in step 1.2; for other parameters it is the finite composition specified by [F2].

Choose the initialization $I_{\theta }(a,y)=0$, identity block $B_{{\theta }_0}((a,y),v)=v$, and layer parameter ${\tau }_0=\theta$. By [F1], this is an admissible one-step architecture and $F_{\theta }(a,y)=\theta a(y-a\cdot 0)=\theta ay$. This polynomial map is continuous and hence Borel by [F4], so it satisfies the learned point-rule interface [F2].

At the training point, $F_{\theta }(1,1)=\theta$. The law $P_{\mathcal{S}}$ assigns mass one to that point, so [F3] gives $L_{\mathcal{S}}(\theta )=R_{P_{\mathcal{S}}}(F_{\theta })=\mathrm{\mid }\theta -1{\mathrm{\mid }}^2$. It is zero exactly at $\theta =1$.

For that zero-loss parameter and any $t>1$, $F_1(t,t)=t^2$, so $r_t=\mathrm{\mid }t{t}^2-t\mathrm{\mid }=t(t^2-1)$.

The test law $Q_t$ assigns mass one to its displayed point, so [F3] gives $R_{Q_t}(1)=\mathrm{\mid }{t}^2-1{\mathrm{\mid }}^2=(t^2-1{)}^2$. Both $r_t$ and $R_{Q_t}(1)$ tend to $+\mathrm{\infty }$ as $t\to +\mathrm{\infty }$. For every finite $C$, choosing $t$ sufficiently large violates at least one claimed bound (in fact both), while the training risk remains zero by step 2.1. Therefore the refuted assertion is false.