Finite unrolling alone gives no regularization convergence or risk guarantee

Fix an admissible $(A,y)$. By the definition of the update maps, $x_1=U_{{\theta }_0}^{A,y}(I_{\theta }(A,y))$. If $x_k=U_{{\theta }_{k-1}}^{A,y}\circ \cdots \circ U_{{\theta }_0}^{A,y}(I_{\theta }(A,y))$, then applying the next defined update gives $x_{k+1}=U_{{\theta }_k}^{A,y}\circ U_{{\theta }_{k-1}}^{A,y}\circ \cdots \circ U_{{\theta }_0}^{A,y}(I_{\theta }(A,y))$. Induction through $k=N-1$ yields the stated finite composition. Its Borel point-rule convention is the interface in [F3].

For the repeated scalar update $T(x)=2x$, induction gives $T^0(1)=1$ and $T^{k+1}(1)=2T^k(1)=2^{k+1}$ whenever $T^k(1)=2^k$. Thus every finite prefix is defined, but $T^k(1)=2^k\to +\mathrm{\infty }$ as $k\to \mathrm{\infty }$. Finite evaluability alone therefore does not imply convergence of an infinite continuation.

Under the differentiability and strict-branch assumptions in the Given clause, [F2] forms, for each output coordinate, the finite graph of the $K$-step recurrence and calls forward mode in each parameter-coordinate direction. By [F6], every returned tangent is the corresponding directional derivative of that finite scalar-output graph. Assembling these entries gives the Jacobian of the finite recurrence, not the derivative of a limiting fixed point.

To check the zero-residual boundary, fix $q\in (0,1)$ and set $G(z,\theta )=qz+(1-q)\theta$, ${\theta }_0=0$, and $z_0(\theta )=0$. At ${\theta }_0$, the recurrence gives $z_k=0$ for every finite $k$, so the fixed-point residual $z_k-G(z_k,{\theta }_0)$ is zero. The exact fixed point satisfies $z_{\ast }(\theta )=\theta$, and therefore $\frac{d{z}_{\ast }}{d\theta }(0)=1$.

Use the scalar architecture with declared domain containing $(A,y)=(1,1)$ and every $(t,t)$ for $t>1$, zero initialization, identity update block, and one layer parameter ${\tau }_0=\theta$. Choose the deployed parameter $\theta =1$. Its one-layer formula is ${\mathrm{\Phi }}_{1,1}(a,y)=ay$, so ${\mathrm{\Phi }}_{1,1}(1,1)=1$.

The map $(a,y)\mapsto ay$ is continuous and hence Borel by [F5], so it is a point reconstruction rule by [F3]. On the singleton training law $P_{\mathcal{S}}={\delta }_{((1,1),1)}$, [F4] gives $L_{\mathcal{S}}(1)=R_{P_{\mathcal{S}}}({\mathrm{\Phi }}_{1,1})=\mathrm{\mid }1-1{\mathrm{\mid }}^2=0$.

Differentiating the finite recurrence gives $e_0=0$ and $e_{k+1}=q{e}_k+(1-q)$. Induction yields $e_K=1-q^K$ for each $K\ge 1$. Since $0<q^K<1$, $e_K\mathrm{\ne }1$ although the finite residual in step 1.4 is zero. Thus even contraction and zero finite residual do not identify the finite-map derivative with the fixed-point derivative.

For any shifted scalar operator $A=t>1$ with $y=t$ and $x^{†}=1$, the same learned parameter gives ${\mathrm{\Phi }}_{1,1}(t,t)=t^2$. Its forward residual is $\mathrm{\mid }t{\mathrm{\Phi }}_{1,1}(t,t)-t\mathrm{\mid }=t(t^2-1)$, and [F4] gives squared risk $(t^2-1{)}^2$ under the test law $Q_t={\delta }_{((t,t),1)}$. Both tend to $+\mathrm{\infty }$ as $t\to +\mathrm{\infty }$, while the training-law risk from step 2.1 remains zero. Given any $M>0$, choosing $t$ sufficiently large makes both quantities exceed $M$, proving the claimed absence of a uniform residual or test-risk bound from training risk alone.