Backpropagation through a two-layer network with a shared weight

Given: The two-layer scalar network, the tied layer weight $\theta$, input $x$, identity activations, zero biases, and loss $J=\frac{1}{2}{y}^2$.

Finite feedforward computation graphs and layered MLPs: a layered perceptron evaluates a finite acyclic graph by applying each vertex's activation to its bias plus the weighted sum of its parents; output vertices may use the identity activation.

Backpropagation through a finite shared-parameter recurrence: for a repeated transition using the same $\theta$, reverse mode adds each transition's parameter pullback to a single parameter-gradient accumulator.

Reverse accumulation sums gradients over shared parameter uses: the reverse recurrence returns the loss's direct parameter derivative plus all transition-use contributions and the initial-state contribution.

Chain sum product and composition rules for Banach derivatives: on the real Banach space $\mathbb{R}$, multiplication $(a,b)\mapsto ab$ is bounded bilinear because $\mathrm{\mid }ab\mathrm{\mid }\le \mathrm{\mid }a\mathrm{\mid }\mathrm{\mid }b\mathrm{\mid }$. The coordinate projections on ${\mathbb{R}}^2$ are bounded linear, hence Fréchet differentiable. Applying the product rule to the projections of $(\theta ,h)$ gives $D_{\theta }{\mathrm{\Phi }}_t=h_{t-1}$ and $D_h{\mathrm{\Phi }}_t=\theta$; applying it to $\mathrm{\ell }(\theta ,z)=\frac{1}{2}{z}^2$ gives $D_{\theta }\mathrm{\ell }=0$ and $D_z\mathrm{\ell }=z$. These derivative factors are continuous, so the transitions and loss satisfy the recurrence's $C^1$ premise.