A univariate ReLU network with H hidden layers has a product bound on affine intervals

Before any hidden layer, the input is affine on $I$. Suppose all values through layer $\mathrm{\ell }-1$ are affine on each of $M$ subintervals. On any one such interval, every preactivation in layer $\mathrm{\ell }$ is affine, and each has at most one zero unless it is identically zero. The at most $n_{\mathrm{\ell }}$ zeros divide that interval into at most $n_{\mathrm{\ell }}+1$ subintervals on each of which all ReLU outputs are affine.

Induction over the hidden layers gives at most ${\prod }_{\mathrm{\ell }=1}^H(n_{\mathrm{\ell }}+1)$ intervals; the affine output adds no breakpoints. For $H\ge 1$, AM–GM gives $\prod (n_{\mathrm{\ell }}+1)\le (1+H^{-1}\sum n_{\mathrm{\ell }}{)}^H=(1+U\mathrm{/}H{)}^H$. With no hidden layer the output is affine on the entire interval.