How statement and proof provenance work
The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.
- Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
- AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
- AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.
These labels describe origin, not correctness: citations and verification chips remain separate evidence.
S4 convolution-filter arithmetic and numerical cost
Statement
Let , let be fixed, and let with diagonal , , and , where denotes conjugate transpose. Let , assume is invertible, and choose to be the least power of two with . For the length- filter procedure, take the transformed output vector as supplied input, where . Assume that every root-grid diagonal resolvent and low-rank correction used by the procedure is invertible. In the exact complex arithmetic model, each scalar addition, multiplication, division, and zero test has unit cost. The length- filter is . The procedure returns the first coefficients of this filter using at most roots-of-unity Cauchy products, operations, and working space. The operation count is for filter evaluation from ; it does not include forming that vector from . This bound applies in particular when the fixed matrices of the HiPPO low-rank theorem are used as time-independent state matrices, in their specified equivalent DPLR coordinates and subject to the same invertibility hypotheses. For LegS this means using the constant coefficient matrix ; it does not assert a convolution representation for the original time-dependent LegS dynamics with coefficient .
For numerical evaluation, put , , and at roots , where . Define the scaled blocks , , , , and . Matrix infinity norm means maximum absolute row sum; for a column it is maximum absolute entry. Suppose certified constants bound, over these roots, by , and by . For each scalar scaled-block Cauchy sum, require a certified pole/root separation and a weight bound with ; the finite constant contributions from and the direct value at are formed separately. Empty sums need no separation certificate. Set .
For , a numerical implementation returns with using scalar operations and scalar words. On families with bounded , this is . The numerical model uses variable-precision scalar operations, unrestricted exponent range and supplied certified input/root approximations; each operation and word access has unit cost. Word lengths and sufficient precision depend on separation, correction conditioning, intermediate magnitudes and accuracy. The finite circuit precision certificate in the proof is required; these counts exclude bit cost, computing certificates, and forming . They imply no complete training-cost or long-context-accuracy bound.
The numerical procedure also receives the exact index partition and , where , so it can select the exceptional constant terms without a finite-precision equality test.
Statement references
Pan, “Fast Approximate Computations with Cauchy Matrices and Polynomials,” §§3.3, 6.1–6.4, 7.1
HiPPO LegS, LegT, and LagT matrices have normal-plus-low-rank form after specified similarities
DPLR SSM filter values reduce to Cauchy products
Exact roots-of-unity Cauchy matvec has quasi-linear arithmetic cost
Numerical Cauchy products on a root grid with explicit accuracy scale
Facts & Assumptions
Given: The finite DPLR parameters, fixed rank, , the least power of two at least , the supplied transformed output vector, and the bilinear and root-grid invertibility hypotheses and, for the numerical procedure, the supplied exact partition of diagonal entries equal to and those unequal to it.
S4 state-space recurrence and convolution procedure: with supplied , the filter procedure evaluates the truncated generating function at the th roots, handles by its finite value, applies the inverse radix-two transform, and returns the first coefficients.
DPLR SSM filter values reduce to Cauchy products: each root value is given by the DPLR resolvent formula, the value is , and the four matrix-vector families reduce to scalar Cauchy products.
Exact roots-of-unity Cauchy matvec has quasi-linear arithmetic cost: for poles and an -root grid with no collisions, the exact Cauchy product costs operations and space.
Roots-of-unity orthogonality and radix-two FFT cost: the radix-two DFT and inverse cost operations and space, and inverse transformation recovers the polynomial coefficients exactly.
HiPPO LegS, LegT, and LagT matrices have normal-plus-low-rank form after specified similarities: the fixed LegS coefficient matrix and the LegT and LagT generators, after the specified state-coordinate changes, admit DPLR form with rank at most two. The original LegS coefficient ODE has the additional time-dependent factor .
Numerical Cauchy products on a root grid with explicit accuracy scale: a root-grid Cauchy sum with certified scale has absolute error in scalar operations and linear word space. Its arc-tree truncation and variable-precision circuit estimates are proved locally, including root/input approximation errors and certified routing.
Proof
By [F5], each fixed matrix admits equivalent DPLR coordinates of bounded rank. When such a matrix is used as a constant state matrix and satisfies the stated invertibility hypotheses, it is an admissible input here. This does not identify the original time-dependent LegS ODE with a constant-matrix model. The operation argument applies to any supplied DPLR matrices of the same dimensions. By [F2], each required root value has the stated Cauchy-product expression, including the direct value at . Under the noncollision and correction-invertibility assumptions, the filter procedure in [F1] therefore computes all generating values exactly.
For , . Substitution in [F2] gives and . Also . The pole mapping in [F2] expresses each entry of as for , and as the constant otherwise. Thus the weights in [F6] are exactly for the applicable vector pair . Their preparation and the exceptional constant sums cost for fixed . At , retain the direct value from [F2], without a resolvent division.
Each scalar product has at most poles and is evaluated on roots. If it has at least one pole, [F3] bounds its cost by operations and space; if it has no poles, its constant value is formed by a single finite sum in operations and space. There are at most such products by [F2], a constant number because is fixed. The per-root correction matrix products and inversions cost operations by finite-dimensional elimination, which is for fixed .
The generating values are the DFT of the coefficients for . By [F4], the inverse radix-two transform recovers these coefficients exactly; [F1] then returns their first entries, which are precisely the length- filter.
Let and first consider exact arithmetic on truncated Cauchy sums having scalar entry errors at most , with finite constant sums exact. Denote the resulting blocks by . Each of their infinity-norm errors is at most , since each row has at most entries and . Hence , and . Writing , the geometric series converges in infinity norm because , is the inverse of by finite geometric identities and passage to the norm limit, and has norm at most . Therefore . The identity gives .
Since is the least power of two at least , for every integer . Combining the Cauchy products, per-root corrections, and inverse transform gives exact complex arithmetic operations. Processing the Cauchy products sequentially uses product-tree workspace; their root-value arrays, the correction matrices, and the inverse transform use additional space. Fixed and give total working space. The proof counts the supplied as input and does not count its formation from .
Expanding the difference of the two triple products as and using step 2.3 bounds its modulus by . Adding the error in and using bounds each generating-value error by . The direct value has no truncation error. The exact inverse DFT averages unit-modulus multiples in each output coordinate, so its infinity operator norm is at most one; consequently the exact inverse transform of these approximations has coefficient error at most .
Apply [F6] with absolute tolerance to each scalar sum; its exact truncated circuit has error at most , as required above. Although [F6] is stated on , the filter grid is ; run each numerical Cauchy product on the positive grid and permute its outputs to the filter order, in operations and words, preserving the maximum error and pole/root separation bounds. It uses terms. There are at most sums, processed sequentially; their arrays and the fixed-size blocks use words. The per-root inverses can be computed by the adjugate/determinant formula, which is constant work for fixed and has nonzero determinants by step 2.3. Add the radix-two inverse transform and finite sums: the scalar count is . The replacement gives the stated count and space.
To include every rounding error, regard the entire selected computation, from supplied DPLR parameters, transformed vector and root/arc constants through weight formation, Cauchy sums, finite constants, adjugate inverses and inverse FFT, as one finite rational circuit. Include conjugations as exact sign changes. Require certified with an upper bound on its gates, an upper bound on exact intermediate moduli, and a lower bound on every exact divisor modulus. Such bounds exist for each input because step 2.3 proves all truncated correction inverses exist and all other divisions are nonzero by the hypotheses. Put and select input and gate accuracy , also sufficient for the certified overlapping routing tests of [F6]. The gate recurrence proved in [F6] then gives final additional error at most , including errors in supplied parameters, roots, weights, , correction inverses and twiddle factors. Together with step 3.2 this is less than . No precision selection changes the scalar count or scalar-word count, although their bit cost can grow. Bounded gives ; arbitrary conditioning retains the explicit term.
Remarks
S4 Appendix C.3, Proposition 5 reports numerical operations and linear space, without specifying the accuracy norm, word length or conditioning margins. The theorem above preserves that numerical asymptotic on explicitly bounded-conditioning families and gives an absolute-error bound with the conditioning factor exposed for general certified inputs. It does not assert a uniform fixed-precision or bit-complexity interpretation of the reported claim. Arbitrarily near pole/root collisions require growing input precision; nearly singular Woodbury corrections amplify block errors. Neither qualification concerns training or empirical long-context performance.
Depends on
- DPLR structured state-space parameterization
- S4 state-space recurrence and convolution procedure
- HiPPO LegS, LegT, and LagT matrices have normal-plus-low-rank form after specified similarities
- Roots-of-unity orthogonality and radix-two FFT cost
- Exact roots-of-unity Cauchy matvec has quasi-linear arithmetic cost
- Numerical Cauchy products on a root grid with explicit accuracy scale
- DPLR SSM filter values reduce to Cauchy products
Used by
Dependency tree · two levels
12 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.
Sources
- Gu, Goel, and Ré, Efficiently Modeling Long Sequences with Structured State Spaces, Theorem 3 and Appendix C.3 (standard reference, not scraped)
- Pan, Fast Approximate Computations with Cauchy Matrices and Polynomials, §§3.3, 6.1–6.4, 7.1 (standard reference, not scraped)