Rank restrictions and empirical transfer claims

Low-rank adaptation constrains the update of one linear layer to rank at most $r$ and uses $r(d+k)$ trainable scalar coordinates for the two factors, excluding the frozen base matrix and any optional biases. These algebraic facts do not imply that a downstream task has an optimal or effective update of rank at most $r$, that a finite-budget training procedure reaches an optimum, that transfer performance improves, or that low rank prevents catastrophic forgetting. Empirical rank, quality, throughput, memory, and latency findings for a particular model, data set, and hardware configuration remain empirical observations under that setup; they are not consequences of the factorization or gradient formulas. Merging the factors has an exact real-arithmetic output identity, but this does not assert bitwise equality, floating-point accuracy, adapter-switching latency, storage savings, or hardware speed.

Rank capacity and factorization ambiguity of a low-rank adapter: rank capacity and factorization symmetry proved for finite real matrices.

Exact merging and gradients for a low-rank adapter: exact real-arithmetic merge and factor-gradient formulas.

Hu et al., §§4–5: LoRA parameterization and source-specific empirical context