LemmaStatement: Literature-sourcedProof: AI-generatedPipeline-generatedprecheck pass

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

Pseudo-residuals are the negative gradient in sample-score space

Statement

Let $N \geq 1$ , let $(x_{i}, y_{i})_{i = 1}^{N}$ be a finite sample, let $f : X \to R$ , and let $U \subseteq R$ be open with $z_{i} = f (x_{i}) \in U$ for every $i$ . Suppose each scalar loss $t \mapsto ℓ (y_{i}, t)$ is defined on $U$ and differentiable at $z_{i}$ . Write $\partial_{2} ℓ (y_{i}, z_{i}) = {\frac{d}{d t} ℓ (y_{i}, t) ∣}_{t = z_{i}}$ and define $Φ : U^{N} \to R$ by $Φ (w) = \sum_{i = 1}^{N} ℓ (y_{i}, w_{i})$ . Then $Φ$ is totally differentiable at $z$ , its derivative in a score increment $v = (v_{1}, \dots, v_{N})$ is

$D Φ (z) v = \sum_{i = 1}^{N} \partial_{2} ℓ (y_{i}, z_{i}) v_{i},$

and its Euclidean gradient is $\nabla Φ (z) = (\partial_{2} ℓ (y_{i}, z_{i}))_{i = 1}^{N}$ . For these possibly local losses, define the pseudo-residual by $r_{i} (f) = - \partial_{2} ℓ (y_{i}, z_{i})$ ; this agrees with the usual definition when the loss is finite-valued on all of $Y \times R$ . Then the pseudo-residual vector associated with $f$ on this sample is exactly $- \nabla Φ (z)$ in sample-score space.

Statement references

Trevor Hastie, Robert Tibshirani, and Jerome Friedman, The Elements of Statistical Learning, 2nd ed., §§10.10.1–10.10.3, equations (10.35)–(10.37), printed pp. 358–361

Alexey Natekin and Alois Knoll, “Gradient boosting machines, a tutorial,” §§2.3–2.4

Facts & Assumptions

Given: Sample scores $z_{i} = f (x_{i})$ in an open scalar domain $U$ and scalar losses differentiable at those scores.

[F1]

Negative-gradient pseudo-residuals for a finite empirical loss: For a finite-valued loss on $Y \times R$ whose sampled sections are differentiable at the scores, the empirical score loss is $Φ (z) = \sum_{i} ℓ (y_{i}, z_{i})$ , and the pseudo-residual is $r_{i} = - \partial_{2} ℓ (y_{i}, z_{i})$ .

[F2]

The total (Fréchet) derivative $D f (a)$ as the linear first-order approximation with $o (∥ h ∥_{2})$ remainder: Total differentiability at a point means a linear first-order approximation whose remainder divided by the increment norm tends to zero.

[F3]

The chain rule for total derivatives: $D (g \circ f) (a) = D g (f (a)) \circ D f (a)$ : If $f$ is totally differentiable at $a$ and $g$ at $f (a)$ , then $g \circ f$ is totally differentiable and $D (g \circ f) (a) = D g (f (a)) \circ D f (a)$ .

[F4]

Sums and scalar multiples of totally differentiable maps are totally differentiable with the expected derivatives: Finite sums of totally differentiable maps are totally differentiable and their derivatives add.

[F5]

Directional derivatives and partial derivatives of a map $U \subseteq R^{m} \to R^{n}$ : A coordinate partial derivative is the directional derivative along the corresponding standard basis direction.

[F6]

A total derivative computes every directional derivative, and its matrix is the Jacobian: When a total derivative exists, its value on each coordinate direction is the corresponding partial derivative.

[F7]

The Jacobian matrix of partial derivatives and the gradient in the scalar-valued case: For a scalar-valued function, the Euclidean gradient is the vector of coordinate partial derivatives in standard coordinate order.

Proof

1.1givenF2construct

For each $i$ , let $π_{i} (z) = z_{i}$ and $g_{i} (t) = ℓ (y_{i}, t)$ . The coordinate projection has derivative $D π_{i} (z) v = v_{i}$ because its linear approximation has zero remainder. Differentiability of $g_{i}$ at $z_{i}$ gives $g_{i} (z_{i} + u) = g_{i} (z_{i}) + \partial_{2} ℓ (y_{i}, z_{i}) u + o (∣ u ∣)$ , so $D g_{i} (z_{i}) u = \partial_{2} ℓ (y_{i}, z_{i}) u$ .

2.1F3step 1.1

By the chain rule, $ψ_{i} = g_{i} \circ π_{i}$ is totally differentiable and $D ψ_{i} (z) v = \partial_{2} ℓ (y_{i}, z_{i}) v_{i}$ .

3.1F4step 2.1induction

Since $Φ = \sum_{i = 1}^{N} ψ_{i}$ , the finite-sum rule gives $D Φ (z) v = \sum_{i = 1}^{N} \partial_{2} ℓ (y_{i}, z_{i}) v_{i}$ .

4.1givenF1F5F6F7step 3.1construct∎

Evaluating step 3.1 on the coordinate direction that changes only $z_{i}$ gives $\partial_{z_{i}} Φ (z) = \partial_{2} ℓ (y_{i}, z_{i})$ by [F5] and [F6], so [F7] gives the asserted gradient. Extend the sampled loss sections by zero outside $U$ to obtain a finite-valued loss on ${y_{1}, \dots, y_{N}} \times R$ . Since $U$ is open, this extension agrees with each original section near $z_{i}$ and has the same derivative there. Thus [F1] applied to the extension yields exactly the locally defined $r_{i} (f)$ , independently of the values outside $U$ . Consequently $(r_{i} (f))_{i = 1}^{N} = - \nabla Φ (z)$ , also agreeing with [F1] whenever the original loss is globally defined.

Depends on

Used by

Dependency tree · two levels

16 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Trevor Hastie, Robert Tibshirani, and Jerome Friedman, The Elements of Statistical Learning, 2nd ed., 12th printing (standard reference, not scraped)
Alexey Natekin and Alois Knoll, "Gradient boosting machines, a tutorial," Frontiers in Neurorobotics 7 (2013), Article 21 (standard reference, not scraped)