TheoremStatement: Literature-sourcedProof: AI-generatedPipeline-generatedjudge pass (gpt-6-luna)

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

Binary logistic derivatives and the exact Newton/IRLS update

Statement

Let $n, q \geq 1$ be finite, let $X \in R^{n \times q}$ be a fixed design matrix whose first column is the intercept column of ones, and let $y \in {0, 1}^{n}$ . For finite $β \in R^{q}$ , put $η = X β$ , $p_{i} = σ (η_{i})$ , $W = diag (p_{i} (1 - p_{i}))$ , and $ℓ (β) = \sum_{i} [y_{i} η_{i} - \log (1 + e^{η_{i}})]$ . Then

\nabla ℓ (β) = X^{T} (y - p), \nabla^{2} ℓ (β) = - X^{T} W X .

The diagonal entries of $W$ are positive, so $- ℓ$ is convex; if $X$ has full column rank, $X^{T} W X$ is positive definite. In that full-rank case, the exact Newton step for maximizing the unpenalized log-likelihood is

β_{n e w} = β + (X^{T} W X)^{- 1} X^{T} (y - p) = (X^{T} W X)^{- 1} X^{T} W z, z = X β + W^{- 1} (y - p) .

This is the Bernoulli-logit IRLS step with the weighted least-squares convention using weights $p_{i} (1 - p_{i})$ . The identities describe one update and do not assert convergence of repeated updates.

Statement references

Binary logistic and multiclass softmax conditional models

Binary logistic log-likelihood and L1-penalized objective

A $C^{2}$ function is convex exactly when its Hessian is positive semidefinite

For a linear map $T : V \to W$ between finite-dimensional inner-product spaces, $x$ minimises $∥ T x - b ∥$ if and only if $T^{*} (T x - b) = 0$ , equivalently $T^{*} T x = T^{*} b$ ; minimisers exist and any two differ by an element of $\ker T$

Generalized linear model with explicit conditioning

Explicit exponential-family response contracts

General-link score and Fisher information

One minimum-norm IRLS step

IRLS equals the Fisher-scoring step when information is invertible

Hastie, Tibshirani, and Friedman, ESL, §§4.4.1 and 4.4.4, equations (4.20)–(4.31)

Stanford CS229 Spring 2022 lecture notes, §§2.1 and 3.2.2

Facts & Assumptions

Given: The finite fixed design, binary responses, finite current coefficient vector, binary conditional logistic model, and log-likelihood in the Statement.

[F1]

The binary conditional model has $\Pr (Y = 1 ∣ x) = σ (β_{0} + x^{T} β)$ , with $0 < σ (t) < 1$ for finite $t$ . Binary logistic and multiclass softmax conditional models

[F2]

The fixed-design log-likelihood is $ℓ (β) = \sum_{i} [y_{i} x_{i}^{T} β - \log (1 + e^{x_{i}^{T} β})]$ . Binary logistic log-likelihood and L1-penalized objective

[F3]

A Bernoulli GLM observation family has fixed support, finite second moments, and locally integrable first- and second-derivative envelopes when those moment identities are used. Explicit exponential-family response contracts

[F4]

For a canonical link, the natural parameter is the linear predictor; the GLM uses its specified common scale. Generalized linear model with explicit conditioning

[F5]

Under the stated GLM hypotheses, $U = X^{T} ({\dot{μ}}_{i} (y_{i} - μ_{i}) / (a V_{i}))_{i}$ and $I = X^{T} diag ({\dot{μ}}_{i}^{2} / (a V_{i})) X$ ; with positive weights and full column rank, $I$ is positive definite. General-link score and Fisher information

[F6]

The IRLS procedure uses $z_{i} = η_{i} + (y_{i} - μ_{i}) / {\dot{μ}}_{i}$ , $w_{i} = {\dot{μ}}_{i}^{2} / (a V_{i})$ and returns a weighted least-squares minimizer. One minimum-norm IRLS step

[F7]

Under the stated differentiability, common-support, moment and invertible-information hypotheses, the IRLS output equals $β + I^{- 1} U$ . IRLS equals the Fisher-scoring step when information is invertible

[F8]

A weighted least-squares minimizer satisfies the normal equation and is unique when the weighted design has trivial kernel. For a linear map $T : V \to W$ between finite-dimensional inner-product spaces, $x$ minimises $∥ T x - b ∥$ if and only if $T^{*} (T x - b) = 0$ , equivalently $T^{*} T x = T^{*} b$ ; minimisers exist and any two differ by an element of $\ker T$

[F9]

A $C^{2}$ function on an open convex Euclidean domain is convex when its Hessian is positive semidefinite everywhere. A $C^{2}$ function is convex exactly when its Hessian is positive semidefinite

Proof

1.1F1F2algebra

For one observation, $ℓ_{i} (η_{i}) = y_{i} η_{i} - \log (1 + e^{η_{i}})$ has derivative $y_{i} - σ (η_{i})$ and second derivative $- σ (η_{i}) (1 - σ (η_{i}))$ . Applying the coordinate chain rule to $η = X β$ gives $\nabla ℓ = X^{T} (y - p)$ and $\nabla^{2} ℓ = - \sum_{i} p_{i} (1 - p_{i}) x_{i} x_{i}^{T} = - X^{T} W X$ .

1.2F1F9algebra

Every finite $η_{i}$ gives $0 < p_{i} < 1$ , so $W$ is positive diagonal and $h^{T} X^{T} W X h = \sum_{i} p_{i} (1 - p_{i}) (x_{i}^{T} h)^{2} \geq 0$ . Therefore the Hessian of $- ℓ$ is positive semidefinite on $R^{q}$ , an open convex domain; [F9] gives convexity. If $X$ has full column rank, $h \neq 0$ implies $X h \neq 0$ , so at least one squared term is nonzero and the same sum is positive.

1.3F3F4givenalgebra

For Bernoulli data the mass is $\exp (y_{i} η_{i} - \log (1 + e^{η_{i}}))$ on ${0, 1}$ with counting measure, common support, $a = 1$ , canonical natural parameter $θ_{i} = η_{i}$ , $b (θ) = \log (1 + e^{θ})$ , $μ_{i} = p_{i}$ , $V_{i} = b^{''} (θ_{i}) = p_{i} (1 - p_{i})$ , and ${\dot{μ}}_{i} = p_{i} (1 - p_{i})$ . The response has finite second moment and bounded score; on every compact interval of finite $η_{i}$ the mass and its first two derivatives are bounded on the finite support, giving the stated local envelopes. Thus the response-family and canonical-link conditions [F3, F4] hold.

2.1F5F7step 1.1step 1.3algebra

By [F5], the score is $U = X^{T} (y - p)$ and the expected information is $I = X^{T} W X$ . Full column rank makes $I$ positive definite. The observed Hessian in step 1.1 is $- I$ , so the maximizing Newton formula $β - (\nabla^{2} ℓ)^{- 1} \nabla ℓ$ equals $β + I^{- 1} U$ . The generic IRLS result [F7] gives the same $β + I^{- 1} U$ update under these verified hypotheses.

3.1F6F8step 1.3step 2.1algebra∎

By [F6], the Bernoulli working response is $z = X β + W^{- 1} (y - p)$ and the weights are $W$ . The weighted normal equation [F8] and full rank give the unique IRLS fit $β_{n e w} = (X^{T} W X)^{- 1} X^{T} W z$ . Expanding $z$ yields $β + (X^{T} W X)^{- 1} X^{T} (y - p)$ , the Newton step from step 2.1.

Depends on

Used by

Nothing in the library uses this result yet.

Dependency tree · two levels

32 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Hastie, Tibshirani, and Friedman, The Elements of Statistical Learning, 2nd ed., corrected 12th printing (standard reference, not scraped)
Stanford CS229 Spring 2022 lecture notes (standard reference, not scraped)