TheoremStatement: Literature-sourcedProof: AI-generatedPipeline-generatedjudge pass (gpt-6-luna)

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

Complete separation gives an unattained logistic likelihood supremum

Statement

Let $n, q \geq 1$ be finite, let $x_{i} \in R^{q}$ be a fixed design row with an intercept coordinate when one is used, and let $y_{i} \in {0, 1}$ . Suppose the unpenalized binary logistic log-likelihood is $ℓ (β) = \sum_{i} [y_{i} x_{i}^{T} β - \log (1 + e^{x_{i}^{T} β})]$ . If there is a vector $v \in R^{q}$ such that $(2 y_{i} - 1) x_{i}^{T} v > 0$ for every $i$ , then $\sup_{β \in R^{q}} ℓ (β) = 0$ , and no finite coefficient vector attains this supremum.

Statement references

Binary logistic and multiclass softmax conditional models

Binary logistic log-likelihood and L1-penalized objective

Hastie, Tibshirani, and Friedman, ESL, §4.4.5 and Exercise 4.5

Facts & Assumptions

Given: A finite fixed binary logistic data set with $n \geq 1$ , $y_{i} \in {0, 1}$ , the unpenalized log-likelihood, and a strict separating vector $v$ satisfying the signed-margin inequalities in the Statement.

[F1]

The log-likelihood is the sum of the Bernoulli terms $y_{i} η_{i} - \log (1 + e^{η_{i}})$ with $η_{i} = x_{i}^{T} β$ . Binary logistic log-likelihood and L1-penalized objective

[F2]

At every finite coefficient vector, the binary conditional probability is strictly between zero and one. Binary logistic and multiclass softmax conditional models

Proof

1.1F1givenconstructalgebra

Put $β (t) = t v$ for $t > 0$ . If $y_{i} = 1$ , the separation condition gives $a_{i} = x_{i}^{T} v > 0$ and the $i$ th log-likelihood term is $- \log (1 + e^{- t a_{i}}) \to 0$ . If $y_{i} = 0$ , the condition gives $a_{i} = x_{i}^{T} v < 0$ and the term is $- \log (1 + e^{t a_{i}}) \to 0$ . Since the sample is finite, $ℓ (β (t)) \to 0$ , so the supremum is at least zero.

2.1F2givenstep 1.1algebra∎

For any finite $β$ , each $η_{i}$ is finite and [F2] gives $0 < p_{i} < 1$ . The observed Bernoulli log-probability is $\log p_{i} < 0$ when $y_{i} = 1$ and $\log (1 - p_{i}) < 0$ when $y_{i} = 0$ . As $n \geq 1$ , $ℓ (β) < 0$ for every finite $β$ . Thus the supremum is at most zero and is attained by no finite coefficient vector; step 1.1 shows that it equals zero.

Depends on

Used by

A strictly separated two-point sample has an unattained logistic likelihood supremum Example

Dependency tree · two levels

3 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Hastie, Tibshirani, and Friedman, The Elements of Statistical Learning, 2nd ed., corrected 12th printing (standard reference, not scraped)
Stanford CS229 Spring 2022 lecture notes (standard reference, not scraped)