LemmaStatement: AI-adaptedProof: AI-adaptedPipeline-generated

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

LMNN hinge subgradient formula and a uniform data-dependent bound

Statement

Fix the finite labeled sample, fixed target sets, and $μ \in [0, 1]$ of the LMNN objective. For each $j \in N_{i}$ and each impostor index $l$ with $y_{l} \neq y_{i}$ , set $C_{i j} = (x_{i} - x_{j}) (x_{i} - x_{j})^{T}$ , $D_{i j l} = C_{i j} - C_{i l}$ , and $h_{i j l} (M) = 1 + ⟨ M, D_{i j l} ⟩_{F}$ , where $⟨ A, B ⟩_{F} = \sum_{r, s} A_{r s} B_{r s}$ . Define $γ_{i j l} (M) = 1$ if $h_{i j l} (M) > 0$ and $γ_{i j l} (M) = 0$ if $h_{i j l} (M) \leq 0$ . Put

$G (M) = (1 - μ) \sum_{i} \sum_{j \in N_{i}} C_{i j} + μ \sum_{i} \sum_{j \in N_{i}} \sum_{l : y_{l} \neq y_{i}} γ_{i j l} (M) D_{i j l} .$

For the convex objective $F_{μ}$ on $S_{+}^{d}$ , a Frobenius subgradient at $M$ means a symmetric matrix $G$ such that $F_{μ} (N) \geq F_{μ} (M) + ⟨ G, N - M ⟩_{F}$ for every $N \in S_{+}^{d}$ . Then $G (M)$ is a subgradient at every $M \in S_{+}^{d}$ . Moreover, uniformly in $M$ ,

$∥ G (M) ∥_{F} \leq B_{d a t a} : = (1 - μ) \sum_{i} \sum_{j \in N_{i}} ∥ x_{i} - x_{j} ∥_{2}^{2} + μ \sum_{i} \sum_{j \in N_{i}} \sum_{l : y_{l} \neq y_{i}} (∥ x_{i} - x_{j} ∥_{2}^{2} + ∥ x_{i} - x_{l} ∥_{2}^{2}) .$

At an exact hinge kink $h_{i j l} (M) = 0$ , the selected subgradient coefficient is $0$ .

Statement references

Fixed-target pull/push objective

Convexity on the PSD cone

Subgradient inequality

Frobenius norm and its triangle inequality

Weinberger and Saul, §3.2, printed pp. 216–218, and Appendix A.1, pp. 238–239

Bubeck, §3.1, printed pp. 262–266

Bertsekas, MIT 6.253 Lecture 15, printed pp. 200–205

Facts & Assumptions

Given: The finite sample, fixed target sets and weight $μ$ in the Statement, and $M \in S_{+}^{d}$ .

[F1]

The objective is the finite weighted sum of linear pull terms and positive-part hinge terms with affine arguments (Fixed-target large-margin nearest-neighbor objective).

[F2]

The objective is convex on $S_{+}^{d}$ (The fixed-target LMNN objective is convex).

[F3]

A subgradient is a vector whose inner product gives a global supporting affine lower bound on the convex function (Subgradients and the subdifferential of a convex function).

[F4]

The Frobenius norm is the Euclidean entry norm and its triangle inequality follows from the inner-product norm theorem (The Frobenius norm $∥ A ∥_{F} = (\sum_{i, j} ∣ a_{i j} ∣^{2})^{1 / 2}$ on real or complex matrices, The inner-product norm is definite, homogeneous, and satisfies the triangle inequality).

[F5]

The Euclidean norm on $R^{d}$ satisfies $∥ a ∥_{2}^{2} = \sum_{r} a_{r}^{2}$ (The Euclidean inner product $⟨ x, y ⟩ = \sum_{k < n} x_{k} y_{k}$ on $R^{n}$ ).

Proof

1.1F1algebra

For $a_{i j} = x_{i} - x_{j}$ , the entrywise inner product satisfies $⟨ M, C_{i j} ⟩_{F} = \sum_{r, s} M_{r s} (a_{i j})_{r} (a_{i j})_{s} = a_{i j}^{T} M a_{i j} = q_{M} (x_{i}, x_{j})$ . Thus each hinge has the form $ϕ (M) = \max {0, 1 + ⟨ M, D_{i j l} ⟩_{F}}$ , with affine argument $h_{i j l} (M)$ .

1.2F4F5algebra

For any $a \in R^{d}$ , $∥ a a^{T} ∥_{F}^{2} = \sum_{r, s} (a_{r} a_{s})^{2} = (\sum_{r} a_{r}^{2})^{2}$ , so $∥ C_{i j} ∥_{F} = ∥ x_{i} - x_{j} ∥_{2}^{2}$ by [F5]. By [F4], $∥ D_{i j l} ∥_{F} \leq ∥ C_{i j} ∥_{F} + ∥ C_{i l} ∥_{F} = ∥ x_{i} - x_{j} ∥_{2}^{2} + ∥ x_{i} - x_{l} ∥_{2}^{2}$ .

2.1step 1.1algebra

If $h_{i j l} (M) > 0$ , then $ϕ (M) = h_{i j l} (M)$ and $ϕ (N) \geq h_{i j l} (N) = ϕ (M) + ⟨ D_{i j l}, N - M ⟩_{F}$ for every $N$ . If $h_{i j l} (M) \leq 0$ , then $ϕ (M) = 0$ and $ϕ (N) \geq 0 = ϕ (M) + ⟨ 0, N - M ⟩_{F}$ for every $N$ . Hence $γ_{i j l} (M) D_{i j l}$ is a subgradient of this hinge, including the selected zero coefficient at equality.

3.1step 1.1step 2.1F2F3algebra

Each $C_{i j}$ and $D_{i j l}$ is symmetric, so their weighted sum $G (M)$ is symmetric. Each pull term is linear with Frobenius subgradient $C_{i j}$ . Adding its supporting equality to the hinge supporting inequalities in step 2.1, weighted by the nonnegative coefficients $1 - μ$ and $μ$ , proves $F_{μ} (N) \geq F_{μ} (M) + ⟨ G (M), N - M ⟩_{F}$ for every $N \in S_{+}^{d}$ . Thus $G (M)$ is a subgradient by [F3].

4.1step 1.2givenF4algebra∎

Since each $γ_{i j l} (M) \in {0, 1}$ , the triangle inequality and step 1.2 give $∥ G (M) ∥_{F} \leq (1 - μ) \sum_{i} \sum_{j \in N_{i}} ∥ C_{i j} ∥_{F} + μ \sum_{i} \sum_{j \in N_{i}} \sum_{l : y_{l} \neq y_{i}} ∥ D_{i j l} ∥_{F} \leq B_{d a t a}$ . The bound is finite because the sample and all index sets are finite, and it is independent of $M$ .

Depends on

Used by

Dependency tree · two levels

41 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Kilian Q. Weinberger and Lawrence K. Saul, Distance Metric Learning for Large Margin Nearest Neighbor Classification, JMLR 10 (2009), 207–244 (standard reference, not scraped)
Sébastien Bubeck, Convex Optimization: Algorithms and Complexity, Foundations and Trends in Machine Learning 8(3–4) (2015) (standard reference, not scraped)
Dimitri P. Bertsekas, Convex Analysis and Optimization, MIT 6.253 Complete Lecture Notes (standard reference, not scraped)