LemmaStatement: Literature-sourcedProof: AI-generatedPipeline-generatedprecheck pass

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

Fletcher–Reeves descent and direction-growth bound

Statement

Let $g_{k} = \nabla f (x_{k}) \neq 0$ and let the Fletcher–Reeves directions be $d_{0} = - g_{0}, d_{k} = - g_{k} + β_{k - 1} d_{k - 1}, β_{k - 1} = \frac{∥ g_{k} ∥_{2}^{2}}{∥ g_{k - 1} ∥_{2}^{2}} (k \geq 1) .$ Suppose every accepted step along $d_{k - 1}$ satisfies the strong Wolfe curvature inequality with constant $0 < c_{2} < 1 / 2$ . Define $q_{k} = - \frac{g_{k}^{T} d_{k}}{∥ g_{k} ∥_{2}^{2}}, δ = \frac{1 - 2 c_{2}}{1 - c_{2}}, C_{3} = \frac{1 + c_{2}}{1 - c_{2}}, S_{k} = \sum_{j = 0}^{k} \frac{1}{∥ g_{j} ∥_{2}^{2}} .$ Then, for every $k$ , $δ \leq q_{k} \leq \frac{1}{1 - c_{2}}, ∥ d_{k} ∥_{2}^{2} \leq C_{3} ∥ g_{k} ∥_{2}^{4} S_{k} .$ In particular every direction is strictly descending, and its normalized directional derivative has a uniform positive lower bound.

Facts & Assumptions

Given: The nonzero gradients, Fletcher–Reeves recurrence, and strong Wolfe curvature hypotheses in the Statement.

[F1]

Fletcher–Reeves and safeguarded Polak–Ribière nonlinear conjugate gradients: FR uses $d_{0} = - g_{0}$ , $d_{k} = - g_{k} + β_{k - 1} d_{k - 1}$ , and $β_{k - 1} = ∥ g_{k} ∥_{2}^{2} / ∥ g_{k - 1} ∥_{2}^{2}$ .

[F2]

Wolfe and strong Wolfe line-search conditions: a strong Wolfe step along $d_{k - 1}$ satisfies $∣ g_{k}^{T} d_{k - 1} ∣ \leq c_{2} ∣ g_{k - 1}^{T} d_{k - 1} ∣$ , with $0 < c_{2} < 1 / 2$ .

Proof

Given: The hypotheses and notation in the Statement.

1.1givenF1algebra

From $d_{0} = - g_{0}$ , we have $q_{0} = 1$ .

1.2F1algebra

For $k \geq 1$ , the recurrence and the definition of $β_{k - 1}$ give $q_{k} = 1 - \frac{g_{k}^{T} d_{k - 1}}{∥ g_{k - 1} ∥_{2}^{2}} .$

2.1givenF2step 1.1step 1.2inductionalgebra

If $0 < q_{k - 1} \leq (1 - c_{2})^{- 1}$ , then [F2] and step 1.2 imply $1 - c_{2} q_{k - 1} \leq q_{k} \leq 1 + c_{2} q_{k - 1} .$ Induction from step 1.1 gives $q_{k} \leq \sum_{j = 0}^{k} c_{2}^{j} \leq (1 - c_{2})^{- 1}$ , and then $q_{k} \geq 1 - c_{2} / (1 - c_{2}) = δ > 0$ .

3.1F1F2step 2.1algebra

Expanding $d_{k} = - g_{k} + β_{k - 1} d_{k - 1}$ , applying [F2], and using step 2.1 yields $∥ d_{k} ∥_{2}^{2} \leq ∥ g_{k} ∥_{2}^{2} + 2 β_{k - 1} ∣ g_{k}^{T} d_{k - 1} ∣ + β_{k - 1}^{2} ∥ d_{k - 1} ∥_{2}^{2} \leq C_{3} ∥ g_{k} ∥_{2}^{2} + \frac{∥ g_{k} ∥_{2}^{4}}{∥ g_{k - 1} ∥_{2}^{4}} ∥ d_{k - 1} ∥_{2}^{2},$ because $∣ g_{k}^{T} d_{k - 1} ∣ \leq c_{2} q_{k - 1} ∥ g_{k - 1} ∥_{2}^{2}$ and $q_{k - 1} \leq (1 - c_{2})^{- 1}$ .

4.1step 3.1inductionalgebra∎

For $k = 0$ , $∥ d_{0} ∥_{2}^{2} = ∥ g_{0} ∥_{2}^{2} \leq C_{3} ∥ g_{0} ∥_{2}^{4} S_{0}$ . If the claimed bound holds at $k - 1$ , step 3.1 gives $∥ d_{k} ∥_{2}^{2} \leq C_{3} ∥ g_{k} ∥_{2}^{2} + C_{3} ∥ g_{k} ∥_{2}^{4} S_{k - 1} = C_{3} ∥ g_{k} ∥_{2}^{4} S_{k} .$ Thus induction proves the direction-growth bound.

Remarks

The first bound is on $- g_{k}^{T} d_{k} / ∥ g_{k} ∥_{2}^{2}$ , not on the cosine of the angle. It does not by itself give a uniform angle bound.

Depends on

Used by

Fletcher–Reeves has vanishing gradient liminf under strong Wolfe Theorem

Dependency tree · two levels

23 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Jorge Nocedal and Stephen J. Wright, Numerical Optimization, second edition (standard reference, not scraped)
Jean Charles Gilbert and Jorge Nocedal, Global Convergence Properties of Conjugate Gradient Methods for Optimization, SIAM Journal on Optimization 2(1), 21–42 (1992), DOI 10.1137/0802003 (standard reference, not scraped)