Bfgs curvature pairs and Hessian/inverse updates

Fix an integer $n\ge 1$. For successive points $x,x^{+}\in {\mathbb{R}}^n$ and vectors $g,g^{+}\in {\mathbb{R}}^n$, define the step vector $s:=x^{+}-x$ and the gradient-difference vector $y:=g^{+}-g$. Their curvature is $s^{T}y$, using the Euclidean inner product of The Euclidean inner product $⟨x,y⟩={\sum }_{k<n}{x}_k{y}_k$ on ${\mathbb{R}}^n$.

A real matrix $B\in {\mathbb{R}}^{n\times n}$ is symmetric positive definite (SPD) when $B=B^T$ and $z^{T}Bz>0$ for every nonzero $z\in {\mathbb{R}}^n$. For an SPD matrix $B$, write $H=B^{-1}$ for its inverse; $I$ denotes the identity matrix of Rectangular matrix multiplication and the identity matrix $I_n$, including zero-sized shapes. If $s\mathrm{\ne }0$ and $s^{T}y>0$, set $\rho :=(s^{T}y{)}^{-1}$. The BFGS Hessian update and its inverse form are $B^{+}:=B-\frac{Bs{s}^{T}B}{s^{T}Bs}+\frac{y{y}^T}{s^{T}y},H^{+}:=(I-\rho s{y}^T)H(I-\rho y{s}^T)+\rho s{s}^T\mathrm{.}$ The denominators are positive under the stated hypotheses.

For Powell damping, first set $c:=s^{T}Bs$. If $s^{T}y\ge 0.2c$, set $\tilde{y}:=y$. If $s^{T}y<0.2c$, set $\theta :=\frac{0.8c}{c-s^{T}y},\tilde{y}:=\theta y+(1-\theta )Bs\mathrm{.}$ The damped BFGS update is the same formula with $y$ replaced by $\tilde{y}$ and $\rho =(s^T\tilde{y}{)}^{-1}$.

The secant equations are $B^{+}s=y$ and $H^{+}y=s$ for an undamped pair; for a damped pair they use $\tilde{y}$. An inverse pair $(B,H)$ is consistent when $BH=HB=I$.

The two update formulas are algebraically equivalent when $B$ and $H$ are consistent. A positive curvature pair is required for both updates. Powell damping changes the gradient-difference vector used in the secant equation; a convergence theorem for the undamped update does not automatically cover the damped branch.