Bfgs updates preserve the secant equations, symmetry, and positive definiteness

Put $c:=s^{T}Bs>0$. The matrix $B^{+}$ is symmetric. For any $z$, the first two terms of $z^T{B}^{+}z$ satisfy $z^{T}Bz-\frac{(z^{T}Bs{)}^2}{c}={\min }_{\lambda \in \mathbb{R}}(z-\lambda s{)}^{T}B(z-\lambda s)\ge 0,$ because completing the square gives the minimum at $\lambda =(s^{T}Bz)\mathrm{/}c$. Equality holds exactly when $z$ is a scalar multiple of $s$. If $z\mathrm{\ne }0$ is such a multiple, the last term $(z^{T}y{)}^2\mathrm{/}d$ is strictly positive; if it is not a multiple, the displayed minimum is strictly positive. Hence $B^{+}$ is SPD.

The matrix $H^{+}$ is symmetric. For any $z$, put $w:=z-(s^{T}z)y\mathrm{/}d$. Then $z^T{H}^{+}z=w^{T}Hw+\frac{(s^{T}z{)}^2}{d}\mathrm{.}$ If $s^{T}z\mathrm{\ne }0$, the second term is positive. If $s^{T}z=0$ and $z\mathrm{\ne }0$, then $w=z$ and $z^{T}Hz>0$. Thus $H^{+}$ is SPD.

Substitution of $s$ into the Hessian update gives $B^{+}s=Bs-Bs(s^{T}Bs)\mathrm{/}c+y(y^{T}s)\mathrm{/}d=y$. In the inverse update, $(I-y{s}^T\mathrm{/}d)y=0$, so $H^{+}y=s{s}^{T}y\mathrm{/}d=s$.

Let $u:=Bs$ and $A:=B-u{u}^T\mathrm{/}c=B^{+}-y{y}^T\mathrm{/}d$. Since $u^{T}H=s^T$, we have $AH=I-u{s}^T\mathrm{/}c$. Using $B^{+}s=y$ from step 1.3, $B^{+}{H}^{+}=(B^{+}-y{y}^T\mathrm{/}d)H(I-y{s}^T\mathrm{/}d)+(1\mathrm{/}d)y{s}^T=(I-u{s}^T\mathrm{/}c)(I-y{s}^T\mathrm{/}d)+(1\mathrm{/}d)y{s}^T=I,$ because $s^{T}y=d$. Thus $H^{+}=(B^{+}{)}^{-1}$, since $B^{+}$ is SPD by step 1.1.