Equality and inequality augmented Lagrangians and multiplier updates

Use the $C^2$ program, dimensions, gradient conventions, and Lagrangian $L(x,\nu ,\lambda )=f(x)+{\nu }^{T}h(x)+{\lambda }^{T}g(x)$ of Inequality constrained programs, tangent cones, constraint qualifications, and KKT. Let $\rho >0$, $\nu \in {\mathbb{R}}^p$, and $\lambda \in {\mathbb{R}}_{+}^m$. The equality augmented Lagrangian is ${\mathcal{L}}_{\rho }^E(x,\nu )=f(x)+{\nu }^{T}h(x)+\frac{\rho }{2}\mathrm{\parallel }h(x){\mathrm{\parallel }}_2^2\mathrm{.}$ For inequalities, with $[z{]}_i^{+}=\max \{z_i,0\}$, the projected (Powell–Hestenes–Rockafellar) augmented Lagrangian is ${\mathcal{L}}_{\rho }(x,\nu ,\lambda )=f(x)+{\nu }^{T}h(x)+\frac{\rho }{2}\mathrm{\parallel }h(x){\mathrm{\parallel }}_2^2+\frac{\mathrm{\parallel }[\lambda +\rho g(x){]}_{+}{\mathrm{\parallel }}_2^2-\mathrm{\parallel }\lambda {\mathrm{\parallel }}_2^2}{2\rho }\mathrm{.}$ After an inner candidate $x^{+}$ is produced, the multiplier update is ${\nu }^{+}=\nu +\rho h(x^{+}),{\lambda }^{+}=[\lambda +\rho g(x^{+}){]}_{+}\mathrm{.}$ The bracket is componentwise. A declared inner stationarity residual is $\mathrm{\parallel }{\mathrm{\nabla }}_x{\mathcal{L}}_{\rho }(x^{+},\nu ,\lambda ){\mathrm{\parallel }}_2$; an inner procedure must report the residual it achieved or an explicit failure status.

The equality and inequality formulas are distinct branches. For an inequality with ${\lambda }_i+\rho g_i(x)>0$, its summand equals ${\lambda }_i{g}_i(x)+\rho g_i(x{)}^2\mathrm{/}2$; when ${\lambda }_i+\rho g_i(x)\le 0$, its summand is constant in $x$.