One minimum-norm IRLS step

Inputs are an observed response vector $y$ with $f_i(y_i;{\theta }_i,\varphi )>0$ for every $i$, a finite design matrix $X$, an admissible finite current parameter $\beta$, and a GLM with common scale $a(\varphi )>0$. Require each ${\dot{\mu }}_i=d{\mu }_i\mathrm{/}d{\eta }_i$ to be finite and nonzero and each $V_i=b^{\mathrm{\prime }\mathrm{\prime }}({\theta }_i)$ to be finite and positive.

1. Compute ${\eta }_i=x_i^{\mathsf{T}}\beta$ and ${\mu }_i=g^{-1}({\eta }_i)$.
2. Compute the working response and positive weight $z_i={\eta }_i+\frac{y_i-{\mu }_i}{{\dot{\mu }}_i},w_i=\frac{{\dot{\mu }}_i^2}{a(\varphi )V_i}\mathrm{.}$
3. Form $W^{1\mathrm{/}2}=\mathrm{diag}(\sqrt{w_i})$ using Square roots exist: a unique $\sqrt{a}\ge 0$ with $(\sqrt{a}{)}^2=a$; the positives are $\{x^2:x\mathrm{\ne }0\}$, and put $A=W^{1\mathrm{/}2}X$, $r=W^{1\mathrm{/}2}z$. By The Moore--Penrose pseudoinverse $A^{+}$ as the solution of the four Penrose equations and Every finite real or complex matrix has a unique Moore--Penrose pseudoinverse, form $A^{+}$. Return ${\beta }_{\mathrm{n}\mathrm{e}\mathrm{w}}=A^{+}r$, the unique minimum-Euclidean-norm member of the nonempty weighted least-squares minimizer set by For every right-hand side $b$, $A^{+}b$ is the unique least-squares solution of minimum Euclidean norm.

The Moore--Penrose convention makes the procedure single-valued even when $X$ is rank deficient; the underlying least-squares minimizer need not then be unique. Existence and the minimum-norm property follow from Every finite real or complex matrix has a unique Moore--Penrose pseudoinverse and For every right-hand side $b$, $A^{+}b$ is the unique least-squares solution of minimum Euclidean norm. The output need not lie in $\mathrm{\Theta }$, and this definition makes no claim that repeated updates terminate or converge to a maximizer.