Exact EM updates for a finite Gaussian mixture

Inputs. Assume AC. Supply positive integers $n,p,K$, observations $x_1,\dots ,x_n\in {\mathbb{R}}^p$ whose affine span is ${\mathbb{R}}^p$ (equivalently, no nonzero $v\in {\mathbb{R}}^p$ has $v^{\mathsf{T}}{x}_i$ constant over all $i$), an initial finite Gaussian mixture parameter ${\theta }_0=(({\pi }_k^{(0)},{\mu }_k^{(0)},{\mathrm{\Sigma }}_k^{(0)}){)}_{k=1}^K$ in the domain of the finite-mixture definition, and a positive integer $T$.

Procedure. For $t=0,1,\dots ,T-1$, compute the responsibilities

${\gamma }_{ik}^{(t)}:=\frac{{\pi }_k^{(t)}{\varphi }_p(x_i;{\mu }_k^{(t)},{\mathrm{\Sigma }}_k^{(t)})}{{\sum }_{j=1}^K{\pi }_j^{(t)}{\varphi }_p(x_i;{\mu }_j^{(t)},{\mathrm{\Sigma }}_j^{(t)})},N_k^{(t)}:={\sum }_{i=1}^n{\gamma }_{ik}^{(t)}\mathrm{.}$

Then make the exact updates

${\pi }_k^{(t+1)}:=\frac{N_k^{(t)}}{n},{\mu }_k^{(t+1)}:=\frac{1}{N_k^{(t)}}{\sum }_{i=1}^n{\gamma }_{ik}^{(t)}{x}_i,$

${\mathrm{\Sigma }}_k^{(t+1)}:=\frac{1}{N_k^{(t)}}{\sum }_{i=1}^n{\gamma }_{ik}^{(t)}(x_i-{\mu }_k^{(t+1)})(x_i-{\mu }_k^{(t+1)}{)}^{\mathsf{T}}\mathrm{.}$

Output. Return the finite parameter sequence $({\theta }_0,{\theta }_1,\dots ,{\theta }_T)$. Every E-step uses the current parameter and every M-step uses that E-step's responsibilities. The affine-span hypothesis is imposed for the unrestricted positive-definite covariance update; constrained covariance domains require their own M-step. This definition makes no claim about convergence of parameters or a global maximum.