Finite LLE affine least-squares weights have a unique minimum-norm minimizer

Let $w^0=(1,0,\dots ,0)\in {\mathbb{R}}^k$, let $L=\{z\in {\mathbb{R}}^k:{\sum }_s{z}_s=0\}$, and define $T:L\to {\mathbb{R}}^p$ by $Tz={\sum }_s{z}_s{x}_{j_s}$. Every feasible $w$ has a unique form $w=w^0+z$ with $z\in L$, and its residual is $x_i-x_{j_1}-Tz$. Thus the constrained problem is an ordinary finite-dimensional least-squares problem for $T$ on the inner-product subspace $L$.

By [F2], choose a least-squares minimizer $z_0\in L$ and all minimizers are $z_0+N$, where $N=\ker T\subseteq L$. Therefore all feasible minimizing weights form the nonempty affine set $w^0+z_0+N$. In the degenerate case $T=0$, $z_0=0$ is a minimizer and $N=L$, so this set contains every feasible weight.

Put $w_{\ast }=w^0+z_0$. By [F4], decompose $w_{\ast }=u+v$ with $u\in N$ and $v\in N^{\perp }$ in ${\mathbb{R}}^k$. Since $-u\in N$, $v=w_{\ast }-u$ remains in the affine minimizer set. For any other minimizer $v+h$ with $h\in N$, orthogonality gives $\mathrm{\parallel }v+h{\mathrm{\parallel }}_2^2=\mathrm{\parallel }v{\mathrm{\parallel }}_2^2+\mathrm{\parallel }h{\mathrm{\parallel }}_2^2$, which is at least $\mathrm{\parallel }v{\mathrm{\parallel }}_2^2$ with equality only if $h=0$. Hence $v$ is the unique minimum-norm minimizer. When $k=1$, $L=N=\{0\}$ and this gives the unique feasible weight $w^0=(1)$.

The proof applies the least-squares existence result once to this fixed finite-dimensional problem and uses a finite orthogonal decomposition; it selects no sequence of approximate minimizers and uses no Countable Choice.