Positive bias makes the Hermite feature kernel strictly positive definite on every sphere

The marginal variance is $q$, and the standardized correlation is $r(x,y)=\frac{{\beta }^2+(x\cdot y)\mathrm{/}{n}_0}{q}=\alpha (x\cdot y)+\delta ,\alpha =\frac{1}{1+n_0{\beta }^2}>0,\delta =\frac{n_0{\beta }^2}{1+n_0{\beta }^2}>0,\alpha +\delta =1.$ The correlation lies in $[-1,1]$ because $\mathrm{\mid }x\cdot y\mathrm{\mid }\le 1$ and the displayed covariance is a valid two-dimensional Gram covariance of the input and bias features. Applying [F2] to $\mu$ gives $k(x,y)={\beta }^2+{\sum }_{r=0}^{\mathrm{\infty }}{a}_r^2(\alpha x\cdot y+\delta {)}^r\mathrm{.}$ The series is absolutely convergent since $\mathrm{\mid }r(x,y)\mathrm{\mid }\le 1$ and ${\sum }_r{a}_r^2<\mathrm{\infty }$.

Expanding by the binomial theorem, for each $x,y$ on the sphere the absolute double series is ${\sum }_{r=0}^{\mathrm{\infty }}{a}_r^2{\sum }_{m=0}^r\left(\genfrac{}{}{0px}{}{r}{m}\right){\alpha }^m{\delta }^{r-m}\mathrm{\mid }x\cdot y{\mathrm{\mid }}^m={\sum }_{r=0}^{\mathrm{\infty }}{a}_r^2(\alpha \mathrm{\mid }x\cdot y\mathrm{\mid }+\delta {)}^r\le {\sum }_{r=0}^{\mathrm{\infty }}{a}_r^2<\mathrm{\infty }$, since $\mathrm{\mid }x\cdot y\mathrm{\mid }\le 1$ and $\alpha +\delta =1$. Tonelli applied to this nonnegative absolute series justifies reordering the sign-changing binomial terms. Define $d_m={\mathbf{1}}_{\{m=0\}}{\beta }^2+{\alpha }^m{\sum }_{r=m}^{\mathrm{\infty }}{a}_r^2\left(\genfrac{}{}{0px}{}{r}{m}\right){\delta }^{r-m}(m\ge 0)$; then $k(x,y)={\sum }_{m=0}^{\mathrm{\infty }}{d}_m(x\cdot y{)}^m$. Also, [F3] and $\alpha +\delta =1$ give ${\sum }_m{d}_m={\beta }^2+{\sum }_r{a}_r^2<\mathrm{\infty }$.

The rescaled function $\mu (z)=\sigma (\sqrt{q}z)$ is continuous and nonpolynomial: if it were a polynomial, then $\sigma (u)=\mu (u\mathrm{/}\sqrt{q})$ would be a polynomial because $q>0$. It is globally Lipschitz, so [F1] supplies its $L^2(\gamma )$ membership and Parseval. The function $\mu$ has infinitely many nonzero Hermite coefficients: otherwise [F1] would make it equal almost everywhere to a polynomial, and continuity would make that equality hold everywhere, contradicting nonpolynomiality. Thus for each $m$ there is an $r\ge m$ with $a_r\mathrm{\ne }0$, and the corresponding term in [F3]’s sum is strictly positive because $\alpha ,\delta >0$. Hence $d_m>0$ for every $m$, including $m=0$ because $\beta >0$.

For any $c\in {\mathbb{R}}^N$, the quadratic form is ${\sum }_{i,j}{c}_i{c}_{j}k(x_i,x_j)={\sum }_{m=0}^{\mathrm{\infty }}{d}_m{\parallel {\sum }_{i=1}^N{c}_i{x}_i^{\otimes m}\parallel }_2^2\ge 0.$ Indeed, $⟨x_i^{\otimes m},x_j^{\otimes m}⟩=(x_i\cdot x_j{)}^m$ by the induced tensor inner product, and the convergent series may be evaluated termwise because $\mathrm{\mid }{x}_i\cdot x_j\mathrm{\mid }\le 1$ and ${\sum }_m{d}_m<\mathrm{\infty }$.

If this quadratic form were zero, positivity of every $d_m$ would force ${\sum }_i{c}_i{x}_i^{\otimes m}=0$ for every $m\ge 0$. Consequently ${\sum }_i{c}_{i}p(x_i)=0$ for every polynomial $p$ on ${\mathbb{R}}^{n_0}$: each monomial of degree $m$ is a coordinate linear functional on $x^{\otimes m}$, and constants correspond to $m=0$. For each index $j$, take $p_j(x)={\prod }_{i\mathrm{\ne }j}\mathrm{\parallel }x-x_i{\mathrm{\parallel }}_2^2$, with the empty product equal to $1$. Then $p_j(x_i)=0$ for $i\mathrm{\ne }j$ and $p_j(x_j)>0$, so $c_j=0$. This holds for every $j$, contradicting $c\mathrm{\ne }0$. Thus the quadratic form in step 3.2 is strictly positive.