Finite Gaussian-process regression observation model

Let $\mathcal{X}$ be a set, let $m:\mathcal{X}\to \mathbb{R}$, and let $k:\mathcal{X}\times \mathcal{X}\to \mathbb{R}$ be symmetric and positive semidefinite, meaning that for every finite ordered list $t_1,\dots ,t_r$ in $\mathcal{X}$ and every $a\in {\mathbb{R}}^r$, ${\sum }_{i,j=1}^r{a}_i{a}_{j}k(t_i,t_j)\ge 0.$ For finite ordered lists $A=(a_i{)}_{i=1}^r$ and $B=(b_j{)}_{j=1}^s$ in $\mathcal{X}$, define $K_{AB}=(k(a_i,b_j){)}_{i,j};$ repeated indices are allowed.

Given a real Gaussian process $f=(f_t{)}_{t\in \mathcal{X}}$ with mean $m(t)=\mathbb{E}[f_t]$ and covariance $k(s,t)=\mathbb{E}[(f_s-m(s))(f_t-m(t))]$, and a nonempty finite training list $X=(x_i{)}_{i=1}^n$ in $\mathcal{X}$, define $f_X=(f_{x_i}{)}_{i=1}^n$ and $m_X=(m(x_i){)}_{i=1}^n$. For a centered Gaussian noise vector $\epsilon \sim N_n(0,R)$ with symmetric positive-semidefinite covariance $R$ that is independent of $f$, define the observation vector $Y=f_X+\epsilon$ and the matrix $A_X=K_{XX}+R$.

For a nonempty finite test list $X_{\ast }=(x_j^{\ast }{)}_{j=1}^p$ in $\mathcal{X}$, define the latent test vector $f_{\ast }=f_{X_{\ast }}$. If a centered Gaussian vector ${\eta }_{\ast }\sim N_p(0,R_{\ast })$ with symmetric positive-semidefinite covariance $R_{\ast }$ is independent of the pair $(f,\epsilon )$, define the noisy test vector $Y_{\ast }=f_{\ast }+{\eta }_{\ast }$.

A Gaussian process has multivariate-normal finite evaluation vectors, including singular laws and repeated indices.

Finite Gaussian laws allow positive-semidefinite, possibly singular covariance matrices.

Independence means independence of the sigma-algebras generated by the random elements.

Rasmussen and Williams, GPML, Chapter 2 §2.2, printed pp. 13–18

Bishop, PRML, §6.4.2, printed pp. 306–310

The single prior node represents one process, so $f_X$ and $f_{\ast }$ may be correlated. Arrows into $Y$ and $Y_{\ast }$ denote vector addition. The training noise $\epsilon$ is independent of $f$, and the test noise ${\eta }_{\ast }$ is independent of the pair $(f,\epsilon )$; the diagram records no posterior independence claim.