Mixed effects and longitudinal models

Assume AC. Let $(U_1,U_2)$ be a jointly multivariate-normal vector with mean $(m_1,m_2)$ and covariance $\left(\begin{array}{cc}S& 0\\ 0& T\end{array}\right),$ where $S$ and $T$ are allowed to be singular positive-semidefinite matrices. Then the random vectors $U_1$ and $U_2$ are independent.

No inverse of $S$ or $T$ is used. Positive definiteness of the full covariance is not required for this independence result.

Assume AC. Let $d\ge 1$, $m\in {\mathbb{R}}^d$, and let $\mathrm{\Sigma }$ be a real symmetric positive-definite $d\times d$ matrix. If $X\sim N_d(m,\mathrm{\Sigma })$, then its law has Lebesgue density $f_{m,\mathrm{\Sigma }}(x)=(2\pi {)}^{-d\mathrm{/}2}\det (\mathrm{\Sigma }{)}^{-1\mathrm{/}2}\exp \text{\hspace{0.17em}⁣}(-\frac{1}{2}(x-m{)}^T{\mathrm{\Sigma }}^{-1}(x-m)),x\in {\mathbb{R}}^d\mathrm{.}$

Positive definiteness is essential for this Lebesgue-density formula. Singular Gaussian laws remain defined, but are supported on lower-dimensional affine subspaces and are not covered by this theorem.

Let $X\in {\mathbb{R}}^{n\times p}$ have full column rank, with $p<n$, and let $\mathrm{col}(X{)}^{\perp }$ denote the orthogonal complement of its column space in Euclidean ${\mathbb{R}}^n$ (The orthogonal complement $W^{\perp }=\{v:⟨v,w⟩=0\text{ for all }w\in W\}$). An error-contrast matrix for $X$ is a matrix $K\in {\mathbb{R}}^{n\times (n-p)}$ whose columns form a basis of $\mathrm{col}(X{)}^{\perp }$. Thus $K$ has full column rank and $X^{T}K=0$, so $K^{T}Y$ is an $(n-p)$-dimensional vector of linear contrasts that annihilate $X\beta$. Such a matrix exists by A full-rank error-contrast matrix exists when n exceeds p ↗.

This column-basis convention makes the product $K^{T}Y$ dimensionally defined. The Gaussian restricted-likelihood definition below fixes one such $K$ independently of the covariance parameter.

The condition $p<n$ is needed for a nonzero error-contrast space. When $p=n$, the orthogonal complement has dimension zero and this page makes no variance-likelihood claim from contrasts.

Let $X\in {\mathbb{R}}^{n\times p}$ have full column rank in a fixed-design first-moment regression model, so ${\mathbb{E}}_{\beta }Y=X\beta$ under each law $P_{\beta }$ (Fixed-design linear regression model). Suppose the same known symmetric positive-definite matrix $V$ is the covariance of $Y$ under every $P_{\beta }$. Put $C=(X^T{V}^{-1}X{)}^{-1}\text{and}{\widehat{\beta }}_{\mathrm{G}\mathrm{L}\mathrm{S}}:=C{X}^T{V}^{-1}Y\mathrm{.}$ This is a statistic and estimator of the identity target $\beta$ under Statistics and estimators and Estimands on a statistical model: for fixed $X$ and $V$, the displayed expression is a linear, hence Borel-measurable, function of the response. The positive definiteness of $X^T{V}^{-1}X$ and existence of its inverse are established by The GLS information matrix is positive definite ↗.

This definition uses the true covariance as known. Replacing $V$ by an estimate defines a plug-in procedure; the theorem below does not assign that procedure the same exact covariance or BLUE guarantee.

The marginal-covariance lemma later identifies $V=ZG{Z}^T+R$ for the mixed model. The GLS definition itself applies to any fixed-design first-moment model with the stated common covariance.

Let $n,p\ge 1$ and $q\ge 0$ be finite integers. A finite-dimensional linear mixed model on one probability space consists of fixed matrices $X\in {\mathbb{R}}^{n\times p}$ and $Z\in {\mathbb{R}}^{n\times q}$, a fixed parameter $\beta \in {\mathbb{R}}^p$, and square-integrable random vectors $b\in {\mathbb{R}}^q$ and $\epsilon \in {\mathbb{R}}^n$ such that $Y=X\beta +Zb+\epsilon ,\mathbb{E}b=0,\mathbb{E}\epsilon =0,\mathrm{Cov}(b)=G,\mathrm{Cov}(\epsilon )=R,$ where $b$ and $\epsilon$ are independent random elements. Here $G$ and $R$ are the resulting symmetric positive-semidefinite covariance matrices. The fixed-design convention for $X$ and the first-moment regression model is that of Fixed-design linear regression model. The explicit scientific record of units, quantities, observation mechanism, and adequacy assumptions is required by Scientific statistical model contracts.

For longitudinal data, let $j$ index subjects and let $i\in S_j$ index the visits scheduled for subject $j$. A row of $Y$, $X$, and $Z$ corresponds to one observed subject–visit pair $(j,i)$; the time $t_{ji}$ is measured from a stated origin. Blocks of $b$ may represent subject-specific latent effects, while $R$ records the residual covariance across observed rows. If a visit pattern is fixed or conditioned on, the corresponding rows are selected from the scheduled vectors and matrices. If the pattern is random, its observation mechanism is specified separately; the displayed model does not assert that this mechanism is independent of the full outcomes or ignorable for likelihood inference.

The definition imposes no Gaussianity, no identical visit schedule, and no claim that a scientific application satisfies the proposed mean or covariance structure.

The vector $b$ is a latent random effect in the stated probability model; this definition gives it no Bayesian interpretation. Singular $G$ and singular $R$ are allowed here. In later results, positive definiteness of the marginal covariance $V=ZG{Z}^T+R$ is an additional hypothesis when an inverse is used.

The Gaussian branch of Finite-dimensional linear mixed model with explicit longitudinal roles adds the following assumption: under the Axiom of Choice, the concatenated vector $W=\left(\begin{array}{c}b\\ \epsilon \end{array}\right)$ has the multivariate-normal law $N_{q+n}(0,\mathrm{diag}(G,R))$ of Multivariate normal law, including singular covariance. Thus the full latent-effect and residual vector is jointly Gaussian. This is an additional branch; the moment-only definition and the covariance identity do not assume Gaussianity.

This definition carries AC because the current multivariate-normal supplier does. Singular $G$ or $R$ is permitted by that supplier; any later inverse or density requires a separate positive-definiteness hypothesis.

Let $m={\sum }_j\mathrm{\mid }{S}_j\mathrm{\mid }$ and let $Y^{\ast }\in {\mathbb{R}}^m$ be the complete scheduled response vector for subjects $j$ and visits $i$. Let the declared observed covariates take values in a measurable space $(\mathcal{X},\mathcal{A})$. The finite set of observation patterns is $\mathcal{R}=\{0,1{\}}^m$; $r_{ji}=1$ means visit $(j,i)$ is observed. A missingness mechanism is a function $g:\mathcal{R}\times {\mathbb{R}}^m\times \mathcal{X}\to [0,1]$ such that, for every $(y^{\ast },x)$, ${\sum }_{r\in \mathcal{R}}g(r\mid y^{\ast },x)=1$, and, for each fixed $r$, $(y^{\ast },x)\mapsto g(r\mid y^{\ast },x)$ is measurable for $\mathcal{B}({\mathbb{R}}^m)\otimes \mathcal{A}$. Thus $g(\cdot \mid y^{\ast },x)$ is a probability mass function on $\mathcal{R}$, and its dependence on the full data state is the finite-output kernel specified by the observation-mechanism field in Scientific statistical model contracts, which requires a probability measure at each state and measurable state dependence. For a pattern $r$, write $Y_{\mathrm{o}\mathrm{b}\mathrm{s}}(r)$ for the subvector of $Y^{\ast }$ at coordinates with $r_{ji}=1$.

The mechanism is missing completely at random (MCAR) if, for each pattern $r$, $g(r\mid y^{\ast },x)$ does not depend on $y^{\ast }$. It is missing at random (MAR) if, for each $r$, it depends on the full response vector only through $Y_{\mathrm{o}\mathrm{b}\mathrm{s}}(r)$ and $x$. It is missing not at random (MNAR) relative to this partition if the MAR condition fails for at least one pattern on the declared support. Monotone dropout means that, within each subject, observed visits form an initial segment of the scheduled visits, possibly empty or complete.

These are properties of the declared observation kernel. A mixed-model mean or covariance equation does not specify the kernel or imply MCAR, MAR, or MNAR.

For an LMM written on observed rows, $O_j=\{i:r_{ji}=1\}$ determines which coordinates and design rows are retained. If $r$ depends on the full scheduled outcomes, the distribution of the selected rows conditional on that pattern need not equal the corresponding unselected marginal subvector law.

For every full-column-rank $X\in {\mathbb{R}}^{n\times p}$ with $p<n$, there exists an error-contrast matrix $K\in {\mathbb{R}}^{n\times (n-p)}$ as defined in Error-contrast matrix for a full-rank fixed design.

If $X\in {\mathbb{R}}^{n\times p}$ has full column rank and $V\in {\mathbb{R}}^{n\times n}$ is symmetric positive definite, then $X^T{V}^{-1}X$ is symmetric positive definite. In particular it is invertible.

For the moment model $Y=X\beta +Zb+\epsilon$ of Finite-dimensional linear mixed model with explicit longitudinal roles, the marginal covariance is $\mathrm{Cov}(Y)=V:=ZG{Z}^T+R\mathrm{.}$ This identity uses only the stated means, covariances, and independence; it does not require Gaussianity.

Let $X\in {\mathbb{R}}^{n\times p}$ have full column rank with $p<n$, let $K\in {\mathbb{R}}^{n\times (n-p)}$ be an error-contrast matrix for $X$, and let $V\in {\mathbb{R}}^{n\times n}$ be symmetric positive definite. Then $K^{T}VK$ is symmetric positive definite. No Gaussian assumption is needed for this matrix claim.

Assume AC and the Gaussian branch of the finite-dimensional mixed model. Let $X\in {\mathbb{R}}^{n\times p}$ have full column rank with $p<n$, let $K\in {\mathbb{R}}^{n\times (n-p)}$ be an error-contrast matrix, and let $V=ZG{Z}^T+R=\mathrm{Cov}(Y)$. Then $K^{T}Y\sim N_{n-p}(0,K^{T}VK)\mathrm{.}$ The contrast law does not depend on $\beta$. The matrix $K^{T}VK$ may be singular; positive definiteness is a separate hypothesis when a density is needed.

Assume AC and the Gaussian branch of the finite-dimensional mixed model. Let $V=ZG{Z}^T+R$ be positive definite. With conditional expectation taken componentwise given $\sigma (Y)$, and conditional covariance defined entrywise by conditional expectations of centered products, one has $\mathbb{E}[b\mid Y]=G{Z}^T{V}^{-1}(Y-X\beta ),\mathrm{Cov}(b\mid Y)=G-G{Z}^T{V}^{-1}ZG\mathrm{.}$ The latent covariance $G$ may be singular.

Assume AC. Let $\mathrm{\Theta }$ be a nonempty parameter set and let $\theta \mapsto (G_{\theta },R_{\theta })$ be a declared covariance-component parameterization of the Gaussian mixed model, with $V_{\theta }=Z{G}_{\theta }{Z}^T+R_{\theta }$ positive semidefinite for every $\theta \in \mathrm{\Theta }$. Consider the parametric family $P_{\beta ,\theta }=N_n(X\beta ,V_{\theta }),(\beta ,\theta )\in {\mathbb{R}}^p\times \mathrm{\Theta },$ and the estimand $\psi (\beta ,\theta )=\theta$. Then $\psi$ is identifiable if and only if the covariance map $\theta \mapsto V_{\theta }$ is injective on $\mathrm{\Theta }$.

Assume AC and a Gaussian family of the form in Gaussian branch of the linear mixed model, indexed by a nonempty parameter set $\mathrm{\Theta }$. For each $\theta \in \mathrm{\Theta }$, let $G_{\theta }$ and $R_{\theta }$ be the latent and residual covariance matrices in $V_{\theta }=Z{G}_{\theta }{Z}^T+R_{\theta }\succ 0.$ Let the fixed design $X\in {\mathbb{R}}^{n\times p}$ have full column rank with $p<n$. Fix an error-contrast matrix $K\in {\mathbb{R}}^{n\times (n-p)}$ as in Error-contrast matrix for a full-rank fixed design, independently of $\theta$. By A Gaussian error contrast has the induced multivariate-normal law, the contrast vector $K^{T}Y$ has law $N_{n-p}(0,K^T{V}_{\theta }K)$ and its law does not depend on the fixed-effect parameter $\beta$. The local result The REML error-contrast covariance is positive definite shows that this covariance is positive definite, and A positive-definite multivariate normal law has its Gaussian density gives its Gaussian density. The restricted likelihood for this fixed contrast basis is $L_R(\theta ;y)=(2\pi {)}^{-(n-p)\mathrm{/}2}\det (K^T{V}_{\theta }K{)}^{-1\mathrm{/}2}\exp \text{\hspace{0.17em}⁣}(-\frac{1}{2}(K^{T}y{)}^T(K^T{V}_{\theta }K{)}^{-1}(K^{T}y))\mathrm{.}$ This item defines the likelihood for a fixed, parameter-independent choice of contrast coordinates; it makes no claim about likelihood comparisons across different fixed-effect designs.

The local column-basis convention is $K\in {\mathbb{R}}^{n\times (n-p)}$ with columns spanning $\mathrm{col}(X{)}^{\perp }$, so $K^{T}Y$ is defined. The restriction $p<n$ supplies a nonzero contrast space; when $p=n$, this definition makes no variance-likelihood claim.

Let the linear mixed model have fixed design $X\in {\mathbb{R}}^{n\times p}$ of full column rank and known marginal covariance $V=\mathrm{Cov}(Y)\succ 0$. Then ${\widehat{\beta }}_{\mathrm{G}\mathrm{L}\mathrm{S}}=(X^T{V}^{-1}X{)}^{-1}{X}^T{V}^{-1}Y$ is unbiased for $\beta$, has covariance $(X^T{V}^{-1}X{)}^{-1}$, and is BLUE: for every linear unbiased estimator $\tilde{\beta }$ and every $c\in {\mathbb{R}}^p$, $\mathrm{Var}(c^T{\widehat{\beta }}_{\mathrm{G}\mathrm{L}\mathrm{S}})\le \mathrm{Var}(c^T\tilde{\beta })\mathrm{.}$ Only the fixed-design mean and covariance are used; Gaussianity is not assumed.

Let $X\in {\mathbb{R}}^{n\times p}$ have full column rank with $p<n$, and let $K,K^{\mathrm{\prime }}\in {\mathbb{R}}^{n\times (n-p)}$ be two error-contrast matrices for $X$. For every covariance parameter $\theta$ in Restricted maximum likelihood from error contrasts, there is an invertible matrix $A\in {\mathbb{R}}^{(n-p)\times (n-p)}$, independent of $\theta$, such that $K^{\mathrm{\prime }}=KA$ and $L_R^{K^{\mathrm{\prime }}}(\theta ;y)=\mathrm{\mid }\det A{\mathrm{\mid }}^{-1}{L}_R^K(\theta ;y)$ for every response vector $y$. Thus changing the fixed contrast basis multiplies the likelihood by a positive factor independent of the covariance parameter and leaves its maximizers unchanged.

The numerical likelihood value depends on the chosen contrast coordinates, while the maximizing covariance parameters do not. The result does not compare different fixed-effect designs.

In the moment-only linear mixed model, suppose $X$ has full column rank and the known marginal covariance $V=ZG{Z}^T+R$ is positive definite. Define $\widehat{b}=G{Z}^T{V}^{-1}(Y-X{\widehat{\beta }}_{\mathrm{G}\mathrm{L}\mathrm{S}}),{\widehat{\beta }}_{\mathrm{G}\mathrm{L}\mathrm{S}}=(X^T{V}^{-1}X{)}^{-1}{X}^T{V}^{-1}Y\mathrm{.}$ Here best linear unbiased predictor (BLUP) means that $\widehat{b}$ is unbiased for $\mathbb{E}b=0$ for every fixed $\beta$ and its prediction-error covariance is minimal in positive-semidefinite order among all affine linear predictors $a+BY$ with that same unbiasedness property. Then $\widehat{b}$ is a BLUP and $\mathrm{Cov}(b-\widehat{b})=G-G{Z}^T{V}^{-1}ZG+G{Z}^T{V}^{-1}X(X^T{V}^{-1}X{)}^{-1}{X}^T{V}^{-1}ZG\mathrm{.}$ No Gaussianity or Axiom of Choice is assumed. The covariance formula treats $G$ and $R$ as known; it does not cover the extra uncertainty from estimating variance components.