Response surfaces and optimal design

For $t\in [0,1]$, put $A_t=(1-t)A_0+t{A}_1$ and $H=A_1-A_0$. For every nonzero $v$, $v^{\mathsf{T}}{A}_{t}v=(1-t)v^{\mathsf{T}}{A}_{0}v+t{v}^{\mathsf{T}}{A}_{1}v>0$, so $A_t$ is symmetric positive definite. By [F1] the standard coordinate basis is orthonormal, [F2] makes the symmetric matrix self-adjoint, and [F3] gives an orthonormal eigenbasis. Each eigenvalue ${\lambda }_i(t)$ is positive because for its unit eigenvector $v_i$, ${\lambda }_i(t)=v_i^{\mathsf{T}}{A}_t{v}_i>0$.

In that eigenbasis $A_t=Q\mathrm{\Lambda }{Q}^{\mathsf{T}}$ with $Q{Q}^{\mathsf{T}}=I_p$ and $\mathrm{\Lambda }=\mathrm{diag}({\lambda }_1(t),\dots ,{\lambda }_p(t))$. By [F4] and [F5], $\det A_t=\det Q\det \mathrm{\Lambda }\det Q^{\mathsf{T}}=\det \mathrm{\Lambda }={\prod }_i{\lambda }_i(t)>0$, since [F5] gives $\det Q\det Q^{\mathsf{T}}=\det I_p$ and [F6] gives $\det I_p=1$ from the Leibniz formula. Thus $g(t)=\log \det A_t$ is real-valued on $[0,1]$.

Since $A_t$ is invertible, the determinant differential in [F8] is Jacobi's formula; its cofactor representation and invertible-matrix reduction use [F12], [F13], and [F14]. Because the determinant is differentiable, it is continuous; its positive value at each $A_t$ remains positive on a small neighborhood of that path point, so the logarithm's domain condition in [F15] is met, including at $t=0,1$. The logarithm derivative and total chain rule [F15] then give $g^{\mathrm{\prime }}(t)=(\det A_t{)}^{-1}D\det (A_t)[H]=\mathrm{tr}(A_t^{-1}H)$.

Differentiate this trace along the same line. The product rule [F7], inverse derivative [F9], and trace derivative [F10], composed with the affine path by [F15], give $g^{\mathrm{\prime }\mathrm{\prime }}(t)=-\mathrm{tr}(A_t^{-1}H{A}_t^{-1}H)$.

Fix $t$ and define the symmetric matrix $S=A_t^{-1\mathrm{/}2}H{A}_t^{-1\mathrm{/}2}$ by diagonalizing $A_t$ from [F3] and taking the positive inverse square roots of its eigenvalues. By [F11], $\mathrm{tr}(XY)=\mathrm{tr}(YX)$ because both sides are ${\sum }_{i,j}{X}_{ij}{Y}_{ji}$ with indices interchanged; applying this identity gives $\mathrm{tr}(A_t^{-1}H{A}_t^{-1}H)=\mathrm{tr}(S^2)={\sum }_{i,j}{S}_{ij}^2\ge 0$. Hence [step 4.1] implies $g^{\mathrm{\prime }\mathrm{\prime }}(t)\le 0$.

The formulas in steps 3.1–4.1 make $g$ differentiable on a neighborhood of $[0,1]$ and $g^{\mathrm{\prime }}$ differentiable there; in particular each is continuous on the closed subintervals used below. For $0\le r<s\le 1$, apply [F16] to $g^{\mathrm{\prime }}$ on $[r,s]$; step 5.1 gives $g^{\mathrm{\prime }}(s)-g^{\mathrm{\prime }}(r)=g^{\mathrm{\prime }\mathrm{\prime }}(c)(s-r)\le 0$ for some $c\in (r,s)$. Thus $g^{\mathrm{\prime }}$ is nonincreasing. If $0<t<1$, apply [F16] to $g$ on $[0,t]$ and $[t,1]$. The respective slopes equal $g^{\mathrm{\prime }}(a)$ and $g^{\mathrm{\prime }}(b)$ for $a<t<b$, hence $(g(t)-g(0))\mathrm{/}t\ge (g(1)-g(t))\mathrm{/}(1-t)$. Rearrangement gives $g(t)\ge (1-t)g(0)+tg(1)$. At $t=0,1$ it is equality. This is precisely the stated log-determinant chord inequality.

For any $x,h\in {\mathbb{R}}^k$, symmetry of $B$ gives $m(x+h)-m(x)=h^{\mathsf{T}}(\beta +2Bx)+h^{\mathsf{T}}Bh$. The second term is bounded in absolute value by a constant times $\mathrm{\parallel }h{\mathrm{\parallel }}^2$, so the differential at $x$ is $h\mapsto h^{\mathsf{T}}(\beta +2Bx)$; hence $x$ is stationary exactly when $\beta +2Bx=0$.

Since $B$ is nonsingular, this equation has the unique solution $x_0=-\frac{1}{2}{B}^{-1}\beta$.

At this point, for every $h$, $m(x_0+h)-m(x_0)=h^{\mathsf{T}}Bh$. Regard $B$ as an operator on ${\mathbb{R}}^k$ with the real coordinate inner product [F2], for which the standard basis is orthonormal. By [F3] it is self-adjoint, so [F4] supplies an orthonormal eigenbasis $v_1,\dots ,v_k$ with real eigenvalues ${\lambda }_i$; none is zero because $B$ is nonsingular. Writing $h={\sum }_i{z}_i{v}_i$ yields $h^{\mathsf{T}}Bh={\sum }_i{\lambda }_i{z}_i^2$.

If all ${\lambda }_i>0$, the increment in step 3.1 is positive for every nonzero $h$, and if all ${\lambda }_i<0$ it is negative for every nonzero $h$. If both signs occur, choosing $h$ along a positive-eigenvalue vector or a negative-eigenvalue vector gives increments of opposite signs, so $x_0$ is a saddle. The exact quadratic identity makes the first two classifications global for the polynomial on ${\mathbb{R}}^k$.

If $x_0$ lies in the interior of $\mathcal{R}$, sufficiently small perturbations in every direction remain in $\mathcal{R}$, so step 4.1 also gives the local classification there. Whenever $x_0\in \mathcal{R}$, the strict global inequalities in the definite cases of step 4.1 hold for every $x\in \mathcal{R}\setminus \{x_0\}$, proving strict constrained optimality even at the boundary. A boundary saddle need not persist: $m(x_1,x_2)=x_1^2-x_2^2$ has an ambient saddle at $0$, but its restriction to $\mathcal{R}=\{(t,0):t\in \mathbb{R}\}$ has a strict minimum there. If $x_0\mathrm{\notin }\mathcal{R}$, it is infeasible.

Fix $x\in \mathcal{X}$ and let ${\delta }_x$ be the design putting weight $1$ at $x$. For $0\le t<1$, ${\xi }_t=(1-t){\xi }^{\ast }+t{\delta }_x$ has matrix $M_t=(1-t)M^{\ast }+tf(x)f(x{)}^{\mathsf{T}}$, which is positive definite because $M^{\ast }$ is positive definite and $f(x)f(x{)}^{\mathsf{T}}$ is positive semidefinite.

Conversely, suppose $u^{\ast }(x)\le p$ for every candidate. Let $\eta$ be any nonsingular approximate design and put $N=M(\eta )$. The segment $M_s=(1-s)M^{\ast }+sN$ is positive definite; by [F3], $g(s)=\log \det M_s$ is concave. By [F4], with its cofactor and adjugate support [F6], [F7], and [F8], and the logarithm-chain rule [F9], and by [F1], $g^{\mathrm{\prime }}(0)=\mathrm{tr}((M^{\ast }{)}^{-1}(N-M^{\ast }))={\sum }_x{\eta }_x{u}^{\ast }(x)-p\le 0$.

If ${\xi }^{\ast }$ is D-optimal, the defining maximum property in [F2] gives $g_x(t)=\log \det M_t\le g_x(0)$ for $0\le t<1$, so its right derivative at $0$ is nonpositive. By [F4], with its cofactor and adjugate support [F6], [F7], and [F8], the logarithm-chain rule [F9] gives $g_x^{\mathrm{\prime }}(0)=\mathrm{tr}((M^{\ast }{)}^{-1}(f(x)f(x{)}^{\mathsf{T}}-M^{\ast }))=u^{\ast }(x)-p$; the trace of the rank-one term is $f(x{)}^{\mathsf{T}}(M^{\ast }{)}^{-1}f(x)$ by the coordinate sum, and $\mathrm{tr}(I_p)=p$ by [F5]. Thus $u^{\ast }(x)\le p$. Since $x$ was arbitrary, the pointwise bound is necessary.

For $0<s<1$, the concavity in step 1.2 gives $g(s)\ge (1-s)g(0)+sg(1)$, hence $g(1)-g(0)\le (g(s)-g(0))\mathrm{/}s$. Let $s\downarrow 0$; the right derivative already computed in step 1.2 gives $g(1)-g(0)\le g^{\mathrm{\prime }}(0)\le 0$. Thus $\log \det N\le \log \det M^{\ast }$. Since $\eta$ was arbitrary among nonsingular designs, ${\xi }^{\ast }$ is D-optimal.

Finally, ${\sum }_x{\xi }_x^{\ast }{u}^{\ast }(x)=\mathrm{tr}((M^{\ast }{)}^{-1}{\sum }_x{\xi }_x^{\ast }f(x)f(x{)}^{\mathsf{T}})=\mathrm{tr}(I_p)=p$ by [F1] and [F5]. Each term is at most $p$ by the bound assumed in step 1.2, so every term with ${\xi }_x^{\ast }>0$ equals $p$. This proves equality on the support and completes both directions.