Exact finite-MLE criterion for a minimal finite exponential Gibbs family

Since every ${\pi }_{\theta }(x_i)>0$, the likelihood is positive. Repeated application of the logarithm product law [F8], followed by $\log (\exp t)=t$ [F7], gives the log likelihood $\mathrm{\ell }(\theta )=\log {\prod }_{i=1}^n{\pi }_{\theta }(x_i)={\sum }_{i=1}^n\log w(x_i)+n(\theta \cdot \bar{T}-A(\theta ))\mathrm{.}$ Because $\log$ is strictly increasing [F8], maximizing this log likelihood is equivalent to maximizing the likelihood. Also $\bar{T}$ is the convex combination with weights $1\mathrm{/}n$ of $T(x_i)$, so $\bar{T}\in K$.

Suppose $\bar{T}\in \mathrm{ri}K$. Minimality makes $K$ full-dimensional, so there is $ϵ>0$ such that the open ball of radius $ϵ$ about $\bar{T}$ lies in $K$; set $\delta =ϵ\mathrm{/}2$, so the closed ball of radius $\delta$ lies in that open ball and hence in $K$. For every unit $u$, $\bar{T}+\delta u$ is a convex combination of the vectors $T(x)$; since $u\cdot u=1$ by [F11], at least one $x$ satisfies $u\cdot (T(x)-\bar{T})\ge \delta$. Put $w_{\min }={\min }_{x\in X}w(x)>0$. If $\theta \mathrm{\ne }0$, write $r=\mathrm{\parallel }\theta {\mathrm{\parallel }}_2$ and $u=\theta \mathrm{/}r$; [F10] gives $r>0$ and $\mathrm{\parallel }u{\mathrm{\parallel }}_2=1$. For some $x$ as above, $\theta \cdot (T(x)-\bar{T})\ge r\delta$. Since the $x$th summand is part of $Z(\theta )$ and is positive by [F13], monotonicity and the product law of $\log$ [F8], together with [F7], give $A(\theta )=\log Z(\theta )\ge \log \left(w_{\min }{e}^{\theta \cdot T(x)}\right)=\log w_{\min }+\theta \cdot T(x),$ and hence $A(\theta )-\theta \cdot \bar{T}\ge r\delta +\log w_{\min }$. Thus $\theta \mapsto A(\theta )-\theta \cdot \bar{T}$ tends to $+\mathrm{\infty }$ as $\mathrm{\parallel }\theta {\mathrm{\parallel }}_2\to \mathrm{\infty }$.

Let $\theta \mathrm{\ne }\phi$ and put $v=\phi -\theta$ and $g(s)=\mathrm{\ell }(\theta +sv)$ for $s\in [0,1]$. By $A\in C^2$ [F3], $g$ is twice continuously differentiable; by [F2] its second derivative is $g^{\mathrm{\prime }\mathrm{\prime }}(s)=-n{v}^{\mathsf{T}}{\mathrm{\nabla }}^{2}A(\theta +sv)v<0$. Thus $g^{\mathrm{\prime }}$ is strictly decreasing: for $0\le a<b\le 1$, the mean value theorem [F9] applied to $g^{\mathrm{\prime }}$ gives $g^{\mathrm{\prime }}(b)-g^{\mathrm{\prime }}(a)=g^{\mathrm{\prime }\mathrm{\prime }}(c)(b-a)<0$ for some $c\in (a,b)$. Fix $t\in (0,1)$. Applying [F9] to $g$ on $[0,t]$ and $[t,1]$ gives $\alpha \in (0,t)$ and $\beta \in (t,1)$ such that $\frac{g(t)-g(0)}{t}=g^{\mathrm{\prime }}(\alpha )>g^{\mathrm{\prime }}(\beta )=\frac{g(1)-g(t)}{1-t}\mathrm{.}$ Rearranging yields $g(t)>(1-t)g(0)+tg(1)$. By [F12], $\mathrm{\ell }$ is strictly concave on ${\mathbb{R}}^k$.

The function $\mathrm{\Phi }(\theta ):=A(\theta )-\theta \cdot \bar{T}$ is continuous: $A$ is continuous by [F3], and the linear form is continuous by the Cauchy–Schwarz bound [F10]. It tends to $+\mathrm{\infty }$ by step 1.2. Choose $R>0$ large enough that $\mathrm{\Phi }(\theta )>\mathrm{\Phi }(0)$ whenever $\mathrm{\parallel }\theta {\mathrm{\parallel }}_2>R$. The closed ball ${\bar{B}}_2(0,R)$ is nonempty and compact by [F6], so [F5] gives a minimizer $\widehat{\theta }$ on it. Since $0$ lies in the ball and every point outside has value greater than $\mathrm{\Phi }(0)$, $\widehat{\theta }$ is a global minimizer on ${\mathbb{R}}^k$. The log-likelihood formula in step 1.1 and $n\ge 1$ show that it is a finite MLE. If two distinct finite MLEs existed, their midpoint would have a strictly larger log likelihood by step 2.1, a contradiction; hence it is unique.

Suppose $\bar{T}\mathrm{\notin }\mathrm{ri}K$. Since $\bar{T}\in K$ by step 1.1 and $K$ is full-dimensional, $\bar{T}$ is a boundary point. By [F4] choose a unit $u$ with $u\cdot (T(x)-\bar{T})\le 0$ for every $x$. Full affine span implies at least one of these inequalities is strict; otherwise $u\cdot T$ would be constant on $X$. Every ${\pi }_{\theta }(x)$ is positive, so $u\cdot (\mathrm{\nabla }A(\theta )-\bar{T})={\sum }_x{\pi }_{\theta }(x)u\cdot (T(x)-\bar{T})<0$ for every finite $\theta$. Therefore the directional derivative of $\mathrm{\ell }$ along $u$ is $nu\cdot (\bar{T}-\mathrm{\nabla }A(\theta ))>0$ at every finite $\theta$. By the definition of a positive derivative, a sufficiently small positive step along $u$ increases $\mathrm{\ell }$, and therefore increases the likelihood by [F8]. No finite MLE exists. This proves both directions.