Inverse Gibbs models, identifiability and correlated uncertainty — Examples

The two statistic values are $-1$ and $+1$, so their affine hull is $\mathbb{R}$, the family is minimal, the convex hull is $[-1,1]$, and the sample mean is $1$, which is not in its interior $(-1,1)$. Hence [F2] already implies there is no finite MLE.

Let $S(h)=e^h+e^{-h}$. Both exponential terms and hence $S(h)$ are positive by [F7]. For the fixed all-plus IID sample the product likelihood is $L(h)={\prod }_{i=1}^n{\pi }_h(+1)=(e^h\mathrm{/}S(h){)}^n>0$ by [A1, F1]. Applying the positive-product law [F8] repeatedly to these $n\ge 1$ factors, then its quotient law and the inverse identity [F7], gives $\log L(h)=n(\log (e^h)-\log S(h))=n(h-\log S(h))=\mathrm{\ell }(h)$. Since $\log$ is strictly increasing, this has exactly the same maximizers as $L$. By [F3] and [F5], the two exponential terms have derivatives $e^h$ and $-e^{-h}$; [F6] therefore gives $S^{\mathrm{\prime }}(h)=e^h-e^{-h}$. The identity function has derivative $1$ directly from its difference quotient. Applying the logarithm derivative [F4] and chain rule [F5] to this derived log likelihood gives ${\mathrm{\ell }}^{\mathrm{\prime }}(h)=n(1-\frac{e^h-e^{-h}}{e^h+e^{-h}})=\frac{2n{e}^{-h}}{e^h+e^{-h}}>0$ for every finite $h$. The derived formula is differentiable on all of $\mathbb{R}$, hence continuous there: its increment is its convergent difference quotient times the real increment. For any $h_1<h_2$, [F9] applied on $[h_1,h_2]$ gives $\mathrm{\ell }(h_2)-\mathrm{\ell }(h_1)={\mathrm{\ell }}^{\mathrm{\prime }}(c)(h_2-h_1)>0$ for some $c\in (h_1,h_2)$. Thus $\mathrm{\ell }$ is strictly increasing; by [F8] the likelihood also increases. In particular each finite field has a larger likelihood at a greater field, and no finite field maximizes it.

The four unnormalized Gibbs weights are $\exp (J),\exp (-J),\exp (-J),\exp (J)$, so the partition function is $Z_J=2\exp (J)+2\exp (-J)=4\cosh J$. Therefore the joint likelihood of $(+1,+1)$ is $\exp (J)\mathrm{/}(4\cosh J)$.

Given either other spin is $+1$, the conditional probability for the remaining spin to be $+1$ is $\exp (J)\mathrm{/}(\exp (J)+\exp (-J))=\exp (J)\mathrm{/}(2\cosh J)$. Thus the pseudo-likelihood is the product $[\exp (J)\mathrm{/}(2\cosh J){]}^2$.

The exact likelihood ratio for $J=1$ against $J=0$ is $r=e\mathrm{/}\cosh 1$, while the pseudo-likelihood ratio is $r^2$. By [F3] and [F4], $e>1$ and $e^{-1}=1\mathrm{/}e<1$, so $\cosh 1=(e+e^{-1})\mathrm{/}2<e$. Hence $r>1$ and $r\mathrm{\ne }{r}^2$. Under the equal prior, posterior odds equal likelihood ratios, so the exact and pseudo posterior probabilities differ. The pseudo-likelihood product therefore cannot be substituted as the joint likelihood for this model.

Put $c=\cosh 1$. For each equal-spin configuration, both site conditionals are $e\mathrm{/}(2c)$; for each opposite-spin configuration, both are $\exp (-1)\mathrm{/}(2c)=1\mathrm{/}(2ec)$ by [F1], [F2], and [F4]. Summing the four products gives ${\sum }_{\sigma \in S^{\mathrm{\Lambda }}}P{L}_1(\sigma )=\frac{e^2+(1\mathrm{/}e{)}^2}{2c^2}=2\frac{(e+1\mathrm{/}e{)}^2-2}{(e+1\mathrm{/}e{)}^2}=2-\frac{1}{c^2}\mathrm{.}$ By [F3] and [F4], $e>1$ and $e^{-1}=1\mathrm{/}e$, so $c-1=(e-1{)}^2\mathrm{/}(2e)>0$; hence the sum is greater than one. Thus the pseudo-likelihood is not a probability mass function on the configuration space, whereas the joint Gibbs likelihood is.