Synthetic likelihood and indirect inference

Let $\mathsf{E}=((\mathcal{X},\mathcal{A}),\mathrm{\Theta },(P_{\theta }{)}_{\theta \in \mathrm{\Theta }})$ be a statistical experiment and let $S:\mathcal{X}\to {\mathbb{R}}^d$ be a statistic, meaning a measurable map to its declared Borel target as in Statistics and estimators, where $d\ge 1$ is finite. Expectations are Lebesgue integrals under $P_{\theta }$ as in Expectation of a nonnegative or integrable random variable, and variances and covariances use centered second moments as in Moments, variance, and covariance on a probability space. For every $\theta \in \mathrm{\Theta }$, assume ${\mathbb{E}}_{\theta }\text{\hspace{0.17em}⁣}\left[\mathrm{\parallel }S{\mathrm{\parallel }}_2^2\right]<\mathrm{\infty },\mathrm{\parallel }x{\mathrm{\parallel }}_2^2:={\sum }_{j=1}^d{x}_j^2\mathrm{.}$ The binding mean and summary covariance are the pointwise-defined quantities $b_j(\theta ):={\mathbb{E}}_{\theta }[S_j],{\mathrm{\Sigma }}_{jk}(\theta ):={\mathbb{E}}_{\theta }\text{\hspace{0.17em}⁣}[(S_j-b_j(\theta ))(S_k-b_k(\theta ))](1\le j,k\le d)\mathrm{.}$ Each covariance entry is finite since $2\mathrm{\mid }uv\mathrm{\mid }\le u^2+v^2$ and the coordinates are square-integrable. The matrix is symmetric and positive semidefinite because for $a\in {\mathbb{R}}^d$, $a^T\mathrm{\Sigma }(\theta )a={\mathbb{E}}_{\theta }\text{\hspace{0.17em}⁣}[(a^T(S-b(\theta )){)}^2]\ge 0.$ No measurability or continuity of $\theta \mapsto b(\theta )$ or $\theta \mapsto \mathrm{\Sigma }(\theta )$ follows from these pointwise definitions.

Assume AC (The Axiom of Choice). If $\mathrm{\Sigma }(\theta )$ is positive definite, the ideal Gaussian synthetic likelihood for observed data $y\in \mathcal{X}$ is the Gaussian density evaluated at its summary, as supplied by Multivariate normal law, including singular covariance and A positive-definite multivariate normal law has its Gaussian density, $g_{\mathrm{S}\mathrm{L}}(y\mid \theta ):={\phi }_d\text{\hspace{0.17em}⁣}(S(y);b(\theta ),\mathrm{\Sigma }(\theta ))=(2\pi {)}^{-d\mathrm{/}2}\det (\mathrm{\Sigma }(\theta ){)}^{-1\mathrm{/}2}\exp \text{\hspace{0.17em}⁣}[-\frac{1}{2}(S(y)-b(\theta ){)}^T\mathrm{\Sigma }(\theta {)}^{-1}(S(y)-b(\theta ))]\mathrm{.}$ Here ${\phi }_d(\cdot ;m,\mathrm{\Sigma })$ is the Lebesgue density of $N_d(m,\mathrm{\Sigma })$. If $\mathrm{\Sigma }(\theta )$ is singular, this positive-definite-covariance density is not defined by this item.

Let $Q_{\theta }:=P_{\theta }\circ S^{-1}$ be the pushforward law of the statistic under $P_{\theta }$ as in Law or distribution of a random element. For $m\ge 2$, let $S_1,\dots ,S_m$ be IID simulated summaries with common law $Q_{\theta }$, where IID means a common law and mutual independence as in Identical distribution and IID families. Set ${\bar{S}}_m:=m^{-1}{\sum }_{i=1}^m{S}_i,{\widehat{\mathrm{\Sigma }}}_m:=(m-1{)}^{-1}{\sum }_{i=1}^m(S_i-{\bar{S}}_m)(S_i-{\bar{S}}_m{)}^T\mathrm{.}$ The finite-simulation plug-in synthetic likelihood is the random quantity ${\widehat{g}}_{\mathrm{S}\mathrm{L},m}(y\mid \theta ):={\phi }_d(S(y);{\bar{S}}_m,{\widehat{\mathrm{\Sigma }}}_m)\mathrm{.}$ when ${\widehat{\mathrm{\Sigma }}}_m$ is positive definite, and is unavailable otherwise. The finite plug-in quantity is a function of simulated summaries; it is not defined as an unbiased estimator of the ideal Gaussian density.

The exact experiment’s likelihood density, when the experiment is dominated as in Dominated statistical experiments and given a pointwise density as in Likelihood densities, and this Gaussian summary approximation are different objects. The latter density-selection interface assumes AC and does not assert a common exceptional null set across an arbitrary parameter set; the present definition adds no parameter-indexed measurability. This definition makes no claim that the actual summary law is Gaussian, that $g_{\mathrm{S}\mathrm{L}}$ equals an exact likelihood, or that it yields posterior calibration, consistency, a Bernstein–von Mises theorem, or coverage. No optimizer or optimizer convergence property is included.

If a positive-definite covariance matrix varies differentiably in a real matrix direction $H$, Jacobi’s formula gives $D\det (\mathrm{\Sigma })[H]=\det (\mathrm{\Sigma })\mathrm{tr}({\mathrm{\Sigma }}^{-1}H)$ by The determinant differential is $D\det (A)[H]=\mathrm{tr}(\mathrm{adj}(A)H)$ at every matrix, and Jacobi's formula holds on the invertible locus. This differentiates the determinant factor only; this definition assumes no differentiability of the binding mean or covariance.

A positive-definite matrix has the unique Cholesky factor used in numerical evaluation by A matrix admits a Cholesky factorisation with positive diagonal exactly when it is Hermitian positive definite, and that factor is unique. This factorization does not ensure that a finite-simulation covariance is positive definite.

Let $\mathsf{E}=((\mathcal{X},\mathcal{A}),\mathrm{\Theta },(P_{\theta }{)}_{\theta \in \mathrm{\Theta }})$ be a statistical experiment and let $S:\mathcal{X}\to {\mathbb{R}}^d$ be a statistic, meaning a measurable map to its declared target as in Statistics and estimators, with finite second moment under each $P_{\theta }$. Expectations are integrated against the law as in Law or distribution of a random element and Expectation of a nonnegative or integrable random variable, and covariance uses the centered second moment as in Moments, variance, and covariance on a probability space. The moment binding function for this page is $b:\mathrm{\Theta }\to {\mathbb{R}}^d,b(\theta ):={\mathbb{E}}_{\theta }[S]\mathrm{.}$ The name refers here to the mean of the declared summary vector; classical indirect inference also uses bindings defined by auxiliary-model estimators, which are a different interface.

The summary-law map is itself a parametric model in the sense of Parametric statistical models: $Q:\mathrm{\Theta }\to \mathcal{P}({\mathbb{R}}^d),Q_{\theta }(B):=P_{\theta }(\{x:S(x)\in B\})$ for each Borel set $B\subseteq {\mathbb{R}}^d$. It records the full law of the summary, whereas $b$ records only its mean.

For the finite-dimensional minimum-distance target, assume $p\ge 1$, a nonempty compact parameter set $\mathrm{\Theta }\subseteq {\mathbb{R}}^p$ with its Euclidean subspace metric (Open cover, subcover, compact metric space, and compact subset of a metric space Metric space: $d(x,y)=0$ iff $x=y$, symmetry, and the triangle inequality; pseudometric and ultrametric), a binding map $b$ continuous from this metric to the Euclidean metric on ${\mathbb{R}}^d$ (Continuity of a map between metric spaces, at a point and globally, in the $\epsilon$-$\delta$ form), and a fixed summary vector $s\in {\mathbb{R}}^d$ (either the observed summary or a specified deterministic probability limit). Define $q_s(\theta ):=\mathrm{\parallel }b(\theta )-s{\mathrm{\parallel }}_2^2\mathrm{.}$ Here $\mathrm{\parallel }\cdot {\mathrm{\parallel }}_2$ is the Euclidean norm. Require $q_s:\mathrm{\Theta }\to \mathbb{R}$ to be continuous as part of this target’s admissible input contract. Its minimizer set is $M_s:={\mathrm{arg\hspace{0.17em}min}}_{\theta \in \mathrm{\Theta }}{q}_s(\theta )=\{\theta \in \mathrm{\Theta }:q_s(\theta )={\inf }_{\eta \in \mathrm{\Theta }}{q}_s(\eta )\}\mathrm{.}$ The indirect-inference target ${\theta }^{\mathrm{I}\mathrm{I}}(s)$ is the lexicographically least member of $M_s$: among all minimizers, minimize the first coordinate; among those, minimize the second; continue through coordinate $p$. The target is well-defined under exactly the stated compactness and continuity contract by The compact binding target exists and has a unique lexicographic tie-break ↗.

For moment-based parameter recovery, the proposed limiting summary must satisfy $s=b({\theta }_0)$ for some ${\theta }_0\in \mathrm{\Theta }$. The moment-identification condition at ${\theta }_0$ is $\{\theta \in \mathrm{\Theta }:b(\theta )=b({\theta }_0)\}=\{{\theta }_0\}\mathrm{.}$ Global moment identification means that $b$ is injective. Identifiability of $b$ as an estimand in Identifiability of an estimand is weaker: equality of sampling laws already implies equality of these means and does not ensure parameter recovery.

If the method instead compares the full summary laws, parameter identification requires injectivity of $\theta \mapsto Q_{\theta }$. By Identifiability of an estimand and Identifiability is constancy on observational classes, this is exactly identifiability of the identity estimand in the summary experiment. These identification conditions are additional assumptions for parameter recovery; the minimum-distance target and its tie rule do not require them.

No consistency theorem for an estimated minimizer follows from this target definition. Such a theorem needs additional hypotheses controlling sampling, simulation error, separation of the population target and the optimizer; the plan does not assert those hypotheses here.

The BCS-20 prerequisite page introduces numerical optimization procedures. Their iterates are computational outputs; this target remains the exact argmin with the tie rule specified above. No convergence of a numerical optimization routine to this target is claimed.

For any two simulated vectors $z_1,z_2\in {\mathbb{R}}^2$, put $\bar{z}=(z_1+z_2)\mathrm{/}2$ and define the Bessel-corrected sample covariance $\widehat{\mathrm{\Sigma }}:={\sum }_{i=1}^2(z_i-\bar{z})(z_i-\bar{z}{)}^T\mathrm{.}$ Then $\widehat{\mathrm{\Sigma }}$ is not positive definite.

Let $\mathrm{\Theta }\subseteq {\mathbb{R}}^p$ be the nonempty compact parameter set of Binding function and indirect-inference target, and fix an admissible $s$ for which its objective $q_s:\mathrm{\Theta }\to \mathbb{R}$ is continuous. Then $M_s:={\mathrm{arg\hspace{0.17em}min}}_{\theta \in \mathrm{\Theta }}{q}_s(\theta )$ is a nonempty compact set and has a unique lexicographically least element. Thus the target selector specified in Binding function and indirect-inference target exists.

Assume AC only as required by the synthetic-likelihood definition and its Gaussian-density supplier (The Axiom of Choice Ideal and finite-simulation synthetic likelihood A positive-definite multivariate normal law has its Gaussian density). IID means common law and mutual independence as in Identical distribution and IID families. This algorithm states the vector sample mean and covariance directly; covariance uses centered second moments as in Moments, variance, and covariance on a probability space.

Inputs: A parameter $\theta$, observed data $y$, a summary statistic $S:\mathcal{X}\to {\mathbb{R}}^d$ satisfying the contract of Ideal and finite-simulation synthetic likelihood, an integer simulation count $m\ge 2$, and a simulator that returns $m$ mutually independent observations with common law $P_{\theta }$.

Output: Either the finite-simulation plug-in synthetic-likelihood value at $S(y)$, or the explicit result unavailable.

1. Simulate $Y_1,\dots ,Y_m$ and compute $S_i:=S(Y_i)$. For each coordinate $j$, compute ${\bar{S}}_{m,j}:=m^{-1}{\sum }_{i=1}^m{S}_{i,j}$.
2. Set ${\bar{S}}_m:=({\bar{S}}_{m,1},\dots ,{\bar{S}}_{m,d}{)}^T$ and compute ${\widehat{\mathrm{\Sigma }}}_m:=(m-1{)}^{-1}{\sum }_{i=1}^m(S_i-{\bar{S}}_m)(S_i-{\bar{S}}_m{)}^T\mathrm{.}$
3. If ${\widehat{\mathrm{\Sigma }}}_m$ is not positive definite, stop and return unavailable. Do not add a ridge term or otherwise modify the covariance.
4. If ${\widehat{\mathrm{\Sigma }}}_m$ is positive definite, take its unique Cholesky factor $L$ with positive diagonal by A matrix admits a Cholesky factorisation with positive diagonal exactly when it is Hermitian positive definite, and that factor is unique. Return the plug-in Gaussian density $(2\pi {)}^{-d\mathrm{/}2}\det ({\widehat{\mathrm{\Sigma }}}_m{)}^{-1\mathrm{/}2}\exp \text{\hspace{0.17em}⁣}[-\frac{1}{2}(S(y)-{\bar{S}}_m{)}^T{\widehat{\mathrm{\Sigma }}}_m^{-1}(S(y)-{\bar{S}}_m)]$ from Ideal and finite-simulation synthetic likelihood and A positive-definite multivariate normal law has its Gaussian density.

This algorithm evaluates one parameter value. It does not optimize over $\theta$ and asserts no unbiasedness, convergence, consistency, posterior calibration, or optimizer property.

Under AC for the Gaussian-density interface (The Axiom of Choice), the finite-simulation mean, covariance, and returned plug-in value depend on the simulated summaries. The approximation may be unavailable: the local lemma A two-simulation covariance in two dimensions is singular shows that with two two-dimensional summaries the covariance is never positive definite, so Finite-simulation plug-in synthetic-likelihood evaluation returns unavailable. A companion example gives two positive-probability simulation outcomes with different covariances (Two simulated square summaries give a random singular covariance ↗).

The ideal Gaussian summary density is not asserted to be the exact model likelihood. No unbiasedness or consistency claim is made for the finite plug-in quantity.