Approximate Bayesian computation and summary-based inference

Let $(\mathrm{\Theta },\mathcal{T})$ and $(\mathsf{Y},\mathcal{Y})$ be standard Borel spaces. Let $\mathrm{\Pi }$ be a prior probability measure on $(\mathrm{\Theta },\mathcal{T})$ and let $K$ be a probability kernel from $(\mathrm{\Theta },\mathcal{T})$ to $(\mathsf{Y},\mathcal{Y})$, as in Bayesian model as a prior and sampling kernel. Let $S:\mathsf{Y}\to {\mathbb{R}}^d$ be a measurable statistic for a fixed integer $d\ge 1$, and fix an observed summary $s_0\in {\mathbb{R}}^d$. The Euclidean norm is $\mathrm{\parallel }x{\mathrm{\parallel }}_2=({\sum }_{j=1}^d{x}_j^2{)}^{1\mathrm{/}2}$.

Under the prior-simulator joint law, first draw $\theta \sim \mathrm{\Pi }$ and then $Y\sim K(\theta ,\cdot )$. The summary-predictive probability measure $M$ on $\mathrm{\Theta }\times {\mathbb{R}}^d$ is the law of $(\theta ,S(Y))$; on measurable rectangles it is $M(A\times C)={\int }_{A}K(\theta ,S^{-1}(C))\mathrm{\Pi }(d\theta )\mathrm{.}$ The prior and simulator define a joint law by A prior and model kernel define a joint law, and the change-of-variables formula identifies $M$ as the law after applying $S$ (Change of variables for expectation).

For $ϵ\ge 0$, define the closed tolerance ball $B_{ϵ}(s_0)=\{s\in {\mathbb{R}}^d:\mathrm{\parallel }s-s_0{\mathrm{\parallel }}_2\le ϵ\}\mathrm{.}$ Its prior-predictive acceptance probability is $p_{ϵ}(s_0):=M(\mathrm{\Theta }\times B_{ϵ}(s_0))\in [0,1]\mathrm{.}$ Only when $p_{ϵ}(s_0)>0$ define the indicator ABC target by ${\mathrm{\Pi }}_{ϵ}(A\mid s_0):=\frac{M(A\times B_{ϵ}(s_0))}{p_{ϵ}(s_0)}(A\in \mathcal{T})\mathrm{.}$ This is exactly the conditional law of $\theta$ under the augmented event $S(Y)\in B_{ϵ}(s_0)$. The denominator is finite because $M$ is a probability, and no quotient is assigned when the event has probability zero. This target is the posterior given a tolerance event, not in general the full-data posterior or the posterior given the exact summary.

A positive tolerance separates summary information loss from tolerance bias; using finitely many accepted draws adds a separate Monte Carlo error. At a null summary value, a regular-conditional version may be changed there without affecting its defining identities, so its value is not determined by conditioning alone. The density regularity in ABC targets the density-defined summary posterior at zero tolerance selects the particular density-based target obtained as the positive-tolerance limit.

The inputs are a proper prior $\mathrm{\Pi }$ on $(\mathrm{\Theta },\mathcal{T})$, a simulator probability kernel $K(\theta ,dy)$, a measurable summary statistic $S:\mathsf{Y}\to {\mathbb{R}}^d$, an observed summary $s_0$, and a tolerance $ϵ\ge 0$ as in Approximate Bayesian computation and the tolerance-conditioned target. The procedure is admissible when $p_{ϵ}(s_0)>0$.

1. Draw a candidate parameter $\theta$ from $\mathrm{\Pi }$.
2. Conditional on $\theta$, draw synthetic data $Y$ from $K(\theta ,\cdot )$, independently of all earlier proposal-simulator pairs.
3. Compute $S(Y)$. If $\mathrm{\parallel }S(Y)-s_0{\mathrm{\parallel }}_2\le ϵ$, return $\theta$ and stop. Otherwise repeat from step 1 with a fresh independent pair.

The output is one accepted parameter value. It does not itself assert a finite-run estimate or an accuracy bound for an empirical distribution of accepted values.

Use the model and notation of Approximate Bayesian computation and the tolerance-conditioned target and the procedure ABC rejection sampling. Let $B=B_{ϵ}(s_0)$ and $p=M(\mathrm{\Theta }\times B)>0$. Suppose the proposals and simulator outputs form an IID sequence $({\mathrm{\Theta }}_j,Y_j{)}_{j\ge 1}$ with the prior-simulator joint law. Then the first accepted index $N:=\inf \{j\ge 1:S(Y_j)\in B\}$ is finite almost surely, and the accepted parameter has law $\mathbb{P}({\mathrm{\Theta }}_N\in A)=\frac{M(A\times B)}{p}={\mathrm{\Pi }}_{ϵ}(A\mid s_0)(A\in \mathcal{T})\mathrm{.}$ Thus the accepted parameter has exactly the tolerance-conditioned target law. This statement does not identify that target with the full-data posterior or assert an expected proposal count.

Assume the Axiom of Countable Choice. In the proper prior-simulator model of Approximate Bayesian computation and the tolerance-conditioned target, let $d\ge 1$, $s_0\in {\mathbb{R}}^d$, and suppose the summary-predictive measure has a specified jointly measurable density version $M(d\theta ,ds)=m_{\theta }(s)\mathrm{\Pi }(d\theta ){\lambda }_d(ds),$ where ${\lambda }_d$ is Lebesgue measure on ${\mathbb{R}}^d$ (Lebesgue measurable sets, the family $\mathcal{L}({\mathbb{R}}^n)$, and the restricted set function ${\lambda }_n$). Assume that $s\mapsto m_{\theta }(s)$ is continuous at $s_0$ for $\mathrm{\Pi }$-almost every $\theta$, that a measurable $H:\mathrm{\Theta }\to [0,\mathrm{\infty }]$ satisfies $m_{\theta }(s)\le H(\theta )\text{for }\mathrm{\Pi }\text{-almost every }\theta \text{ and every }\mathrm{\parallel }s-s_0{\mathrm{\parallel }}_2\le {ϵ}_0$ for some ${ϵ}_0>0$ and $\int Hd\mathrm{\Pi }<\mathrm{\infty }$, and that $0<z_0:={\int }_{\mathrm{\Theta }}{m}_{\theta }(s_0)\mathrm{\Pi }(d\theta )<\mathrm{\infty }\mathrm{.}$ Define the density-selected summary posterior ${\mathrm{\Pi }}_{s_0}^m(A):=\frac{{\int }_A{m}_{\theta }(s_0)\mathrm{\Pi }(d\theta )}{z_0}\mathrm{.}$ Then, for all sufficiently small $ϵ>0$ for which the target is defined, the indicator ABC target ${\mathrm{\Pi }}_{ϵ}(\cdot \mid s_0)$ converges to ${\mathrm{\Pi }}_{s_0}^m$ in total variation as $ϵ\downarrow 0$.

The pointwise density version is part of the statement. The conclusion is the density-selected summary update; it does not claim that every regular conditional distribution has that value at a null summary value.

For an atomless summary, the event $S(Y)=s_0$ can have prior-predictive probability zero. In that case conditional-law identities do not choose a unique posterior value at $s_0$; the specified density version above selects one. The envelope is a substantive hypothesis: without an integrable local bound, moving density spikes can prevent total-variation convergence, as shown by Pointwise continuity alone does not guarantee the zero-tolerance limit ↗.

Assume AC. Let $(\mathrm{\Theta },\mathcal{T})$, $(\mathsf{Y},\mathcal{Y})$, and $(\mathsf{T},\mathcal{C})$ be standard Borel spaces (Standard Borel spaces), let $\mathrm{\Pi }$ be a proper prior, and let $K$ be a sampling probability kernel as in Bayesian model as a prior and sampling kernel. Let $S:\mathsf{Y}\to \mathsf{T}$ be a measurable statistic (Statistics and estimators) that is sufficient in the common-kernel sense of Sufficient statistics. Write $K_S(\theta ,C)=K(\theta ,S^{-1}(C))$, let $M$ be the joint prior-predictive law of $(\theta ,S(Y))$, and let $J_Y$ and $J_S$ be the data and summary marginals. A posterior kernel ${\mathrm{\Pi }}_t^S$ of $\theta$ given $S(Y)=t$ exists. Then ${\widehat{\mathrm{\Pi }}}_y(A):={\mathrm{\Pi }}_{S(y)}^S(A)$ is a data-indexed posterior version for the full observation: for every $A\in \mathcal{T}$ and $D\in \mathcal{Y}$, $J(\mathrm{\Theta }\in A,Y\in D)={\int }_D{\widehat{\mathrm{\Pi }}}_y(A)J_Y(dy)\mathrm{.}$ Thus full-data posterior information is preserved by a sufficient summary, up to the usual almost-everywhere version convention.

In a dominated branch, suppose $\mu$ and $\nu$ are sigma-finite reference measures for the full data and summary, respectively, and the specified jointly measurable densities satisfy $K(\theta ,dy)=p_{\theta }(y)\mu (dy),K_S(\theta ,dt)=h_{\theta }(t)\nu (dt),p_{\theta }(y)=g(y)h_{\theta }(S(y))$ on the specified common versions, with $g:\mathsf{Y}\to [0,\mathrm{\infty }]$ independent of $\theta$. For any $y$ where the factorization holds $\mathrm{\Pi }$-almost everywhere, $0<g(y)<\mathrm{\infty }$, and $0<Z_S(S(y)):={\int }_{\mathrm{\Theta }}{h}_{\theta }(S(y))\mathrm{\Pi }(d\theta )<\mathrm{\infty },$ the explicit full-data and summary posterior updates agree at $y$ by cancellation of $g(y)$. No equality is asserted on zero or infinite evidence fibres.