Bounded posterior functionals and stationary sample error

Let $(E,\mathcal{E})$ be a measurable space, let $\mu ,\nu$ be probability measures on it, and let $f:E\to \mathbb{R}$ be measurable with $\mathrm{\mid }f\mathrm{\mid }\le M$ for some $0\le M<\mathrm{\infty }$. Use the library convention $\mathrm{\parallel }\mu -\nu {\mathrm{\parallel }}_{\mathrm{T}\mathrm{V}}:={\sup }_{A\in \mathcal{E}}\mathrm{\mid }\mu (A)-\nu (A)\mathrm{\mid }\mathrm{.}$ Then $\mathrm{\mid }\mu (f)-\nu (f)\mathrm{\mid }\le 2M\mathrm{\parallel }\mu -\nu {\mathrm{\parallel }}_{\mathrm{T}\mathrm{V}}\mathrm{.}$

For the complete posterior computation error budget defined above, the pathwise total error satisfies $\mathrm{\mid }{\mu }_{\star }(f)-{\bar{f}}_{b,N}\mathrm{\mid }\le {\mathcal{B}}_f\mathrm{.}$ This is a telescoping bound; it does not estimate any component.

Separately, let $(X_n{)}_{n\ge 0}$ be a stationary time-homogeneous Markov chain with transition kernel $K$, invariant law ${\pi }_K$, and initial law $X_0\sim {\pi }_K$, and use the same bounded functional $f$. Put ${\gamma }_k:=\mathrm{Cov}(f(X_0),f(X_k))(k\ge 0)\mathrm{.}$ If ${\sum }_{k=1}^{\mathrm{\infty }}\mathrm{\mid }{\gamma }_k\mathrm{\mid }<\mathrm{\infty }$, then for every burn-in $b\ge 0$ and sample size $N\ge 1$, $\mathbb{E}[\mathrm{\mid }{\bar{f}}_{b,N}-{\pi }_K(f){\mathrm{\mid }}^2]=\frac{1}{N^2}[N{\gamma }_0+2{\sum }_{k=1}^{N-1}(N-k){\gamma }_k]\le \frac{{\gamma }_0+2{\sum }_{k=1}^{\mathrm{\infty }}\mathrm{\mid }{\gamma }_k\mathrm{\mid }}{N}\mathrm{.}$ The sample bound assumes stationarity; for a nonstationary start, the budget keeps $\mathrm{\mid }{\pi }_K(f)-{\mathbb{E}}_{\lambda }{\bar{f}}_{b,N}\mathrm{\mid }$ as a separate burn-in term. This finite-$N$ variance bound is not a central limit theorem or a confidence interval.

The six adjacent posterior, solver, discretization, kernel, burn-in, and Monte Carlo error terms.

Total variation is the event supremum and has no factor $1\mathrm{/}2$ in that convention.

With this convention, a measurable function bounded by $M$ has expectation difference at most $2M$ times total variation.

An invariant law satisfies $\pi K=\pi$.

The one-step conditional law is given by the transition kernel.

Strict stationarity means all finite-dimensional laws are unchanged by time shifts.

Variance and covariance are expectations of centered products.

A finite scalar reference chain telescopes into its adjacent signed errors.

Gibbs and Su, §2, total-variation dual form

Levin, Peres, and Wilmer, §4.1, Proposition 4.5

Geyer, §1.1, stationary sample-mean variance

An ESS is functional-specific and is not a substitute for a Monte Carlo standard error. The variance bound in this Proposition is a population statement under its chain assumptions; it does not validate a finite-chain empirical ESS estimate.

Vehtari et al., §3.2, functional-specific ESS and MCSE