Sub-Gaussian mixture confidence sequences

Given: Assume AC, $\rho >0$, $v\ge 0$, and $s\in \mathbb{R}$.

Standard normal and normal laws: $N(m,{\sigma }^2)$ is the law of the affine image $z\mapsto m+\sigma z$ of the standard normal law, whose density is $e^{-z^2\mathrm{/}2}\mathrm{/}\sqrt{2\pi }$.

Change of variables for expectation: Integrating a nonnegative measurable function against a random element's law equals its expectation on the underlying space.

The standard normal density has total mass one: The standard normal density $e^{-z^2\mathrm{/}2}\mathrm{/}\sqrt{2\pi }$ integrates to one.

A C^1 diffeomorphism satisfies the change-of-variables formula for nonnegative Lebesgue measurable functions: If $T:U\to V$ is a $C^1$ diffeomorphism between open subsets of ${\mathbb{R}}^n$ and $f:V\to [0,\mathrm{\infty }]$ is Lebesgue measurable, then, assuming countable choice, ${\int }_{V}f(y)d{\lambda }_n(y)={\int }_{U}f(T(x))\mathrm{\mid }\det DT(x)\mathrm{\mid }d{\lambda }_n(x)\mathrm{.}$

Continuous functions on Euclidean spaces are Borel measurable and Assuming countable choice, every Borel subset of ${\mathbb{R}}^n$ is Lebesgue measurable: Under countable choice, every continuous function on $\mathbb{R}$ is Borel measurable, and every Borel set is Lebesgue measurable.

Square roots exist: a unique $\sqrt{a}\ge 0$ with $(\sqrt{a}{)}^2=a$; the positives are $\{x^2:x\mathrm{\ne }0\}$: Every nonnegative real has a unique nonnegative square root.

The Axiom of Choice: AC asserts that every family of nonempty sets has a choice function; it is assumed here.

The Axiom of Countable Choice (${\mathrm{A}\mathrm{C}}_{\omega }$): AC implies countable choice, so the countable-choice premise of [F4] follows from the assumed AC.

Given: Assume AC, the conditional sub-Gaussian model, and a fixed real $\lambda$.

Conditionally sub-Gaussian increments: for each $t\ge 1$ and fixed $\lambda$, $e^{\lambda (X_t-\mu )}$ is integrable and its conditional expectation given ${\mathcal{F}}_{t-1}$ is at most $e^{{\lambda }^2{\sigma }^2\mathrm{/}2}$ almost surely.

Taking out what is known: a bounded measurable factor can be taken out of a conditional expectation; the identity also holds for finite measurable factors when both products are integrable.

Monotone convergence for the integral: an increasing sequence of nonnegative measurable functions has integrals increasing to the integral of its pointwise limit.

Martingale submartingale and supermartingale: an adapted integrable process is a supermartingale when $\mathbb{E}[L_t\mid {\mathcal{F}}_{t-1}]\le L_{t-1}$ almost surely at every adjacent pair.

Nonnegative test supermartingales: under AC, an adapted, integrable, almost surely nonnegative supermartingale with initial expectation at most one is a nonnegative test supermartingale.

The Axiom of Choice: AC supplies the conditional-expectation convention used in the supermartingale and test-supermartingale definitions.

Given: $\alpha \in (0,1)$, $\rho >0$, $v\ge 0$, and $s\in \mathbb{R}$.

Gaussian-mixture confidence-sequence boundary: $u_{\alpha }(v)=\sqrt{(v+\rho )\log ((v+\rho )\mathrm{/}({\alpha }^2\rho ))}$.

Order, continuity, range, and the product, quotient, and reciprocal laws for the natural logarithm: the natural logarithm is strictly increasing on $(0,\mathrm{\infty })$ and satisfies $\log (xy)=\log x+\log y$ for positive $x,y$.

Square roots exist: a unique $\sqrt{a}\ge 0$ with $(\sqrt{a}{)}^2=a$; the positives are $\{x^2:x\mathrm{\ne }0\}$: every nonnegative real has a unique nonnegative square root.

Given: Assume AC and the conditional sub-Gaussian model for fixed $\mu$, ${\sigma }^2>0$, and $\rho >0$.

Conditional sub-Gaussian increments generate exponential test supermartingales: for each fixed $\lambda \in \mathbb{R}$, $L_t^{\lambda }(\mu )=\exp (\lambda S_t(\mu )-{\lambda }^{2}t{\sigma }^2\mathrm{/}2)$ is a nonnegative test supermartingale with initial value one.

Mixtures of null test supermartingales: if each component is a nonnegative test supermartingale under every null law and $(\lambda ,\omega )\mapsto L_t^{\lambda }(\omega )$ is product-measurable, its probability mixture is a test supermartingale; the theorem uses a real-valued modification only where the integral is infinite.

Tonelli's theorem for nonnegative measurable functions on a sigma-finite product: for sigma-finite measure spaces and nonnegative product-measurable functions, section integrals are measurable and equal the corresponding iterated integrals.

Closed form for a Gaussian exponential mixture: for $v\ge 0$, $s\in \mathbb{R}$, and $\rho >0$, the mixture of $\exp (\lambda s-v{\lambda }^2\mathrm{/}2)$ under $N(0,1\mathrm{/}\rho )$ equals $\sqrt{\rho \mathrm{/}(v+\rho )}\exp (s^2\mathrm{/}[2(v+\rho )])$.

Standard normal and normal laws: $N(0,1\mathrm{/}\rho )$ is a probability law under the affine-image normal-law convention.

The Axiom of Choice: AC is required by the conditional-expectation and test-supermartingale conventions in [F1] and [F2].

Given: Assume AC and all data, law, filtration, proxy, and tuning hypotheses in the Statement.

Conditionally sub-Gaussian increments: for each fixed $\theta$, time $t\ge 1$, and $\lambda \in \mathbb{R}$, the conditional mgf of $X_t-\theta$ given ${\mathcal{F}}_{t-1}$ is at most $e^{{\lambda }^2{\sigma }^2\mathrm{/}2}$ almost surely.

The Gaussian mixture is a sub-Gaussian test supermartingale: under each $P_{\theta }$, $M_t(\theta )=\sqrt{\rho \mathrm{/}(t{\sigma }^2+\rho )}\exp (S_t(\theta {)}^2\mathrm{/}[2(t{\sigma }^2+\rho )])$ is a nonnegative test supermartingale with $M_0(\theta )=1$.

The Gaussian-mixture threshold is the stated square-root boundary: $M_t(\theta )\ge 1\mathrm{/}\alpha$ exactly when $\mathrm{\mid }{S}_t(\theta )\mathrm{\mid }\ge u_{\alpha }(t{\sigma }^2)$.

Discrete Ville inequality and anytime threshold validity: under AC, for every null law and each $\alpha \in (0,1)$, a nonnegative test supermartingale has $P({\sup }_{t\ge 0}{M}_t\ge 1\mathrm{/}\alpha )\le \alpha$.

Confidence sequences: a $(1-\alpha )$ confidence sequence has ${\mathcal{F}}_t$-measurable membership events and covers each true parameter at all integer times with probability at least $1-\alpha$.

The Axiom of Choice: AC supplies the conditional-expectation and test-supermartingale conventions used in [F2] and [F4].