Finite-state continuous-time Markov jump processes with bounded rates

Fix a finite nonempty set of states $E$, equipped with the discrete sigma-algebra $\mathcal{P}(E)$ consisting of all subsets of $E$, and a finite time horizon $T\ge 0$. A finite-state continuous-time Markov jump process on $[0,T]$ is a family of measurable maps $X_t:(\mathrm{\Omega },\mathcal{F})\to (E,\mathcal{P}(E))$ on a probability space $(\mathrm{\Omega },\mathcal{F},\mathbb{P})$. Thus each $X_t$ is a random element of this measurable state space. Its paths are constant between successive jump times and take the new state at a jump time (right-continuity); path events use the sigma-algebra generated by the coordinate maps $X_t$. The Markov property here means that, conditional on the history through time $s$, the law of the future path depends on that history only through $(s,X_s)$. For each pair of distinct states $x,y\in E$, its transition rate ${\kappa }_{xy}:[0,T]\to [0,\mathrm{\infty })$ is a bounded continuous function. Write

${\lambda }_x(t):={\sum }_{y\in E,y\mathrm{\ne }x}{\kappa }_{xy}(t)\mathrm{.}$

All time integrals without a displayed measure below are oriented Riemann integrals. When a continuous integrand is used as a Lebesgue density, its Lebesgue and Riemann integrals agree by the bounded Riemann-integral comparison theorem. The first-jump rate convention is the following. Conditional on the process history through time $s$ and state $X_s=x$, the survival probability of no jump before time $t$, $s\le t\le T$, is

$H_x(s,t):=\exp (-{\int }_s^t{\lambda }_x(u)du)\mathrm{.}$

For each $y\mathrm{\ne }x$, the conditional density that the next jump occurs at time $t$ and goes to $y$ is $H_x(s,t){\kappa }_{xy}(t)$ with respect to one-dimensional Lebesgue measure; the remaining probability $H_x(s,T)$ is that no jump occurs by time $T$. These are normalized conditional probabilities: for $s<T$, by the chain rule and the first fundamental theorem of calculus, $H_x(s,s)=1$ and $H_x^{\mathrm{\prime }}(s,t)=-{\lambda }_x(t)H_x(s,t)$; the density integrands are continuous, so the second fundamental theorem of calculus gives

$H_x(s,T)+{\sum }_{y\mathrm{\ne }x}{\int }_s^T{H}_x(s,t){\kappa }_{xy}(t)dt=H_x(s,T)+{\int }_s^T{\lambda }_x(t)H_x(s,t)dt=1.$

After each jump the same conditional rule restarts from the new state and jump time. For $s=T$, the next-jump law has unit mass on no further jump. The row generator is the matrix $Q(t)$ with $Q_{xy}(t)={\kappa }_{xy}(t)$ for $x\mathrm{\ne }y$ and $Q_{xx}(t)=-{\lambda }_x(t)$. A time-homogeneous process is the special case in which every ${\kappa }_{xy}(t)$ is constant. When rates carry physical time units, ${\kappa }_{xy}$ has units of inverse time; this definition is mathematical and asserts no physical realization. The process is called nonexplosive on $[0,T]$ when its path has finitely many jumps there almost surely.

Probability measures and probability spaces

Random elements and real random variables

The Axiom of Countable Choice (${\mathrm{A}\mathrm{C}}_{\omega }$)

Lebesgue measurable sets, the family $\mathcal{L}({\mathbb{R}}^n)$, and the restricted set function ${\lambda }_n$

Assuming countable choice, $\mathcal{L}({\mathbb{R}}^n)$ is a sigma-algebra containing every elementary set and ${\lambda }_n$ is a complete measure extending elementary volume

The integral with oriented limits: ${\int }_a^{a}f:=0$ and ${\int }_b^{a}f:=-{\int }_a^{b}f$

The first fundamental theorem: if $f$ is integrable on $[a,b]$ and continuous at $c$, then $F^{\mathrm{\prime }}(c)=f(c)$; in particular a continuous $f$ has $F$ as a primitive

The second fundamental theorem: if $G$ is differentiable on $[a,b]$ with $G^{\mathrm{\prime }}=f$ and $f$ is integrable, then ${\int }_a^{b}f=G(b)-G(a)$

The exponential function is smooth and $(\exp {)}^{\mathrm{\prime }}=\exp$

The chain rule, in one line from Carathéodory: if $g$ is differentiable at $c$ and $f$ is differentiable at $g(c)$, then $f\circ g$ is differentiable at $c$ with $(f\circ g{)}^{\mathrm{\prime }}(c)=f^{\mathrm{\prime }}(g(c))g^{\mathrm{\prime }}(c)$

A bounded Riemann integrable function on a closed bounded interval is Lebesgue measurable and has the same integral