Path entropy production, fluctuation symmetry and work identities

Conditional on state $x_0$ at time $0$, the first-jump convention in [F1] gives the survival factor $H_{x_0}(0,t_1)$ and jump density $H_{x_0}(0,t_1){\kappa }_{x_0{x}_1}(t_1)$. After this jump, the Markov rule restarts at $x_1$. Multiplying the successive conditional survival and jump factors through state $x_n$, and then the final no-jump survival on $[t_n,T]$, gives exactly the stated path density. If $n=0$, the same convention gives $\rho (x_0)H_{x_0}(0,T)$, which is the stated boundary formula.

Let $J=[s,t]$ have length $\delta$ with $M\delta <1$. The density of the first $n$ ordered jumps, with survival factors removed, is at most the product of the $n$ rates. Summing these products over each successive state gives at most $M^n$. The ordered time simplex is contained in the cube $J^n$, whose Lebesgue volume is ${\delta }^n$ by [F2]. Hence $\mathbb{P}(N_J\ge n)\le (M\delta {)}^n$. Pointwise $N_J={\sum }_{n\ge 1}{\mathbf{1}}_{\{N_J\ge n\}}$; [F4] gives $\mathbb{E}[N_J]\le {\sum }_{n\ge 1}(M\delta {)}^n=M\delta \mathrm{/}(1-M\delta )$.

If $T=0$ or $M=0$, there are no jumps. Otherwise choose a positive integer $m>MT$ by [F3] and partition $[0,T]$ into $m$ intervals of length $\delta =T\mathrm{/}m$, so $M\delta <1$. Jump times have densities by step 1.1, hence no jump occurs at a fixed partition endpoint with positive probability. The total count is the sum of the interval counts almost surely; linearity of expectation [F4] and step 1.2 give $\mathbb{E}[N_T]\le mM\delta \mathrm{/}(1-M\delta )<\mathrm{\infty }$. If $\mathbb{P}(N_T=\mathrm{\infty })>0$, then $\mathbb{E}[N_T]\ge n\mathbb{P}(N_T=\mathrm{\infty })$ for every $n$, contradicting finiteness. Thus the process is nonexplosive and the countable path-density collection covers the path law.

If $T=0$, the only time is $t=0$ and $p(0)$ is its initial law. Suppose $T>0$. By the first-jump convention, the no-jump probability from $x$ over $[t,t+h]$ is $H_x(t,t+h)=\exp (-{\int }_t^{t+h}{\lambda }_x(s)ds)=1-{\lambda }_x(t)h+o(h)$. For $x\mathrm{\ne }y$, the probability of exactly one jump $x\to y$ and no further jump is ${\int }_t^{t+h}{H}_x(t,s){\kappa }_{xy}(s)H_y(s,t+h)ds={\kappa }_{xy}(t)h+o(h)$. Paths with at least two jumps contribute $O(h^2)$ by [F2]. Hence the conditional endpoint transition probabilities equal ${\kappa }_{xy}(t)h+o(h)$ for $y\mathrm{\ne }x$, and $1-{\lambda }_x(t)h+o(h)$ for $y=x$. Averaging this expansion over the finite states gives $\mathrm{\mid }{p}_y(t+h)-p_y(t)\mathrm{\mid }=O(h)$; the same estimate from the left shows $p$ is continuous. By the Markov property, for each $y\in E$, $p_y(t+h)=p_y(t)(1-{\lambda }_y(t)h)+{\sum }_{x\mathrm{\ne }y}{p}_x(t){\kappa }_{xy}(t)h+o(h)\mathrm{.}$ There are finitely many states, so the remainder remains $o(h)$ after summation. Dividing by $h$ and taking $h\downarrow 0$ gives the stated forward equation; applying the same expansion at $t-h$ gives the left derivative. Its right-hand side is continuous because the rate functions and each $p_x$ are continuous, hence $p$ is continuously differentiable on the interior, with the corresponding one-sided derivatives at the endpoints.

Suppose $\pi Q(t)=0$ and $p(0)=\pi$. The constant function $p(t)=\pi$ satisfies the forward equation. If $d(t)$ is the difference of any other solution and $\pi$, then $d(0)=0$ and $d^{\mathrm{\prime }}=dQ$. For $t>0$, each component of $d$ is differentiable on $[0,t]$, with one-sided endpoint derivatives, and its derivative is continuous, hence integrable. Applying the second fundamental theorem in [F4] componentwise gives $d(t)={\int }_0^{t}d(s)Q(s)ds\mathrm{.}$ For $t=0$ this identity holds by the zero-interval convention. Use the vector $1$-norm $\mathrm{\parallel }d{\mathrm{\parallel }}_1:={\sum }_x\mathrm{\mid }{d}_x\mathrm{\mid }$ and let $B:={\sup }_{s\in [0,T]}{\max }_x{\sum }_y\mathrm{\mid }{Q}_{xy}(s)\mathrm{\mid }<\mathrm{\infty }$, the maximum absolute row sum. Let $D:={\sup }_{s\in [0,T]}\mathrm{\parallel }d(s){\mathrm{\parallel }}_1<\mathrm{\infty }$. Since $d$ is a row vector, the triangle inequality gives $\mathrm{\parallel }d(s)Q(s){\mathrm{\parallel }}_1\le {\sum }_x\mathrm{\mid }{d}_x(s)\mathrm{\mid }{\sum }_y\mathrm{\mid }{Q}_{xy}(s)\mathrm{\mid }\le B\mathrm{\parallel }d(s){\mathrm{\parallel }}_1$. Iterating the integral equation yields $\mathrm{\parallel }d(t){\mathrm{\parallel }}_1\le D(Bt{)}^n\mathrm{/}n!$ for every integer $n\ge 1$. This bound tends to zero as $n\to \mathrm{\infty }$, so $d(t)=0$ for every $t$. Therefore $p(t)=\pi$.

In the constant-rate case, $\pi Q=0$ is exactly the coordinate condition ${\sum }_{x\mathrm{\ne }y}{\pi }_x{\kappa }_{xy}={\pi }_y{\lambda }_y$ for each $y$, so step 2.1 proves invariance of the stated generator-stationary law.

On the canonical path space, write a path with $n$ jumps as its state sequence and ordered jump times. Its jump count is $N_{xy}(T)={\sum }_{j=1}^n{\mathbf{1}}_{\{x_{j-1}=x,x_j=y\}}\mathrm{.}$ Using the nonnegative path densities [F2], the expectation is the sum over $n\ge 1$, state sequences, and ordered jump-time simplices of this count times the path density. The state-sequence sums are finite at each $n$, and their union over $n$ is countable; the increasing partial sums and [F3] justify the countable sum. For each fixed $n$ and jump index $j$, write the ordered-time simplex integral as an integral over $[0,T{]}^n$ with its simplex indicator. This is a nonnegative measurable function: the factors are continuous rates and survival functions, and the simplex is Borel. Lebesgue measure on the cube is finite and counting measure on the finite state sequences is finite, so [F4] permits integrating first in the selected jump time $s=t_j$ and then summing the other times. Summing over $n$ and $j$ is again an increasing nonnegative sum, so [A1] and [F3] justify its rearrangement.

For each selected jump at time $s$, the Markov property factors its path density into the prefix density through state $x$, the transition factor ${\kappa }_{xy}(s)ds$, and the suffix density starting in $y$. At fixed $s$, all finite prefixes ending in $x$ partition $\{X_s=x\}$ up to the null event of a jump at the fixed time $s$, so their densities sum to $p_x(s)$. All finite suffix densities starting in $y$ sum to $1$ by the normalization and nonexplosion conclusion of [F2]. The nonnegative sum/integral interchange justified in step 1.1 therefore gives the Lebesgue integral ${\int }_{[0,T]}{p}_x(s){\kappa }_{xy}(s)d{\lambda }_1(s)$.

Conditional on $X_s=x$, the probability of at least one jump by time $t>s$ is $1-H_x(s,t)=1-e^{-u}$ with $u:={\int }_s^t{\lambda }_x(r)dr\ge 0$. The inequality $1-e^{-u}\le u$ and the rate bound give this probability at most $M(t-s)$, uniformly in $x$. Hence $\mathrm{\mid }{p}_x(t)-p_x(s)\mathrm{\mid }\le \mathbb{P}(X_t\mathrm{\ne }{X}_s)\le M\mathrm{\mid }t-s\mathrm{\mid }$, so each $p_x$ is continuous. The product $p_x(t){\kappa }_{xy}(t)$ is consequently continuous and bounded on $[0,T]$, so its Lebesgue integral is finite. If $T>0$, [F5] identifies that integral with the Riemann integral in the Statement. If $T=0$, both the count and integral are zero by [F6]. This proves the identity for every $T\ge 0$.

If $T=0$, the single endpoint value is $G_y(0)={\rho }_y$. Suppose $T>0$ and put $K:={\max }_x{\sup }_{s\in [0,T]}\mathrm{\mid }{V}_x(s)\mathrm{\mid }$ and $M:={\max }_x{\sup }_{s\in [0,T]}{\lambda }_x(s)$. Write $w(t):=\exp (-{\int }_0^t{V}_{X_s}(s)ds)$. Almost surely the path has finitely many jumps by [F2], so the integrand is piecewise continuous; its integral is the limit of coordinate-measurable Riemann sums. Thus the weights are measurable (assign any bounded value on the measurable exceptional null set) and satisfy $0<w(t)\le e^{KT}$. For $0\le u<v\le T$ and $h:=v-u$, $\mathrm{\mid }w(v)-w(u)\mathrm{\mid }\le e^{KT}(e^{Kh}-1)$, while [F1] gives $\mathbb{P}(N_{(u,v]}\ge 1)\le 1-e^{-Mh}\le Mh$. Since an endpoint indicator can change only if a jump occurs, $\mathrm{\mid }{G}_y(v)-G_y(u)\mathrm{\mid }\le e^{KT}(e^{Kh}-1+Mh)\mathrm{.}$ This bound tends to zero with $h$, uniformly in $u,v$, proving continuity on the entire closed interval.

For $0\le t\le T-h$, condition on the history through $t$. Starting at $y$, the contribution of no jump to the additional weighted endpoint indicator is $\exp (-{\int }_t^{t+h}({\lambda }_y(s)+V_y(s))ds)=1-({\lambda }_y(t)+V_y(t))h+o(h)$. Starting at $x\mathrm{\ne }y$, exactly one jump to $y$ contributes ${\int }_t^{t+h}{H}_x(t,s){\kappa }_{xy}(s)H_y(s,t+h)\exp (-{\int }_t^s{V}_x(r)dr-{\int }_s^{t+h}{V}_y(r)dr)ds={\kappa }_{xy}(t)h+o(h)\mathrm{.}$ The conditional bound from [F2], obtained by restarting the first-jump rule at $t$, bounds the probability of two or more jumps by $(Mh{)}^2$ when $Mh<1$; their weighted contribution is at most $e^{Kh}(Mh{)}^2$. The finitely many continuous rates and potentials have a common modulus of continuity $\omega (h)\to 0$ on $[0,T]$. Hence the preceding remainders are $O(h\omega (h)+h^2)$ uniformly in the starting time and state. Multiply by $w(t)$ and average, using the Markov property and the bound in step 1.1, to obtain $G_y(t+h)=G_y(t)+h{R}_y(t)+r_y(t,h),R_y(t):={\sum }_{x\mathrm{\ne }y}{G}_x(t){\kappa }_{xy}(t)-G_y(t){\lambda }_y(t)-V_y(t)G_y(t),$ where ${\sup }_{0\le t\le T-h,y\in E}\mathrm{\mid }{r}_y(t,h)\mathrm{\mid }\mathrm{/}h\to 0$.

By step 1.1 and continuity of the rates and potentials, $R$ is continuous. Step 2.1 gives the right derivative $G_y^{\mathrm{\prime }}(t+)=R_y(t)$ for $t<T$. For $t>0$, apply its uniform expansion at $t-h$ to get $(G_y(t)-G_y(t-h))\mathrm{/}h=R_y(t-h)+r_y(t-h,h)\mathrm{/}h\to R_y(t)$. Thus the full derivative exists on $(0,T)$, extends continuously as the stated one-sided derivatives at the endpoints, and satisfies the claimed equation. The defining expectation at zero gives $G_y(0)={\rho }_y$.

If $G$ and $\tilde{G}$ are two solutions with the same initial vector, their difference $D$ satisfies $D(0)=0$ and $D(t)={\int }_0^{t}A(s)D(s)ds$ by [F5], applied componentwise, where $A(s)=Q(s{)}^{\mathsf{T}}-\mathrm{diag}(V(s))$ in column convention. Its coefficients and $D$ are continuous on $[0,T]$. Define $\mathrm{\parallel }z{\mathrm{\parallel }}_{\mathrm{\infty }}:={\max }_y\mathrm{\mid }{z}_y\mathrm{\mid }$, $B:={\sup }_s{\max }_y{\sum }_x\mathrm{\mid }{A}_{yx}(s)\mathrm{\mid }<\mathrm{\infty }$, and $C:={\sup }_s\mathrm{\parallel }D(s){\mathrm{\parallel }}_{\mathrm{\infty }}<\mathrm{\infty }$. Then $\mathrm{\parallel }A(s)z{\mathrm{\parallel }}_{\mathrm{\infty }}\le B\mathrm{\parallel }z{\mathrm{\parallel }}_{\mathrm{\infty }}$. Starting from $\mathrm{\parallel }D(t){\mathrm{\parallel }}_{\mathrm{\infty }}\le C$, induction in the integral equation gives $\mathrm{\parallel }D(t){\mathrm{\parallel }}_{\mathrm{\infty }}\le C(Bt{)}^n\mathrm{/}n!$ for every $n\ge 1$; the induction integrates $s^n$, whose primitive is $s^{n+1}\mathrm{/}(n+1)$, by [F5]. For each fixed $t$ these bounds tend to zero, since successive terms have ratio $Bt\mathrm{/}(n+1)$, eventually at most $1\mathrm{/}2$. Hence $D=0$ and the solution is unique.

Since $X_0\sim \pi$ and $\pi Q=0$, [F3] gives $\mathbb{P}(X_t=x)={\pi }_x$ for every $x\in E$ and every $t\in [0,T]$. Thus the process is stationary on the whole interval under the stated initial law.

Fix a path with $n$ jumps, state sequence $x_0,\dots ,x_n$ and times $0<t_1<\cdots <t_n<T$. Its density is $f(\gamma )={\pi }_{x_0}\left({\prod }_{j=1}^n{q}_{x_{j-1}{x}_j}\right)\exp (-{\sum }_{j=0}^n{\lambda }_{x_j}(t_{j+1}-t_j)),$ where $t_0=0$ and $t_{n+1}=T$, by [F2]. The reversed path has state sequence $x_n,\dots ,x_0$ and jump times $T-t_n,\dots ,T-t_1$, and its holding-time exponent is the same because it spends exactly the same duration in each state. If any forward jump rate in the sequence is zero, symmetric support makes the corresponding reversed rate zero too, so both path densities vanish and the sequence contributes nothing. On every sequence with positive density, symmetric support makes all reversed jump rates positive. The ratio of forward to reversed density is therefore $\frac{f(\gamma )}{f({\gamma }^{†})}=\frac{{\pi }_{x_0}}{{\pi }_{x_n}}{\prod }_{j=1}^n\frac{q_{x_{j-1}{x}_j}}{q_{x_j{x}_{j-1}}}={\prod }_{j=1}^n\frac{{\pi }_{x_{j-1}}{q}_{x_{j-1}{x}_j}}{{\pi }_{x_j}{q}_{x_j{x}_{j-1}}}=e^{S_T(\gamma )}\mathrm{.}$ The middle equality telescopes the factors ${\pi }_{x_{j-1}}\mathrm{/}{\pi }_{x_j}$; for $n=0$ both densities are positive and their ratio is $1$.

For $n=0$, reversal fixes the no-jump path. For $n\ge 1$, time reversal sends $(t_1,\dots ,t_n)$ to $(T-t_n,\dots ,T-t_1)$. This is an affine bijection of the ordered time simplex; its linear part is minus a coordinate-reversal permutation matrix, whose determinant has absolute value $1$. Thus [F9] shows it preserves Lebesgue measure and transports each nonnegative density integral to the reversed-path integral. The map also reverses the finite state sequence and sends $S_T$ to $-S_T$. Thus $-B$ is Borel by [F8]. Integrating the density identity in step 1.2 over paths with $S_T\in B$, and using the measure-preserving change of variables, gives ${\mu }_T(B)={\int }_{\{S_T\in -B\}}{e}^{-S_T}d\mathbb{P}={\int }_{-B}{e}^{-u}{\mu }_T(du)\mathrm{.}$ The countable union over $n$ covers all paths by [F2]. Taking $B=\mathbb{R}$ gives $\mathbb{E}[e^{-S_T}]=1$. The same identity is equivalent, by the change of variable $u=-s$, to ${\mu }_T(ds)=e^s{\mu }_T(-ds)$ as an equality of measures.

There are finitely many directed edges, all nonzero forward and reverse rates are positive, and every ${\pi }_x>0$. Thus the absolute value of each jump increment in [F4] is bounded by a finite constant $C$; if there are no edges, set $C=0$. Hence $\mathrm{\mid }{S}_T\mathrm{\mid }\le C{N}_T$, and [F2] gives $\mathbb{E}\mathrm{\mid }{S}_T\mathrm{\mid }<\mathrm{\infty }$. For every $z>0$, the function $g(z):=z-1-\log z$ has $g(1)=0$ and derivative $g^{\mathrm{\prime }}(z)=1-1\mathrm{/}z$, negative on $(0,1)$ and positive on $(1,\mathrm{\infty })$; the mean value theorem [F6] gives $g(z)\ge 0$. Apply this pointwise with $z=e^{-S_T}$. The function $g(e^{-S_T})$ is nonnegative and integrable because $S_T$ is integrable and $\mathbb{E}[e^{-S_T}]=1$; its expectation is therefore nonnegative. By linearity [F7] and $\log (e^{-S_T})=-S_T$, $0\le \mathbb{E}[g(e^{-S_T})]=\mathbb{E}[e^{-S_T}]-1+\mathbb{E}[S_T]=\mathbb{E}[S_T]\mathrm{.}$ Therefore $\mathbb{E}[S_T]\ge 0$.