Normalized discounted occupancy is a probability distribution

Given: A finite CMDP with initial distribution $\mu$, discount $0\le \gamma <1$, a stationary policy $\pi$, and the recursive state masses and occupancy defined in the linked items. For the signal bound, fix a real $B\ge 0$ and a transition signal $g:\mathcal{X}\to \mathbb{R}$ with $\mathrm{\mid }g(s,a,\rho ,s^{\mathrm{\prime }})\mathrm{\mid }\le B$ for every $(s,a,\rho ,s^{\mathrm{\prime }})\in \mathcal{X}$.

Finite discounted constrained Markov decision process: ${\mu }_0^{\pi }=\mu$ and ${\mu }_{t+1}^{\pi }(s^{\mathrm{\prime }})={\sum }_{s,a,\rho }{\mu }_t^{\pi }(s)\pi (a\mid s)p(s^{\mathrm{\prime }},\rho \mid s,a)$.

Finite Markov decision process with joint reward-transition law: for each admissible $(s,a)$, the joint next-state/reward masses are nonnegative and ${\sum }_{s^{\mathrm{\prime }},\rho }p(s^{\mathrm{\prime }},\rho \mid s,a)=1$.

Finite Markov decision process with joint reward-transition law: a stationary Markov policy has $\pi (a\mid s)\ge 0$ and ${\sum }_{a\in A(s)}\pi (a\mid s)=1$.

Normalized discounted state occupancy: $d^{\pi }(s)=(1-\gamma ){\sum }_{t\ge 0}{\gamma }^t{\mu }_t^{\pi }(s)$.

Expected discounted-cost feasibility constraint: $J_g^{\pi }(\mu )$ is the series of finite sums of the time-$t$ transition masses multiplied by $g(s,a,\rho ,s^{\mathrm{\prime }})$ and ${\gamma }^t$.

For $\mathrm{\mid }r\mathrm{\mid }<1$, ${\sum }_{k\ge 0}{r}^k=1\mathrm{/}(1-r)$, and for $\mathrm{\mid }r\mathrm{\mid }\ge 1$ the series diverges: for $0\le \gamma <1$, ${\sum }_{t=0}^{\mathrm{\infty }}{\gamma }^t=1\mathrm{/}(1-\gamma )$.

Absolute convergence and tail bound for bounded discounted rewards: every real sequence bounded in absolute value by $B$ has an absolutely convergent $\gamma$-discounted series whose absolute value is at most $B\mathrm{/}(1-\gamma )$.

The principle of mathematical induction: a property holding at $0$ and preserved from each natural number to its successor holds for every natural number.