Return-cycle occupation measure and minimality

Define the survival masses ${\alpha }_n(y):={\mathbb{P}}_b(X_n=y,T_b^{+}>n)$ for $n\ge 0$, $y\in E$. For $n\ge 1$, $T_b^{+}>n$ requires $X_1,\dots ,X_n$ all to avoid $b$, so ${\alpha }_n(y)={\mathbb{P}}_b(X_1\mathrm{\ne }b,\dots ,X_n\mathrm{\ne }b,X_n=y)$ and in particular ${\alpha }_n(b)=0$. At $n=0$ the avoidance condition is empty, and [F3] with initial law ${\delta }_b$ gives $X_0=b$ almost surely, so ${\alpha }_0(b)=1$ and ${\alpha }_0(y)=0$ for $y\mathrm{\ne }b$.

For every $y$ we have ${\mu }_b(y)={\sum }_{n\ge 0}{\alpha }_n(y)$: by the monotone convergence theorem [F5] applied to the partial sums of the nonnegative terms ${\mathbf{1}}_{\{X_n=y\}}{\mathbf{1}}_{\{n<T_b^{+}\}}$, the expectation of the series is the series of the expectations ${\mathbb{P}}_b(X_n=y,T_b^{+}>n)={\alpha }_n(y)$.

Put $Q(x,y):=p(x,y){\mathbf{1}}_{\{y\mathrm{\ne }b\}}$. A nonnegative $\nu$ satisfies $\nu (b)=1$ and $\nu (y)={\sum }_x\nu (x)p(x,y)$ for all $y\mathrm{\ne }b$ if and only if $\nu ={\delta }_b+\nu Q$ as extended nonnegative functions, because at $y=b$ the right side is $1+0=\nu (b)$ and at $y\mathrm{\ne }b$ it is $(\nu p)(y)$.

If $E=\mathrm{\varnothing }$ there is no starting state $b$, so the hypotheses cannot be met.

If $b$ is absorbing, the finite-dimensional law [F3] and the initial law ${\delta }_b$ imply $X_n=b$ almost surely for every $n\ge 0$; hence $T_b^{+}=1$ by [F1].

The absorbing path of step 1.5 has exactly one occupation before $T_b^{+}$, at time $0$, so ${\mu }_b={\delta }_b$, ${\sum }_y{\mu }_b(y)=1={\mathbb{E}}_b{T}_b^{+}$.

For every $n\ge 0$ and $y\mathrm{\ne }b$ the recursion ${\alpha }_{n+1}(y)={\sum }_{x\in E}{\alpha }_n(x)p(x,y)$ holds. Since $y\mathrm{\ne }b$, the event $\{T_b^{+}>n,X_{n+1}=y\}$ equals $\{T_b^{+}>n+1,X_{n+1}=y\}$. Partition the first event by $X_n=x$: the cylinder formula [F3] gives ${\mathbb{P}}_b(T_b^{+}>n,X_n=x,X_{n+1}=y)={\alpha }_n(x)p(x,y)$ for each $x$, and countable additivity gives the displayed sum. At $n=0$ only $x=b$ contributes, with mass $p(b,y)$; for $n\ge 1$, the $x=b$ term is zero by step 1.1.

${\mu }_b(b)=1$ by steps 1.1 and 1.2, since the series for ${\mu }_b(b)$ has the single nonzero term ${\alpha }_0(b)=1$.

${\sum }_{y\in E}{\mu }_b(y)={\mathbb{E}}_b{T}_b^{+}$: for every outcome the sum ${\sum }_{y\in E}{\mathbf{1}}_{\{X_n=y\}}$ equals one exactly for the $T_b^{+}$ indices $n<T_b^{+}$, so both sides equal the expectation of ${\sum }_{n\ge 0}{\mathbf{1}}_{\{n<T_b^{+}\}}$; applying [F4] to the nonnegative double sequence ${\mathbb{P}}_b(X_n=y,n<T_b^{+})$ interchanges the sums over $y$ and $n$, [F5] identifies ${\sum }_n{\mathbb{P}}_b(T_b^{+}>n)$ with ${\mathbb{E}}_b{T}_b^{+}$ by the indicator tail identity, and infinite values are allowed on both sides.

For every $n\ge 0$, using the survival-mass definition in step 1.1, the last-step factorization [F3] gives ${\sum }_{x\in E}{\alpha }_n(x)p(x,b)={\mathbb{P}}_b(X_{n+1}=b,T_b^{+}>n)={\mathbb{P}}_b(T_b^{+}=n+1)$, since $T_b^{+}>n$ rules out an earlier return and $X_{n+1}=b$ makes the next time the first return.

Since $b$ is absorbing, $p(b,b)=1$; with ${\mu }_b={\delta }_b$ from step 2.1 and the matrix convention [F2], $({\mu }_{b}p)(b)=1$. Thus the absorbing case satisfies the occupation-mass, return-time, and return-flow identities.

Interchanging the nonnegative sums by [F4] and using step 1.2 gives $({\mu }_{b}p)(b)={\sum }_{x\in E}{\mu }_b(x)p(x,b)={\sum }_{n\ge 0}{\mathbb{P}}_b(T_b^{+}=n+1)={\mathbb{P}}_b(T_b^{+}<\mathrm{\infty })$, since the positive finite return times partition $\{T_b^{+}<\mathrm{\infty }\}$.

For $y\mathrm{\ne }b$, ${\mu }_b(y)={\sum }_{x\in E}{\mu }_b(x)p(x,y)$: by step 1.2, [F4] applied to the nonnegative terms ${\alpha }_n(x)p(x,y)$, and step 2.2, ${\sum }_{x\in E}{\mu }_b(x)p(x,y)={\sum }_{n\ge 0}{\sum }_{x\in E}{\alpha }_n(x)p(x,y)={\sum }_{n\ge 0}{\alpha }_{n+1}(y)={\mu }_b(y)-{\alpha }_0(y)={\mu }_b(y),$ the last step because ${\alpha }_0(y)=0$ for $y\mathrm{\ne }b$.

For such $\nu$ and every $N\ge 1$, $\nu ={\sum }_{n<N}{\delta }_b{Q}^n+\nu Q^N$, where $Q^n$ are powers of the substochastic matrix $Q$. Induct on $N$ from step 1.3; each reassociation of the countable nonnegative matrix sums is justified by [F4]. By step 2.2 and the zero $b$-coordinate in step 1.1, ${\alpha }_n={\delta }_b{Q}^n$ with ${\alpha }_0={\delta }_b$, so the first sum is the $N$th partial sum of the series in step 1.2.

The time-$0$ term contributes ${\mu }_b(b)=1$ although no return has occurred, since the occupation sum starts at $n=0$ while the return time is strictly positive.

If $b$ is transient then $({\mu }_{b}p)(b)={\mathbb{P}}_b(T_b^{+}<\mathrm{\infty })<1$ by step 3.2, so item 4 genuinely uses recurrence; item 3's minimality inequality remains valid.

Letting $N\to \mathrm{\infty }$ in step 3.4 gives $\nu (y)\ge {\sup }_{N\ge 1}{\sum }_{n<N}{\alpha }_n(y)={\mu }_b(y)$ for every $y$, since the remainder $\nu Q^N(y)$ is nonnegative and a nonnegative series is the supremum of its partial sums; this is the asserted pointwise minimality.

If $b$ is recurrent then ${\mathbb{P}}_b(T_b^{+}<\mathrm{\infty })=1$, so $({\mu }_{b}p)(b)=1={\mu }_b(b)$ by steps 2.3 and 3.2, while step 3.3 gives equality at every $y\mathrm{\ne }b$; hence ${\mu }_{b}p={\mu }_b$.

AC [A1] is used through the finite-dimensional-law supplier [F3]; once that chain law is supplied, the recursion and nonnegative summations are finite-time or Tonelli/monotone-convergence calculations with no further choice. The pointwise minimality assertion is one-way, not an if-and-only-if.