The independent reward-stack model has the prescribed conditional arm laws

Partition $B$ by the finitely many past action strings $\alpha =(a_1,\dots ,a_{t-1})\in [k{]}^{t-1}$. Let $c_j(\alpha )$ be the number of times arm $j$ occurs in $\alpha$, and write $B_{\alpha }=B\cap \{(A_1,\dots ,A_{t-1})=\alpha \}$. By the recursive non-anticipating policy rule, each $B_{\alpha }$ is measurable with respect to $U$ and the table entries $X_{s,j}$ with $s\le c_j(\alpha )$: along that fixed action string, those and only those stack entries have been revealed.

If $B_{\alpha }$ is nonempty then $A_t=i$ on it, and the next reward is $Y_t=X_{c_i(\alpha )+1,i}$. This table entry is distinct from every coordinate determining $B_{\alpha }$. By [F2] it is independent of the vector of those coordinates, so the event $B_{\alpha }$ is independent of $\{X_{c_i(\alpha )+1,i}\in C\}$ for every Borel $C$. Its law is $P_i$, hence $\mathbb{P}(B_{\alpha }\cap \{Y_t\in C\})=\mathbb{P}(B_{\alpha })P_i(C)$.

The events $B_{\alpha }$ partition $B$. Summing the identity in step 2.1 over the finite set $[k{]}^{t-1}$ gives $\mathbb{P}(B\cap \{Y_t\in C\})=\mathbb{P}(B)P_i(C)$ for every Borel $C$.

For each $\alpha$, the indicator ${\mathbf{1}}_{B_{\alpha }}$ is a measurable function of the exposed-coordinate vector. By [F2] and [F4], it is independent of $X_{c_i(\alpha )+1,i}$. Both are integrable by [F3], so [F5] and [F6] give $\mathbb{E}[Y_t{\mathbf{1}}_{B_{\alpha }}]={\mu }_i\mathbb{P}(B_{\alpha })$. Summing over the finite action strings yields $\mathbb{E}[Y_t{\mathbf{1}}_B]={\mu }_i\mathbb{P}(B)$.