Adaptive MDP visits admit fixed sample tapes

If $T=0$, take the original initial-state law and an empty tape for each pair: there are no observations to match and the independence assertions are vacuous. Hence assume $T\ge 1$. Fix finite enumerations of the admissible state-action pairs and the integers $1,\dots ,T$, and let $N=\mathrm{\mid }D\mathrm{\mid }T$ be the number of tape coordinates. Take the first coordinate's prescribed $Q$ law as the initial probability measure, use the remaining $N-1$ prescribed laws as constant kernels, and append singleton spaces with point-mass kernels afterward. Apply [F2] to this infinite kernel sequence. Its first $N$ coordinates have the product law: each coordinate has its prescribed law and, conditional on all preceding coordinates, its law remains that same $Q(\cdot \mid s,a)$. AC is used only as declared in [A1].

For $T\ge 1$, extend the tape law using [F2]: first append the original initial-state distribution as a constant kernel independent of the tapes, and then append the policy action kernels evaluated only at the revealed history. Simulate each action from its policy kernel using only the revealed history and its independent internal randomization, then reveal the next unused coordinate in the selected pair's tape. Since the action step does not expose any unused tape coordinate, the selected coordinate remains unrevealed and has conditional law $Q(\cdot \mid s,a)$ by step 1.1. Thus the conditional observation law at every time is the MDP kernel in [F1]. Induction over the finite horizon shows that every observed trajectory has the same probability law as in the original interaction, and each pair's observations are its tape prefix.