Finite group-relative updates preserve admissibility and have bounded oracle calls

Fix an outer iteration begun at a valid parameter. Every returned response belongs to its finite fixed support, so the $G$ rewards and their mean are finite. Each squared deviation is nonnegative by [F3]. Let $S_k$ be the sum of the first $k$ squared deviations, with $S_0=0$. If $S_k$ or the next term is zero, $S_{k+1}$ is the other nonnegative term; if both are positive, [F5] makes their sum positive. Induction gives $S_G\ge 0$. Since $1\mathrm{/}G>0$ by [F5], the standard-deviation radicand is finite and nonnegative. Thus [F3] gives a finite nonnegative standard deviation. If it is positive, the normalized advantages are finite quotients; if it is zero, the explicit branch sets them to zero and makes no division.

At any candidate parameter in $U$, old and new supported policy masses are positive by [F2], so each sampled sequence ratio is finite and has a nonzero denominator. The positive reference masses make every KL log ratio a finite real by [F4]; its sum over each finite support is finite. Clipping, products, minima and finite averages in the exact displayed objective therefore produce a finite real value. This requires no differentiability of the clipped minimum at a boundary.

Initially the last valid parameter is ${\theta }_0\in U$. Suppose it is valid before an inner attempt. The supplied exact gradient oracle terminates; if the total derivative does not exist, it reports undefined-gradient and the procedure returns the unchanged valid parameter. Otherwise its gradient is a finite vector in ${\mathbb{R}}^d$, so the finite step size gives a finite proposal. The supplied exact domain oracle terminates and returns zero or one. A zero returns the previous parameter with invalid-update; a one places the new parameter in $U$, where fixed positive support holds. Induction over inner and outer iterations proves the invariant and stated return statuses. The frozen data and objective are well-defined by steps 1.1 and 1.2.

Each reached outer iteration uses exactly $MG$ supported response draws and reward evaluations, followed by at most $K$ inner attempts. Every attempt makes one gradient-oracle call and at most one domain-oracle call. There are at most $T$ reached outer iterations, so the respective counts are at most $TMG$, $TMG$, $TK$ and $TK$. Every support, group and coordinate computation is finite, and every supplied call has finite cost. Thus the finite sum of actual call costs and finite bookkeeping costs is finite; the procedure completes the prescribed loops or exits earlier as described in step 2.1. This argument decides neither arbitrary differentiability nor arbitrary open-set membership without the supplied oracles.