A dominated pathwise simulation gradient is unbiased

Fix nonzero $h\in {\mathbb{R}}^d$ with ${\theta }_0+h\in U$. For almost every $\omega$, set $q_{\omega }(t)=H({\theta }_0+th,\omega )$ on $[0,1]$. By [F2], $q_{\omega }^{\mathrm{\prime }}(t)=⟨{\mathrm{\nabla }}_{\theta }H({\theta }_0+th,\omega ),h⟩$, which is continuous and hence integrable by [F3]. The FTC [F3] gives $H({\theta }_0+h,\omega )-H({\theta }_0,\omega )-⟨{\mathrm{\nabla }}_{\theta }H({\theta }_0,\omega ),h⟩={\int }_0^1⟨{\mathrm{\nabla }}_{\theta }H({\theta }_0+th,\omega )-{\mathrm{\nabla }}_{\theta }H({\theta }_0,\omega ),h⟩dt\mathrm{.}$

For each such $h$, the random variable $M_h(\omega ):={\sup }_{r\in \mathbb{Q}\cap [0,1]}\mathrm{\parallel }{\mathrm{\nabla }}_{\theta }H({\theta }_0+rh,\omega )-{\mathrm{\nabla }}_{\theta }H({\theta }_0,\omega ){\mathrm{\parallel }}_2$ is measurable: continuity in $t$ makes this countable supremum equal the supremum over $[0,1]$. It satisfies $0\le M_h\le 2M$ almost surely. As $h\to 0$, continuity of the sample gradient at ${\theta }_0$ gives $M_h\to 0$ almost surely.

By [F8], $J(\theta )=\mathbb{E}[H(\theta ,\cdot )]$. Taking expectations in step 1.1, linearity in [F7] moves the finite inner product through expectation, and [F6] bounds it by $M_h\mathrm{\parallel }h{\mathrm{\parallel }}_2$. The modulus bound in [F7] therefore bounds the absolute first-order remainder divided by $\mathrm{\parallel }h{\mathrm{\parallel }}_2$ by $\mathbb{E}{M}_h$. Along every sequence of nonzero $h_n\to 0$, [F4] with majorant $2M$ gives $\mathbb{E}{M}_{h_n}\to 0$. Hence $J({\theta }_0+h)-J({\theta }_0)-⟨\mathbb{E}[{\mathrm{\nabla }}_{\theta }H({\theta }_0,\cdot )],h⟩=o(\mathrm{\parallel }h{\mathrm{\parallel }}_2),$ which is the Fréchet derivative formula.

Each coordinate of the unbiased estimator is square-integrable by the stated second-moment assumption. [F5] therefore gives finite variance for each coordinate.