Gaussian reparameterization gives two gradient estimators

Given: A standard normal $Z$, the affine representation $X_{\theta }=\theta +Z$, and the identity payoff $H(x)=x$.

The Axiom of Choice: every family of nonempty sets has a choice function; AC is inherited by the normal-law and score interfaces below.

Standard normal and normal laws: $Z$ has symmetric density $\varphi (z)=(2\pi {)}^{-1\mathrm{/}2}{e}^{-z^2\mathrm{/}2}$, and $X_{\theta }$ is the location-shifted normal law $N(\theta ,1)$. A positive-variance normal law has its location-scale density gives its density $p_{\theta }(x)=\varphi (x-\theta )$ on $\mathbb{R}$.

Gaussian even moments for Brownian increments: $\mathbb{E}{Z}^2=1$ and $\mathbb{E}{Z}^4=3$.

Pathwise and reparameterized simulation-gradient estimators: with the same $Z$ held fixed, the pathwise estimator is ${\mathrm{\partial }}_{\theta }H(X_{\theta })$.

A dominated pathwise simulation gradient is unbiased: an integrably dominated continuous pathwise derivative is the derivative of the expected payoff and has finite variance under its second-moment hypothesis.

Score and Fisher information: on positive density support the score is ${\mathrm{\partial }}_{\theta }\log p_{\theta }$.

The likelihood-ratio score gradient is unbiased on fixed support: under fixed support, dominated density differentiation, and a finite weighted second moment, ${\mathrm{\partial }}_{\theta }{\mathbb{E}}_{\theta }H={\mathbb{E}}_{\theta }[H{S}_{\theta }]$.

A positive-variance normal law has its location-scale density: $N(0,2)$ has density $(4\pi {)}^{-1\mathrm{/}2}{e}^{-z^2\mathrm{/}4}$.

Integrating against a density agrees with integrating the product: the integral of a nonnegative measurable function under a law with density $p_{\theta }$ is its integral after multiplication by $p_{\theta }$ against Lebesgue measure.

Change of variables for expectation: if $Z_2=\sqrt{2}Z$ with $Z\sim N(0,1)$, then $\mathbb{E}{Z}_2^2=2\mathbb{E}{Z}^2$.

Moments, variance, and covariance on a probability space and Variance and covariance identities for random variables: variance is the centered second moment and equals the second moment minus squared mean.

The Axiom of Choice, AC implies DC implies countable choice, The Axiom of Countable Choice (${\mathrm{A}\mathrm{C}}_{\omega }$) and A C^1 diffeomorphism satisfies the change-of-variables formula for nonnegative Lebesgue measurable functions: AC implies Countable Choice, and applying change of variables to reflection $z\mapsto -z$ with the even density in [F1] shows that the standard normal law is symmetric.

Change of variables for expectation and Linearity, monotonicity, and the modulus bound for expectation: integrable functions of a random variable may be integrated against its law, and expectation is linear.