Score and Fisher information

Let $\mathcal{E}=((\mathcal{X},\mathcal{A}),\mathrm{\Theta },(P_{\theta }{)}_{\theta \in \mathrm{\Theta }})$ be a statistical experiment dominated by a fixed sigma-finite positive measure $\mu$, and let $p_{\theta }$ be selected finite-valued measurable Radon–Nikodym density representatives as in Likelihood densities. The simultaneous selection of this arbitrary-index family uses the Axiom of Choice. No measurability of $\theta \mapsto P_{\theta }$ or common exceptional null set for an arbitrary uncountable $\mathrm{\Theta }$ is implied.

Let $\mathrm{\Theta }$ be open in ${\mathbb{R}}^d$, and fix a parameter $\theta$ at which the selected family $\eta \mapsto p_{\eta }(x)$ is differentiable for $\mu$-almost every $x$, with measurable coordinate derivatives. On $\{x:p_{\theta }(x)>0\}$, define the log-likelihood ${\mathrm{\ell }}_{\theta }(x)=\log p_{\theta }(x)$ and the score $S_{\theta }(x):={\mathrm{\nabla }}_{\theta }{\mathrm{\ell }}_{\theta }(x)={(\frac{\mathrm{\partial }}{\mathrm{\partial }{\theta }_j}\log p_{\theta }(x))}_{j=1}^d\mathrm{.}$ Set $S_{\theta }(x)=0$ where $p_{\theta }(x)=0$, which is $P_{\theta }$-null. For $d=1$, this is $S_{\theta }={\mathrm{\partial }}_{\theta }\log p_{\theta }$.

For $d=1$, the scalar Fisher information is $I(\theta ):={\mathbb{E}}_{\theta }[S_{\theta }(X{)}^2]\in [0,+\mathrm{\infty }]\mathrm{.}$ When $d>1$ and $S_{\theta }$ is square-integrable, the Fisher information matrix is $I(\theta )={\mathbb{E}}_{\theta }[S_{\theta }(X)S_{\theta }(X{)}^{\mathsf{T}}]$; its $i,j$ entry is ${\mathbb{E}}_{\theta }[S_{\theta ,i}{S}_{\theta ,j}]$. No scalar information is defined here for a vector parameter. The information matrix is a second-moment matrix by definition; it equals the score covariance matrix when the score has mean zero.

The derivatives, score and information are defined relative to the specified differentiable representative family. The Radon–Nikodym definition alone does not supply parameter differentiability or a canonical score version.