ExampleConstruction: AI-generatedVerification: AI-generatedPipeline-generatedjudge pass (gpt-6-luna)

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

Mean-zero weights or a nonzero bias do not supply the ReLU half-moment

Example

Compare two one-unit preactivations. In the first, the input is $X = 1$ , the bias is zero, and the weight $W$ equals $- 1$ with probability $2 / 3$ and $2$ with probability $1 / 3$ . In the second, the input $X$ is uniform on $[- 2, 2]$ , the independent weight $W$ equals $- 1$ or $1$ with equal probabilities, and the bias is $1$ . In each case compute $E [ReLU (Z)^{2}]$ and compare it with $\frac{1}{2} Var (Z)$ , where $Z = W X + b$ . The first case tests whether a zero-mean but nonsymmetric weight law suffices; the second tests the effect of a nonzero bias despite symmetric weights and input.

Facts & Assumptions

Given: The two one-unit probability models specified in the Example.

[F1]

If $Z$ is symmetric about zero and has no atom at zero, then $E [ReLU (Z)^{2}] = \frac{1}{2} E [Z^{2}]$ . (Second moments of symmetric rectifiers)

[F2]

Variance is the centered second moment, $Var (Z) = E [(Z - E [Z])^{2}]$ . (Moments, variance, and covariance on a probability space)

[F3]

Expectations of nonnegative random variables are integrals against their probability laws. (Expectation of a nonnegative or integrable random variable)

Verification

1.1givenF2algebra

In the first case, $E [W] = (- 1) (2 / 3) + 2 (1 / 3) = 0$ and $Var (Z) = Var (W) = E [W^{2}] = 2$ . Since $ReLU (Z)$ is $2$ with probability $1 / 3$ and zero otherwise, $E [ReLU (Z)^{2}] = 4 / 3$ , whereas $\frac{1}{2} Var (Z) = 1$ .

1.2givenF1algebra

The first preactivation has $P (Z > 0) = 1 / 3$ and $P (Z < 0) = 2 / 3$ , so its law is not symmetric despite its zero mean and absence of an atom at zero. The symmetric-preactivation hypothesis of [F1] fails; zero mean alone cannot replace it.

1.3givenF2F3algebra

In the second case, conditioning on either value of $W$ shows that $W X$ is uniform on $[- 2, 2]$ . Thus $Z = W X + 1$ is uniform on $[- 1, 3]$ , with mean $1$ and variance $(3 - (- 1))^{2} / 12 = 4 / 3$ ; it has no atom at zero.

2.1givenF2F3algebra∎

The density of $Z$ is $1 / 4$ on $[- 1, 3]$ , so $E [ReLU (Z)^{2}] = \frac{1}{4} \int_{0}^{3} z^{2} d z = 9 / 4$ , while $\frac{1}{2} Var (Z) = 2 / 3$ . The nonzero bias has shifted the symmetric preactivation and invalidated the half-moment identity.

Depends on

Used by

Nothing in the library uses this result yet.

Dependency tree · two levels

9 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, Delving Deep into Rectifiers, §2.2 (standard reference, not scraped)
Simon J. D. Prince, Understanding Deep Learning, §7.5.1 (standard reference, not scraped)
Johns Hopkins University, Handout #8: Neural Network Training and Tokenization, §3 (standard reference, not scraped)