LemmaStatement: AI-adaptedProof: AI-generatedPipeline-generatedjudge pass (gpt-6-luna)

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

An IID ReLU layer forces a zero atom in the next preactivation

Statement

Let $n \geq 1$ and let $Z_{1}, \dots, Z_{n}$ be independent and identically distributed real random variables whose law is symmetric about zero and has no atom at zero. Put $X_{i} = ReLU (Z_{i})$ . For any real random variables $W_{1}, \dots, W_{n}$ on the same probability space, define the next zero-bias preactivation $T = \sum_{i = 1}^{n} W_{i} X_{i}$ . Then $P (T = 0) \geq 2^{- n} > 0.$ In particular, this next preactivation cannot be atomless at zero, regardless of the distribution or dependence of the weights. The same conclusion holds under the weaker assumption that the $X_{i}$ are mutually independent and each $Z_{i}$ is symmetric with no atom at zero; independence of the $Z_{i}$ is not required in that formulation.

Facts & Assumptions

Given: The finite independent family $Z_{1}, \dots, Z_{n}$ , its common symmetric atomless law, and the definitions in the Statement.

[F1]

For a symmetric atomless real random variable, each sign has probability $1 / 2$ . (Second moments of symmetric rectifiers)

[F2]

A finite family of independent random elements has product joint law. (Independent random elements have product joint law)

[F3]

Independence of random elements means independence of their generated sigma-algebras. (Independent random elements)

Proof

1.1givenF1

By [F1], $P (Z_{i} < 0) = 1 / 2$ for each $i$ . On the event $E = ⋂_{i = 1}^{n} {Z_{i} < 0}$ , every $X_{i} = ReLU (Z_{i})$ is zero.

1.2givenF1F2F3

Since ${X_{i} = 0} = {Z_{i} \leq 0}$ and $P (Z_{i} = 0) = 0$ , the event $A = ⋂_{i} {X_{i} = 0}$ agrees with $E$ up to a null set. Under the weaker hypothesis, independence of the $X_{i}$ and [F2] give $P (A) = \prod_{i} P (X_{i} = 0) = 2^{- n}$ . Under the original hypothesis, each $σ (X_{i})$ is a sub-sigma-algebra of $σ (Z_{i})$ because ReLU is measurable, so [F3] implies independence of the $X_{i}$ and the same calculation applies.

2.1step 1.1step 1.2algebra∎

On $E$ , $T = \sum_{i} W_{i} X_{i} = 0$ for every value of the real weights. Hence $P (T = 0) \geq P (E) = 2^{- n} > 0$ , which proves the claim.

Depends on

Used by

A symmetric atomless IID ReLU layer creates a zero atom next Example

Dependency tree · two levels

17 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, Delving Deep into Rectifiers, §2.2 (standard reference, not scraped)
Simon J. D. Prince, Understanding Deep Learning, §7.5.1 (standard reference, not scraped)