ExampleConstruction: AI-adaptedVerification: AI-generatedPipeline-generated

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

A symmetric atomless IID ReLU layer creates a zero atom next

Example

Let $Z_{1}, Z_{2}$ be independent and identically distributed symmetric real random variables with no atom at zero, set $X_{i} = ReLU (Z_{i})$ , and form a second-layer zero-bias preactivation $T = W_{1} X_{1} + W_{2} X_{2}$ with arbitrary real weights. Although neither $Z_{i}$ has an atom at zero, the event that both rectified inputs vanish has probability $1 / 4$ and forces $T = 0$ .

For a complete finite-network signal calculation, also take a two-layer width-one ReLU stack with $X_{0} = 1$ , mutually independent $W_{1}, W_{2} \in {- \sqrt{2}, \sqrt{2}}$ with equal sign probabilities, and an independent random sign $R \in {- 1, 1}$ . Take objective $J (X_{2}) = R X_{2}$ , so $g_{2} = R$ , and use zero-at-kink backpropagation. Then $Var (Z_{1}) = Var (Z_{2}) = 2$ but $P (Z_{2} = 0) = 1 / 2$ and $Var (g_{1}) = 1 / 2$ . The atomless half-factor coefficient would instead give $1$ . The general conditional recurrence allows zero atoms, but its weight/gated-signal independence hypothesis fails in this realized stack. Its half-factor special case also requires $p_{2} = 0$ , whereas here $p_{2} = 1 / 2$ .

Facts & Assumptions

Given: The two-coordinate zero-event model and the independent finite random-sign scalar stack in the Example.

[F1]

For $n$ IID symmetric atomless preactivations followed by ReLU, any next zero-bias weighted sum has an atom at zero of probability at least $2^{- n}$ . (An IID ReLU layer forces a zero atom in the next preactivation)

[F2]

Zero-at-kink backward signals satisfy $h_{2} = 1_{Z_{2} > 0} g_{2}$ and $g_{1} = W_{2} h_{2}$ ; the atomless half-factor formula requires additional hypotheses. (ReLU backward gradient-variance recurrence)

Verification

1.1givenF1algebra

The example has $n = 2$ , so [F1] gives $P (T = 0) \geq 2^{- 2} = 1 / 4$ , independently of the values of $W_{1}, W_{2}$ .

1.2givenalgebra

In the scalar stack, $Z_{1}$ is $\pm \sqrt{2}$ with equal probabilities. If $W_{1} < 0$ then $X_{1} = 0$ and $Z_{2} = 0$ ; if $W_{1} > 0$ then $Z_{2}$ is $2$ or $- 2$ with equal probabilities. Thus the law of $Z_{2}$ gives probabilities $1 / 4, 1 / 2, 1 / 4$ to $- 2, 0, 2$ , respectively. Both preactivations have mean zero and variance $2$ , and the second has zero probability $1 / 2$ .

2.1step 1.1algebra

Therefore the two first-layer preactivations can be atomless while the next preactivation is not atomless. Requiring the no-atom condition at every layer is incompatible with this finite IID ReLU stack.

3.1givenF2step 1.2algebra∎

By [F2], $g_{1} = \sqrt{2} R$ exactly when both weights are positive, an event of probability $1 / 4$ ; it is zero otherwise. Independence and the centered sign $R$ give mean zero and variance $2 (1 / 4) = 1 / 2$ . The atomless coefficient would be $(1 / 2) (1) (2) E [R^{2}] = 1$ . Here $h_{2} = R$ on the same positive-weight event and zero otherwise: $P (h_{2} \neq 0 ∣ W_{2} < 0) = 0$ , whereas $P (h_{2} \neq 0) = 1 / 4$ , so weight/gated-signal independence fails. The covariance identity and the specified kink convention remain valid; the atomless conditional recurrence is inapplicable.

Depends on

Used by

Nothing in the library uses this result yet.

Dependency tree · two levels

12 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, Delving Deep into Rectifiers, §2.2 (standard reference, not scraped)
Simon J. D. Prince, Understanding Deep Learning, §7.5.1 (standard reference, not scraped)