ExampleConstruction: AI-generatedVerification: AI-generatedPipeline-generated

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

Local updates and periodic synchronization give different averages

Example

Let $f_{1} (x) = x^{2} / 2$ and $f_{2} (x) = x^{2}$ on $R$ , let the two local-SGD workers start at $x_{0}^{1} = x_{0}^{2} = 1$ , and use step size $η = 1 / 2$ with one uniform component draw per worker and step from ${f_{1}, f_{2}}$ . Couple two executions by choosing the positive-probability draw sequence in which worker 1 draws $f_{1}$ twice and worker 2 draws $f_{2}$ twice. After two steps without intermediate synchronization their mean is $1 / 8$ . If they instead synchronize after the first step and again after the second, their final mean is $1 / 16$ .

Example references

Stich, Local SGD Converges Fast and Communicates Little, Definition 2.1

Facts & Assumptions

Given: The two differentiable component functions and the common starting point in the Example.

[F1]

Local SGD with periodic synchronization: each step samples a component uniformly and updates $x \leftarrow x - η \nabla f_{j} (x)$ ; a synchronization replaces all worker states by their arithmetic mean.

[A1]

The two workers' component draws are independent and uniform at each step, as in [F1]. The displayed coupled draw sequence therefore has probability $(1 / 2)^{4} > 0$ in either schedule.

Proof technique: direct.

Proof

1.1F1givenalgebra

The gradients are $\nabla f_{1} (x) = x$ and $\nabla f_{2} (x) = 2 x$ . Without an intermediate synchronization, worker 1 follows $1 \mapsto 1 / 2 \mapsto 1 / 4$ and worker 2 follows $1 \mapsto 0 \mapsto 0$ . Their arithmetic mean is $(1 / 4 + 0) / 2 = 1 / 8$ .

2.1F1step 1.1givenalgebra

With synchronization after the first step, the worker states are first $1 / 2$ and $0$ , so their common synchronized value is $1 / 4$ . On the second step, the same coupled draws give worker states $1 / 4 - (1 / 2) (1 / 4) = 1 / 8$ and $1 / 4 - (1 / 2) (2) (1 / 4) = 0$ . Their mean is $(1 / 8 + 0) / 2 = 1 / 16$ .

3.1A1step 1.1step 2.1∎

Since $1 / 8 \neq 1 / 16$ and the draw sequence in [A1] has positive probability under the algorithm, the two schedules can produce different final worker means.

Depends on

Local SGD with periodic synchronization

Used by

Nothing in the library uses this result yet.

Dependency tree · two levels

2 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Sebastian U. Stich, Local SGD Converges Fast and Communicates Little, Definition 2.1 (standard reference, not scraped)