A biased SGD direction can fail positive alignment

The singleton measure has total mass one and is countably additive because at most one set in any disjoint sequence can be nonempty [F5]. By [F3], $\mathrm{\nabla }f(x)=x$ and $f(x)=x^2\mathrm{/}2$. The deterministic direction has conditional mean $-x_t$ and conditional variance zero on the trivial filtration. For any finite horizon, apply [F1] with $y_s=x_{s-1}$, $H_s=-y_s=G_{s-1}$ for $s\ge 1$, and ${\mathcal{H}}_s=\mathcal{F}$ for $s\ge 0$. Its positive-time update $y_{s+1}=y_s-\alpha H_s$ becomes $x_{t+1}=x_t-\alpha (-x_t)=(1+\alpha )x_t$ for $t=s-1\ge 0$.

The recurrence in step 1.1 and $1+\alpha >0$ give $x_t=(1+\alpha {)}^t{x}_0\mathrm{\ne }0$ for every $t\ge 0$ by [F4]. In particular, $f(x_1)=(1+\alpha {)}^2{x}_0^2\mathrm{/}2>x_0^2\mathrm{/}2=f(x_0)$, so this direction can increase the objective in one step.

For every $t$, step 2.1 gives $x_t\mathrm{\ne }0$, so $\mathrm{\nabla }f(x_t)\mathbb{E}[G_t\mid {\mathcal{F}}_t]=x_t(-x_t)=-x_t^2<0$, whereas $\mu \mathrm{\mid }\mathrm{\nabla }f(x_t){\mathrm{\mid }}^2=\mu x_t^2>0$ for every $\mu >0$. Therefore the local positive-alignment hypothesis [F2] fails at every iterate, despite the direction norm equaling the gradient norm and the variance being zero. This comparison invokes the theorem only in its AC branch [A1]; the displayed deterministic calculation does not use AC.