For Hermitian matrices, the Rayleigh quotient and residual converge with the expected rates along power iteration

Let $\rho :=\mathrm{\mid }{\lambda }_2\mathrm{/}{\lambda }_1\mathrm{\mid }<1$. By [L4], the Hermitian matrix is diagonalisable, so [L1] applies. After choosing phases ${\alpha }_k$ of modulus one, one has $\mathrm{\parallel }{\alpha }_k{x}_k-q_1{\mathrm{\parallel }}_2=O({\rho }^k)\mathrm{.}$ Write ${\alpha }_k{x}_k=c_k{q}_1+z_k$ with $z_k\perp q_1$. Then $\mathrm{\mid }{c}_k-1\mathrm{\mid }+\mathrm{\parallel }{z}_k{\mathrm{\parallel }}_2=O({\rho }^k)$.

By [L2], the orthogonal complement $q_1^{\perp }$ is $A$-invariant. Since $x_k$ is unit and ${\mu }_k=⟨A({\alpha }_k{x}_k),{\alpha }_k{x}_k⟩$, the cross terms vanish: ${\mu }_k=\mathrm{\mid }{c}_k{\mathrm{\mid }}^2{\lambda }_1+⟨A{z}_k,z_k⟩\mathrm{.}$ Using $\mathrm{\mid }{c}_k{\mathrm{\mid }}^2+\mathrm{\parallel }{z}_k{\mathrm{\parallel }}_2^2=1$, this becomes ${\mu }_k-{\lambda }_1=⟨A{z}_k,z_k⟩-{\lambda }_1\mathrm{\parallel }{z}_k{\mathrm{\parallel }}_2^2\mathrm{.}$ Therefore $\mathrm{\mid }{\mu }_k-{\lambda }_1\mathrm{\mid }\le (\mathrm{\parallel }A{\mathrm{\parallel }}_2+\mathrm{\mid }{\lambda }_1\mathrm{\mid })\mathrm{\parallel }{z}_k{\mathrm{\parallel }}_2^2=O({\rho }^{2k})\mathrm{.}$

Using $A{q}_1={\lambda }_1{q}_1$, one has ${\alpha }_k{r}_k=A({\alpha }_k{x}_k)-{\mu }_k({\alpha }_k{x}_k)=c_k({\lambda }_1-{\mu }_k)q_1+(A-{\mu }_{k}I)z_k\mathrm{.}$ Since $\mathrm{\mid }{\alpha }_k\mathrm{\mid }=1$, $\mathrm{\parallel }{r}_k{\mathrm{\parallel }}_2\le \mathrm{\mid }{c}_k\mathrm{\mid }\mathrm{\mid }{\lambda }_1-{\mu }_k\mathrm{\mid }+(\mathrm{\parallel }A{\mathrm{\parallel }}_2+\mathrm{\mid }{\mu }_k\mathrm{\mid })\mathrm{\parallel }{z}_k{\mathrm{\parallel }}_2\mathrm{.}$ Step 2.1 and step 1.1 give $\mathrm{\parallel }{r}_k{\mathrm{\parallel }}_2=O({\rho }^k)\mathrm{.}$

The displayed bounds force ${\mu }_k\to {\lambda }_1$ and $r_k\to 0$.