Spectral gap for a unitary representation

Let $G$ be a topological group and let $(\pi ,H)$ be a strongly continuous unitary representation (Strongly continuous unitary representations, invariant linear subspaces and intertwiners) on a Hilbert space $H$ (Hilbert space). Put $H^G:=\{\xi \in H:\pi (g)\xi =\xi \text{ for every }g\in G\}\mathrm{.}$ This is a closed invariant subspace: if ${\xi }_n\in H^G$ and ${\xi }_n\to \xi$, then for every $g$ the isometry $\pi (g)$ gives $\mathrm{\parallel }\pi (g)\xi -\xi \mathrm{\parallel }\le 2\mathrm{\parallel }\xi -{\xi }_n\mathrm{\parallel }\to 0$. Its orthogonal complement $H^G{}^{\perp }$ (Orthogonality and the orthogonal complement) is also closed and $G$-invariant by Invariant orthogonal complements in unitary representations, so the restriction $\pi {\mathrm{\mid }}_{H^G{}^{\perp }}$ is a strongly continuous unitary representation. The representation $\pi$ has spectral gap if $1_G\prec ̸\pi {\mathrm{\mid }}_{H^G{}^{\perp }},$ where $1_G$ is the trivial representation and $\prec$ is weak containment (Weak containment of unitary representations).

If $G$ is locally compact Hausdorff (Locally compact topological space: every point has a compact neighbourhood; and what this says in a metric space, Hausdorff space: distinct points have disjoint open neighbourhoods; every metrizable space is Hausdorff and the indiscrete topology on two points is not) and AC is assumed (The Axiom of Choice), this is equivalent by Weak containment of the trivial representation and almost invariant vectors to the existence of a compact $Q\subseteq G$ containing the identity and an $\epsilon >0$ such that ${\sup }_{g\in Q}\mathrm{\parallel }\pi (g)\xi -\xi \mathrm{\parallel }\ge \epsilon \mathrm{\parallel }\xi \mathrm{\parallel }\text{for every }\xi \in H^G{}^{\perp }\mathrm{.}$ No displacement equivalence is asserted for general topological groups.

The weak-containment lemma says, under its LCH and AC hypotheses, that $1_G\prec \pi {\mathrm{\mid }}_{H^G{}^{\perp }}$ exactly when every compact $Q$ and every $\delta >0$ admit a unit vector in $H^G{}^{\perp }$ with displacement $<\delta$. Negating this statement gives one compact $Q$ and one $\epsilon >0$ for which every unit vector has displacement at least $\epsilon$. Rescaling gives the displayed bound for every nonzero vector, and for $\xi =0$ both sides are zero. Replacing $Q$ by $Q\cup \{e\}$ preserves the bound and ensures the compact set is nonempty. If $H^G{}^{\perp }=\{0\}$, the weak-containment condition fails and the displacement inequality holds vacuously apart from its true zero-vector equality.