Weak containment of the trivial representation and almost invariant vectors

Assume the Axiom of Choice. Let $G$ be an LCH group, let $1_G$ denote the trivial representation on $\mathbb{C}$ ($1_G(g)z=z$, a strongly continuous unitary representation) and let $\pi$ be a strongly continuous unitary representation of $G$ on a Hilbert space $H$ (Strongly continuous unitary representations, invariant linear subspaces and intertwiners). Then $1_G\prec \pi$ (Weak containment of unitary representations) if and only if for every compact $Q\subseteq G$ and every $ϵ>0$ there is a unit vector $\xi \in H$ with ${\sup }_{g\in Q}\mathrm{\parallel }\pi (g)\xi -\xi \mathrm{\parallel }<ϵ$.

The LCH hypothesis cannot be dropped for the finite-sum coefficient definition of weak containment used here. Let $K_k=\{(g_d{)}_{d\ge 1}\in {\prod }_{d\ge 1}U(d):\mathrm{rank}(g_d-I)\le k\text{ for every }d\}$. The group $U(d)$ is closed and bounded in ${\mathbb{C}}^{d^2}\cong {\mathbb{R}}^{2d^2}$, since $g^{\ast }g=I$ is a closed condition and each entry has modulus at most one; it is therefore compact by A subset of ${\mathbb{R}}^n$ with the product topology is compact exactly when it is closed and bounded, the product topology being the Euclidean metric topology. Each complex rank condition is closed: the real matrix of a complex-linear map has twice its complex rank (its image is the realification of the complex image), so use the vanishing of all $(2k+1)$-minors of the real matrix, by A matrix has rank at least $r$ exactly when it has a nonzero $r$-rowed minor; the condition is vacuous when $k\ge d$. Thus every $K_k$ is compact by Tychonoff's theorem: an arbitrary product of compact spaces is compact in the product topology, assuming the Axiom of Choice. Put $G={\bigcup }_{k\ge 1}{K}_k$ with the final topology of this increasing compact sequence. It is Hausdorff, since that topology contains the ambient product topology. The finite-product theorem for these direct limits (Gloeckner--Gramlich--Hartnick, Proposition 4.7, printed pp. 12--13) identifies $G\times G$ with . Indeed every compact Hausdorff stage is a $k_{\omega }$ space, using its constant compact exhaustion. Coordinatewise multiplication restricts continuously to $K_k\times K_k\to K_{2k}$ because $g{g}^{\mathrm{\prime }}-I=(g-I)+g(g^{\mathrm{\prime }}-I)$ and ranks are subadditive; inversion preserves $K_k$ because $g^{-1}-I=-g^{-1}(g-I)$. Thus $G$ is a Hausdorff topological group.

Every compact subset of $G$ lies in one $K_k$. Otherwise choose distinct points $x_n$ of that compact subset outside $K_n$. Every subset of $\{x_n:n\ge 1\}$ has finite, hence closed, intersection with each $K_k$, so is closed in the final topology. This would give an infinite closed discrete subspace of a compact Hausdorff space, a contradiction. The representation $\rho ={\widehat{⨁}}_{d\ge 1}{\mathbb{C}}^d$ with coordinatewise standard action is strongly continuous: each orbit map is continuous in the ambient product topology by truncating its square-summable tail, hence in the finer final topology (Hilbert direct sums of unitary representations).

The functions ${\varphi }_d(g)=d^{-1}\mathrm{tr}(g_d)$ are finite sums of diagonal coefficients of $\rho$, using the vectors $d^{-1\mathrm{/}2}{e}_1,\dots ,d^{-1\mathrm{/}2}{e}_d$ in the $d$th summand. By Complex spectral theorem: a normal endomorphism of a finite-dimensional complex inner product space has an orthonormal eigenbasis, and conversely, a unitary has an orthonormal eigenbasis; all eigenvalues have modulus one, and $\mathrm{rank}(g_d-I)\le k$ allows at most $k$ nonidentity eigenvalues. Hence ${\sup }_{K_k}\mathrm{\mid }1-{\varphi }_d\mathrm{\mid }\le 2k\mathrm{/}d$. Compact containment therefore gives $1_G\prec \rho$. However, $K_1$ is compact, and for any unit vector $\xi =({\xi }_d)$ choose $g_d$ to act as $-I$ on $\mathbb{C}{\xi }_d$ and as $I$ on its orthogonal complement when ${\xi }_d\mathrm{\ne }0$, and to be the identity otherwise. Then $g\in K_1$ and $\rho (g)\xi =-\xi$, so ${\sup }_{g\in K_1}\mathrm{\parallel }\rho (g)\xi -\xi \mathrm{\parallel }=2$. There are no almost invariant unit vectors. This proves that the general topological-group version of the equivalence is false.