Restriction of the regular representation to a closed subgroup

Fix $f\in C_c(G)$ and put $S=\mathrm{supp}f$. For $x\in G$ define $a_x\in C_c(H)$ by $a_x(h):={\left(\frac{{\mathrm{\Delta }}_G((xh{)}^{-1})}{r(xh)}\right)}^{1\mathrm{/}2}f((xh{)}^{-1})$. Its support is contained in the compact set $x^{-1}{S}^{-1}\cap H$. For $k\in H$ put $B(xH,k):=⟨R_H(k)a_x,a_x⟩$. If $t\in H$, then $a_{xt}(h)=a_x(th)$; changing $h$ by the left translation $t$ in the Haar integral shows $B(xtH,k)=B(xH,k)$. Thus $B$ is well defined on $(G\mathrm{/}H)\times H$.

The function $(x,k,h)⟼{\mathrm{\Delta }}_H(k{)}^{1\mathrm{/}2}{a}_x(hk)\overline{a_x(h)}$ is continuous. At each $(x_0,k_0)\in G\times H$, choose compact neighborhoods $C\ni x_0$ and $K\ni k_0$ by [F2]. For $x\in C$ and $k\in K$, its support in $h$ lies in the fixed compact set $D:=C^{-1}{S}^{-1}\cap H$, because the second factor vanishes unless $xh\in S^{-1}$; compactness of $D$ follows from [F9]. The support of the restricted kernel on $C\times K\times H$ is contained in the compact product $C\times K\times D$, which is compact by A product of finitely many compact spaces is compact in the product topology. The compact-kernel result [F5] therefore makes the integral over $h$ continuous jointly in $(x,k)$ near $(x_0,k_0)$. The map $p\times {\mathrm{id}}_H$ is open on product-basis rectangles and surjective, hence is a quotient map; since $B$ is constant on its fibers, it descends continuously. If $q\mathrm{\notin }T:=p(S^{-1})$, no representative $x$ has $xh\in S^{-1}$, so $a_x=0$ and $B(q,k)=0$. The set $T$ is compact by continuity of $p$ and [F9].

The coefficient $c_f(k):=⟨{\lambda }_G(k)f,f⟩$ has the formula $c_f(k)={\int }_{G\mathrm{/}H}B(q,k)d\mu (q)$. Indeed, inversion [F4] gives $c_f(k)={\int }_G{\mathrm{\Delta }}_G(y^{-1})f((yk{)}^{-1})\overline{f(y^{-1})}dy$. The inversion input is in $C_c(G)$, so its weighted absolute integral is finite by continuity of ${\mathrm{\Delta }}_G$ and compact finiteness of Haar measure. The resulting integrand is continuous and supported in the compact set $S^{-1}\cap S^{-1}{k}^{-1}$. Apply the Weil formula [F1] after dividing it by $r(y)$; for a complex integrand apply the real formula to its real and imaginary parts. At $y=xh$ the resulting integrand is $\frac{{\mathrm{\Delta }}_G((xh{)}^{-1})}{r(xh)}f((xhk{)}^{-1})\overline{f((xh{)}^{-1})}$. The rho covariance [F3] gives ${\mathrm{\Delta }}_H(k{)}^{1\mathrm{/}2}{a}_x(hk)\overline{a_x(h)}=\frac{{\mathrm{\Delta }}_G((xh{)}^{-1})}{r(xh)}f((xhk{)}^{-1})\overline{f((xh{)}^{-1})}$, which is exactly the inner product defining $B(xH,k)$. Since $B$ vanishes off compact $T$ and $\mu (T)<\mathrm{\infty }$, this quotient integral is finite.

Fix a compact $Q\subseteq H$ and $ϵ>0$. If $Q=\mathrm{\varnothing }$, the approximation condition is vacuous, so take one zero vector. If $\mu (T)=0$, the coefficient formula is zero on $Q$, so again take one zero vector. Otherwise $\mu (T)>0$. Set $\delta :=ϵ\mathrm{/}(2\mu (T))>0$. For each $q_0\in T$, joint continuity of $B$ gives, at each $k_0\in Q$, neighborhoods $U_{k_0}$ of $q_0$ and $V_{k_0}$ of $k_0$ such that both $B(q^{\mathrm{\prime }},k)$ and $B(q^{\mathrm{\prime }\mathrm{\prime }},k)$ lie within $\delta \mathrm{/}2$ of $B(q_0,k_0)$ whenever $q^{\mathrm{\prime }},q^{\mathrm{\prime }\mathrm{\prime }}\in U_{k_0}\cap T$ and $k\in V_{k_0}$. By [F11], compactness of $Q$ gives finitely many $V_{k_0}$ covering it; intersect their corresponding $U_{k_0}$ to obtain a neighborhood $U_{q_0}$ on which $\mathrm{\mid }B(q^{\mathrm{\prime }},k)-B(q^{\mathrm{\prime }\mathrm{\prime }},k)\mathrm{\mid }<\delta$ for all $q^{\mathrm{\prime }},q^{\mathrm{\prime }\mathrm{\prime }}\in U_{q_0}\cap T$ and $k\in Q$. By [F11], compactness of $T$ gives a finite subcover $U_1,\dots ,U_N$. The compact set $T$ is closed in the Hausdorff space $G\mathrm{/}H$ by [F9]; form the disjoint Borel partition $E_j=(U_j\cap T)\setminus {\bigcup }_{i<j}{U}_i$, omitting empty pieces. It covers $T$, and each $E_j\subseteq U_j$. Choose $q_j\in E_j$ and a lift $x_j\in G$ with $p(x_j)=q_j$. Radon finiteness gives $\mu (E_j)<\mathrm{\infty }$. The square root in $v_j:=\mu (E_j{)}^{1\mathrm{/}2}{a}_{x_j}$ exists by [F10], and $⟨R_H(k)v_j,v_j⟩=\mu (E_j)B(q_j,k)$. Since the $E_j$ partition $T$ and $B$ vanishes off $T$, $c_f(k)={\sum }_j{\int }_{E_j}B(q,k)d\mu (q)$. Comparing each integral with $\mu (E_j)B(q_j,k)$ gives ${\sup }_{k\in Q}\mid c_f(k)-{\sum }_j⟨R_H(k)v_j,v_j⟩\mid \le \delta \mu (T)=ϵ\mathrm{/}2<ϵ$. Thus every $C_c(G)$ diagonal coefficient is uniformly approximated on $Q$ by a finite sum of right-regular diagonal coefficients.

Define $J:C_c(H)\to C_c(H)$ by $(Jb)(h):={\mathrm{\Delta }}_H(h{)}^{-1\mathrm{/}2}b(h^{-1})$. The function $Jb$ is continuous with compact support because inversion is a homeomorphism and ${\mathrm{\Delta }}_H$ is positive continuous. Applying inversion [F4] to $u(h)={\mathrm{\Delta }}_H(h)\mathrm{\mid }b(h){\mathrm{\mid }}^2$ gives $\mathrm{\parallel }Jb{\mathrm{\parallel }}_2^2={\int }_H{\mathrm{\Delta }}_H(h{)}^{-1}\mathrm{\mid }b(h^{-1}){\mathrm{\mid }}^{2}dh={\int }_H\mathrm{\mid }b(h){\mathrm{\mid }}^{2}dh=\mathrm{\parallel }b{\mathrm{\parallel }}_2^2$. The homomorphism law in [F3] gives $J^{2}b(h)={\mathrm{\Delta }}_H(h{)}^{-1\mathrm{/}2}{\mathrm{\Delta }}_H(h^{-1}{)}^{-1\mathrm{/}2}b(h)=b(h)$. Also $J{\lambda }_H(k)b(h)={\mathrm{\Delta }}_H(h{)}^{-1\mathrm{/}2}b((hk{)}^{-1})$, while $R_H(k)Jb(h)={\mathrm{\Delta }}_H(k{)}^{1\mathrm{/}2}{\mathrm{\Delta }}_H(hk{)}^{-1\mathrm{/}2}b((hk{)}^{-1})={\mathrm{\Delta }}_H(h{)}^{-1\mathrm{/}2}b((hk{)}^{-1})$, so $J{\lambda }_H(k)=R_H(k)J$. By density and completeness in [F6], $J$ extends to an isometry on $L^2(H)$; $J^2=I$ makes it onto, hence unitary. Thus $⟨R_H(k)v_j,v_j⟩=⟨{\lambda }_H(k)J{v}_j,J{v}_j⟩$, and replacing every $v_j$ in step 4.1 by $J{v}_j$ converts its sum to left-regular coefficients.

Now let $\xi \in L^2(G)$, compact $Q\subseteq H$, and $ϵ>0$. Set $\alpha :=\min \{1,ϵ\mathrm{/}(8(\mathrm{\parallel }\xi {\mathrm{\parallel }}_2+1))\}>0$. By $C_c(G)$-density [F6] choose $f\in C_c(G)$ with $\mathrm{\parallel }\xi -f{\mathrm{\parallel }}_2<\alpha$. Then $(\mathrm{\parallel }\xi {\mathrm{\parallel }}_2+\mathrm{\parallel }f{\mathrm{\parallel }}_2)\mathrm{\parallel }\xi -f{\mathrm{\parallel }}_2\le (2\mathrm{\parallel }\xi {\mathrm{\parallel }}_2+\alpha )\alpha <ϵ\mathrm{/}2$. For every $k\in H$, unitarity and Cauchy--Schwarz [F8] give $\mathrm{\mid }⟨{\lambda }_G(k)\xi ,\xi ⟩-⟨{\lambda }_G(k)f,f⟩\mathrm{\mid }\le (\mathrm{\parallel }\xi {\mathrm{\parallel }}_2+\mathrm{\parallel }f{\mathrm{\parallel }}_2)\mathrm{\parallel }\xi -f{\mathrm{\parallel }}_2<ϵ\mathrm{/}2$. Apply steps 4.1 and 5.1 to $f$, $Q$, and tolerance $ϵ\mathrm{/}2$, and combine the two bounds. The resulting finite sum of ${\lambda }_H$ diagonal coefficients approximates the coefficient of $\xi$ within $ϵ$ uniformly on $Q$. By [F7] this is ${\lambda }_G{\mathrm{\mid }}_H\prec {\lambda }_H$.