Epsilon Dichotomy for Linear Models: The Archimedean Case

Abstract

Abstract Let $G=\textrm {GL}_{2n}({\mathbb {R}})$ or $G=\textrm {GL}_n({\mathbb {H}})$ and $H=\textrm {GL}_n({\mathbb {C}})$ regarded as a subgroup of $G$. Here, ${\mathbb {H}}$ is the quaternion division algebra over ${\mathbb {R}}$. For a character $\chi $ on ${\mathbb {C}}^\times $, we say that an irreducible smooth admissible moderate growth representation $\pi $ of $G$ is $\chi _H$-distinguished if $\operatorname {Hom}_H(\pi , \chi \circ \det _H)\neq 0$. We compute the root number of a $\chi _H$-distinguished representation $\pi $ twisted by the representation induced from $\chi $. This proves an Archimedean analogue of the conjecture by Prasad and Takloo-Bighash (J. Reine Angew. Math., 2011). The proof is based on the analysis of the contribution of $H$-orbits in a flag manifold of $G$ to the Schwartz homology of principal series representations. A large part of the argument is developed for general real reductive groups of inner type. In particular, we prove that the Schwartz homology $H_\ast (H, \pi \otimes \chi )$ is finite-dimensional and hence it is Hausdorff for a reductive symmetric pair $(G, H)$ and a finite-dimensional representation $\chi $ of $H$.