Schur $P$-positivity and Involution Stanley Symmetric Functions

Abstract

Abstract The involution Stanley symmetric functions$\hat{F}_y$ are the stable limits of the analogs of Schubert polynomials for the orbits of the orthogonal group in the flag variety. These symmetric functions are also generating functions for involution words and are indexed by the involutions in the symmetric group. By construction, each $\hat{F}_y$ is a sum of Stanley symmetric functions and therefore Schur positive. We prove the stronger fact that these power series are Schur $P$-positive. We give an algorithm to efficiently compute the decomposition of $\hat{F}_y$ into Schur $P$-summands and prove that this decomposition is triangular with respect to the dominance order on partitions. As an application, we derive pattern avoidance conditions which characterize the involution Stanley symmetric functions which are equal to Schur $P$-functions. We deduce as a corollary that the involution Stanley symmetric function of the reverse permutation is a Schur $P$-function indexed by a shifted staircase shape. These results lead to alternate proofs of theorems of Ardila–Serrano and DeWitt on skew Schur functions which are Schur $P$-functions. We also prove new Pfaffian formulas for certain related involution Schubert polynomials.