Sonine Formulas and Intertwining Operators in Dunkl Theory

Abstract

Abstract Let $V_k$ denote Dunkl’s intertwining operator associated with some root system $R$ and multiplicity $k$. For two multiplicities $k, k^{\prime }$ on $R$, we study the intertwiner $V_{k^{\prime },k} = V_{k^{\prime }}\circ V_k^{-1}$ between Dunkl operators with multiplicities $k$ and $k^{\prime }.$ It has been a long-standing conjecture that $V_{k^{\prime },k}$ is positive if $k^{\prime } \geq k \geq 0.$ We disprove this conjecture by constructing counterexamples for root system $B_n$. This matter is closely related to the existence of Sonine-type integral representations between Dunkl kernels and Bessel functions with different multiplicities. In our examples, such Sonine formulas do not exist. As a consequence, we obtain necessary conditions on Sonine formulas for Heckman–Opdam hypergeometric functions of type $BC_n$ and conditions for positive branching coefficients between multivariable Jacobi polynomials.