The higher spin Laplace operator has been constructed recently as the generalization of the Laplacian in higher spin theory. This acts on functions taking values in arbitrary irreducible representations of the Spin group. In this paper, we first provide a decomposition of the higher spin Laplace operator in terms of Rarita-Schwinger operators. With such a decomposition, a connection between the fundamental solutions for the higher spin Laplace operator and the fundamental solutions for the Rarita-Schwinger operators is provided. Further, we show that the two components in this decomposition are conformally invariant differential operators. An alternative proof for the conformal invariance property is also pointed out, which can be connected to Knapp-Stein intertwining operators. Last but not least, we establish a Borel-Pompeiu type formula for the higher spin Laplace operator. As an application, we give a Green type integral formula.