Symbolic conditioning of arrays in probabilistic programs

Abstract

Probabilistic programming systems make machine learning more modular by automating inference . Recent work by Shan and Ramsey makes inference more modular by automating conditioning . Their technique uses a symbolic program transformation that treats conditioning generally via the measure-theoretic notion of disintegration . This technique, however, is limited to conditioning a single scalar variable. As a step towards modular inference for realistic machine learning applications, we have extended the disintegration algorithm to symbolically condition arrays in probabilistic programs. The extended algorithm implements lifted disintegration , where repetition is treated symbolically and without unrolling loops. The technique uses a language of index variables for tracking expressions at various array levels. We find that the method works well for arbitrarily-sized arrays of independent random choices, with the conditioning step taking time linear in the number of indices needed to select an element.