Affiliation:
1. University of Waterloo, Canada
Abstract
In probabilistic programming languages (PPLs), a critical step in optimization-based inference methods is constructing, for a given model program, a trainable guide program.
Soundness and effectiveness of inference rely on constructing good guides, but the expressive power of a universal PPL poses challenges.
This paper introduces an approach to automatically generating guides for deep amortized inference in a universal PPL.
Guides are generated using a type-directed translation per a novel behavioral type system.
Guide generation extracts and exploits independence structures using a syntactic approach to conditional independence, with a semantic account left to further work.
Despite the control-flow expressiveness allowed by the universal PPL, generated guides are guaranteed to satisfy a critical soundness condition and moreover, consistently improve training and inference over state-of-the-art baselines for a suite of benchmarks.
Publisher
Association for Computing Machinery (ACM)
Subject
Safety, Risk, Reliability and Quality,Software
Reference48 articles.
1. 2022. https://docs.pyro.ai/en/1.8.0/infer.autoguide.html 2022. https://docs.pyro.ai/en/1.8.0/infer.autoguide.html
2. Guillaume Baudart and Louis Mandel . 2021 . Automatic guide generation for Stan via NumPyro . In Int’l Conf. on Probabilistic Programming (PROBPROG). arxiv:2110 .11790. Guillaume Baudart and Louis Mandel. 2021. Automatic guide generation for Stan via NumPyro. In Int’l Conf. on Probabilistic Programming (PROBPROG). arxiv:2110.11790.
3. Pyro: Deep universal probabilistic programming;Bingham Eli;Journal of Machine Learning Research (JMLR),2019
4. A lambda-calculus foundation for universal probabilistic programming
5. Gen: a general-purpose probabilistic programming system with programmable inference
Cited by
1 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. Probabilistic Programming with Stochastic Probabilities;Proceedings of the ACM on Programming Languages;2023-06-06