Abstract
Hierarchical stochastic processes, such as the hierarchical Dirichlet process, hold an important position as a modelling tool in statistical machine learning, and are even used in deep neural networks. They allow, for instance, networks of probability vectors to be used in general statistical modelling, intrinsically supporting information sharing through the network. This paper presents a general theory of hierarchical stochastic processes and illustrates its use on the gamma process and the generalised gamma process. In general, most of the convenient properties of hierarchical Dirichlet processes extend to the broader family. The main construction for this corresponds to estimating the moments of an infinitely divisible distribution based on its cumulants. Various equivalences and relationships can then be applied to networks of hierarchical processes. Examples given demonstrate the duplication in non-parametric research, and presents plots of the Pitman–Yor distribution.
Subject
General Physics and Astronomy
Reference48 articles.
1. Teh, Y. (2006, January 17–21). A hierarchical Bayesian language model based on Pitman-Yor processes. Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the ACL; ACL ’06, Sydney, Australia.
2. Kneser, R., and Ney, H. (1995, January 9–12). Improved backing-off for m-gram language modeling. Proceedings of the Internatinal Conference on Acoustics, Speech, and Signal Processing, Detroit, MI, USA.
3. Hierarchical Dirichlet Processes;J. ASA,2006
4. Buntine, W., and Mishra, S. (2014, January 24–27). Experiments with Non-parametric Topic Models. Proceedings of the 20th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, New York, NY, USA.
5. Improvements to the Sequence Memoizer;Adv. Neural Inf. Process. Syst.,2010