Abstract
Abstract
We consider an empirical Bayes method for Boltzmann machines and propose an algorithm for it. The empirical Bayes method allows for estimation of the values of the hyperparameters of the Boltzmann machine by maximizing a specific likelihood function referred to as the empirical Bayes likelihood function in this study. However, the maximization is computationally hard because the empirical Bayes likelihood function involves intractable integrations of the partition function. The proposed algorithm avoids this computational problem by using the replica method and the Plefka expansion. Our method is quite simple and fast because it does not require any iterative procedures and gives reasonable estimates at a certain condition. However, our method introduces a bias to the estimate, which exhibits an unnatural behavior with respect to the size of the dataset. This peculiar behavior is supposed to be due to the approximate treatment by the Plefka expansion. A possible extension to overcome this behavior is also discussed.
Funder
The COI Program from the JST
JST CREST
JSPS KAKENHI, Japan
Basic Science Research Projects from the Sumitomo Foundation
Subject
General Physics and Astronomy,Mathematical Physics,Modelling and Simulation,Statistics and Probability,Statistical and Nonlinear Physics
Cited by
1 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. Empirical Bayes Method for Boltzmann Machines;Sublinear Computation Paradigm;2021-10-20