Inductive Principles for Restricted Boltzmann Machine Learning

Benjamin Marlin, Kevin Swersky, Bo Chen, Nando de Freitas ; JMLR W&CP 9:509-516, 2010.

Abstract

Recent research has seen the proposal of several new inductive principles designed specifically to avoid the problems associated with maximum likelihood learning in models with intractable partition functions. In this paper, we study learning methods for binary restricted Boltzmann machines (RBMs) based on ratio matching and generalized score matching. We compare these new RBM learning methods to a range of existing learning methods including stochastic maximum likelihood, contrastive divergence, and pseudo-likelihood. We perform an extensive empirical evaluation across multiple tasks and data sets.