## Augmentable Gamma Belief Networks

*Mingyuan Zhou, Yulai Cong, Bo Chen*; 17(163):1−44, 2016.

### Abstract

To infer multilayer deep representations of high-dimensional
discrete and nonnegative real vectors, we propose an augmentable
gamma belief network (GBN) that factorizes each of its hidden
layers into the product of a sparse connection weight matrix and
the nonnegative real hidden units of the next layer. The GBN's
hidden layers are jointly trained with an upward-downward Gibbs
sampler that solves each layer with the same subroutine. The
gamma-negative binomial process combined with a layer-wise
training strategy allows inferring the width of each layer given
a fixed budget on the width of the first layer. Example results
illustrate interesting relationships between the width of the
first layer and the inferred network structure, and demonstrate
that the GBN can add more layers to improve its performance in
both unsupervisedly extracting features and predicting heldout
data. For exploratory data analysis, we extract trees and
subnetworks from the learned deep network to visualize how the
very specific factors discovered at the first hidden layer and
the increasingly more general factors discovered at deeper
hidden layers are related to each other, and we generate
synthetic data by propagating random variables through the deep
network from the top hidden layer back to the bottom data layer.

[abs][pdf][bib]