Home Page

Papers

Submissions

News

Editorial Board

Proceedings

Open Source Software

Search

Statistics

Login

Frequently Asked Questions

Contact Us



RSS Feed

A Generalised Linear Model Framework for β-Variational Autoencoders based on Exponential Dispersion Families

Robert Sicks, Ralf Korn, Stefanie Schwaar; 22(233):1−41, 2021.

Abstract

Although variational autoencoders (VAE) are successfully used to obtain meaningful low-dimensional representations for high-dimensional data, the characterization of critical points of the loss function for general observation models is not fully understood. We introduce a theoretical framework that is based on a connection between β-VAE and generalized linear models (GLM). The equality between the activation function of a β-VAE and the inverse of the link function of a GLM enables us to provide a systematic generalization of the loss analysis for β-VAE based on the assumption that the observation model distribution belongs to an exponential dispersion family (EDF). As a result, we can initialize β-VAE nets by maximum likelihood estimates (MLE) that enhance the training performance on both synthetic and real world data sets. As a further consequence, we analytically describe the auto-pruning property inherent in the β-VAE objective and reason for posterior collapse.

[abs][pdf][bib]       
© JMLR 2021. (edit, beta)