Home Page

Papers

Submissions

News

Editorial Board

Special Issues

Open Source Software

Proceedings (PMLR)

Transactions (TMLR)

Search

Statistics

Login

Frequently Asked Questions

Contact Us



RSS Feed

Variational Inference in high-dimensional linear regression

Sumit Mukherjee, Subhabrata Sen; 23(304):1−56, 2022.

Abstract

We study high-dimensional bayesian linear regression with product priors. Using the nascent theory of “non-linear large deviations" (Chatterjee and Dembo, 2016), we derive sufficient conditions for the leading-order correctness of the naive mean-field approximation to the log-normalizing constant of the posterior distribution. Subsequently, assuming a true linear model for the observed data, we derive a limiting infinite dimensional variational formula for the log normalizing constant for the posterior. Furthermore, we establish that under an additional “separation" condition, the variational problem has a unique optimizer, and this optimizer governs the probabilistic properties of the posterior distribution. We provide intuitive sufficient conditions for the validity of this “separation" condition. Finally, we illustrate our results on concrete examples with specific design matrices.

[abs][pdf][bib]       
© JMLR 2022. (edit, beta)

Mastodon