Hierarchical Gaussian Process Regression
Sunho Park (POSTECH) and Seungjin Choi (POSTECH);
JMLR W&P 13:95-110, 2010.
Abstract
We address an approximation method for Gaussian process (GP) regression,
where we approximate covariance by a block matrix such that
diagonal blocks are calculated exactly while off-diagonal blocks are
approximated. Partitioning input data points, we present a two-layer
hierarchical model for GP regression, where prototypes of clusters in
the upper layer are involved for coarse modeling by a GP and data
points in each cluster in the lower layer are involved for fine modeling
by an individual GP whose prior mean is given by the corresponding
prototype and covariance is parameterized by data points in the
partition. In this hierarchical model, integrating out latent variables
in the upper layer leads to a block covariance matrix, where diagonal
blocks contain similarities between data points in the same partition
and off-diagonal blocks consist of approximate similarities calculated
using prototypes. This particular structure of the covariance matrix
divides the full GP into a pieces of manageable sub-problems whose
complexity scales with the number of data points in a partition. In
addition, our hierarchical GP regression (HGPR) is also useful for
cases where partitions of data reveal different characteristics. Experiments
on several benchmark datasets confirm the useful behavior of
our method.
