Bernoulli Mixture Models for Markov Blanket Filtering and Classification
Mehreen Saeed; JMLR W&CP 3:77:91, 2008.
This paper presents the use of Bernoulli mixture models for Markov blanket
filtering and classification of binary data. Bernoulli mixture models can
be seen as a tool for partitioning an n-dimensional hypercube, identifying
regions of high data density on the corners of the hypercube. Once Bernoulli
mixture models are computed from a training dataset we use them for determining
the Markov blanket of the target variable. An algorithm for Markov blanket
filtering was proposed by Koller and Sahami (1996), which is a greedy search
method for feature subset selection and it outputs an approximation to the
optimal feature selection criterion. However, they use the entire training
instances for computing the conditioning sets and have to limit the size
of these sets for computational efficiency and avoiding data fragmentation.
We have adapted their algorithm to use Bernoulli mixture models instead,
hence, overcoming the short comings of their algorithm and increasing the
efficiency of this algorithm considerably. Once a feature subset is identified
we perform classification using these mixture models. We have applied this
algorithm to the causality challenge datasets. Our prediction scores were
ranked fourth on SIDO and our feature scores were ranked the best for test
sets 1 and 2 of the same dataset.