# Dynamic Conditional Random Fields: Factorized Probabilistic Models for Labeling and Segmenting Sequence Data

Charles Sutton, Andrew McCallum, Khashayar Rohanimanesh.

Year: 2007, Volume: 8, Issue: 25, Pages: 693−723

#### Abstract

In sequence modeling, we often wish to represent complex interaction
between labels, such as when performing multiple, cascaded labeling
tasks on the same sequence, or when long-range dependencies exist. We
present *dynamic conditional random fields (DCRFs)*, a
generalization of linear-chain conditional random fields (CRFs) in
which each time slice contains a set of state variables and edges---a
distributed state representation as in dynamic Bayesian networks
(DBNs)---and parameters are tied across slices. Since exact inference
can be intractable in such models, we perform approximate inference
using several schedules for belief propagation, including tree-based
reparameterization (TRP). On a natural-language chunking task, we show
that a DCRF performs better than a series of linear-chain CRFs,
achieving comparable performance using only half the training data.
In addition to maximum conditional likelihood, we present two
alternative approaches for training DCRFs: *marginal likelihood
training*, for when we are primarily interested in predicting only a
subset of the variables, and *cascaded training*, for when we
have a distinct data set for each state variable, as in transfer
learning. We evaluate marginal training and cascaded training on both
synthetic data and real-world text data, finding that marginal
training can improve accuracy when uncertainty exists over the latent
variables, and that for transfer learning, a DCRF trained in a
cascaded fashion performs better than a linear-chain CRF that predicts
the final task directly.