Factored Latent-Dynamic Conditional Random Fields for Single and\n Multi-label Sequence Modeling

Abstract

Conditional Random Fields (CRF) are frequently applied for labeling and\nsegmenting sequence data. Morency et al. (2007) introduced hidden state\nvariables in a labeled CRF structure in order to model the latent dynamics\nwithin class labels, thus improving the labeling performance. Such a model is\nknown as Latent-Dynamic CRF (LDCRF). We present Factored LDCRF (FLDCRF), a\nstructure that allows multiple latent dynamics of the class labels to interact\nwith each other. Including such latent-dynamic interactions leads to improved\nlabeling performance on single-label and multi-label sequence modeling tasks.\nWe apply our FLDCRF models on two single-label (one nested cross-validation)\nand one multi-label sequence tagging (nested cross-validation) experiments\nacross two different datasets - UCI gesture phase data and UCI opportunity\ndata. FLDCRF outperforms all state-of-the-art sequence models, i.e., CRF,\nLDCRF, LSTM, LSTM-CRF, Factorial CRF, Coupled CRF and a multi-label LSTM model\nin all our experiments. In addition, LSTM based models display inconsistent\nperformance across validation and test data, and pose diffculty to select\nmodels on validation data during our experiments. FLDCRF offers easier model\nselection, consistency across validation and test performance and lucid model\nintuition. FLDCRF is also much faster to train compared to LSTM, even without a\nGPU. FLDCRF outshines the best LSTM model by ~4% on a single-label task on UCI\ngesture phase data and outperforms LSTM performance by ~2% on average across\nnested cross-validation test sets on the multi-label sequence tagging\nexperiment on UCI opportunity data. The idea of FLDCRF can be extended to joint\n(multi-agent interactions) and heterogeneous (discrete and continuous) state\nspace models.\n