ComputingComputer ScienceComputational LinguisticsMachine LearningNatural Language ProcessingDeep LearningGenerative ModelingLanguage ModelingLoss FunctionPattern RecognitionRepresentation LearningDiscriminative FunctionsLogistic Regression
, which are programs that label subsets of the data, but that are noisy and may conflict. We show that by explicitly representing this training set labeling process as a generative model, we can "denoise" the generated training set, and establish theoretically that we can recover the parameters of these generative models in a handful of settings. We then show how to modify a discriminative loss function to make it noise-aware, and demonstrate our method over a range of discriminative models including logistic regression and LSTMs. Experimentally, on the 2014 TAC-KBP Slot Filling challenge, we show that data programming would have led to a new winning score, and also show that applying data programming to an LSTM model leads to a TAC-KBP score almost 6 F1 points over a state-of-the-art LSTM baseline (and into second place in the competition). Additionally, in initial user studies we observed that data programming may be an easier way for non-experts to create machine learning models when training data is limited or unavailable.
5
Sepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 98.1K citations
Incremental knowledge base construction using DeepDive
Jaeho Shin, Sen Wu, Feiran Wang et al. · Proceedings of the VLDB Endowment · 2015 · 235 citations