arXiv (Cornell University) · 2020 · 11 citations · 39 references
ComputingComputer ScienceAlgorithmsArtificial IntelligenceMachine LearningDeep LearningNeural NetworksRepresentation LearningAutomatic Feature ConstructionBenchmark DatasetsDomain ConstraintsHypothesis SpaceInductive BiasMonotonic ModelsMonotonicity AssumptionMonotonicity Constraints
The widespread adoption of deep learning is often attributed to its automatic feature construction with minimal inductive bias. However, in many real-world tasks, the learned function is intended to satisfy domain-specific constraints. We focus on monotonicity constraints, which are common and require that the function's output increases with increasing values of specific input features. We develop a counterexample-guided technique to provably enforce monotonicity constraints at prediction time. Additionally, we propose a technique to use monotonicity as an inductive bias for deep learning. It works by iteratively incorporating monotonicity counterexamples in the learning process. Contrary to prior work in monotonic learning, we target general ReLU neural networks and do not further restrict the hypothesis space. We have implemented these techniques in a tool called COMET. Experiments on real-world datasets demonstrate that our approach achieves state-of-the-art results compared to existing monotonic learners, and can improve the model quality compared to those that were trained without taking monotonicity constraints into account.
39
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
UCI Repository of machine learning databases
Catherine Blake · Medical Entomology and Zoology · 1998 · 10.5K citations
Deep Sparse Rectifier Neural Networks
Xavier Glorot, Antoine Bordes, Yoshua Bengio · 2012 · 5.4K citations
End to End Learning for Self-Driving Cars
Mariusz Bojarski, Davide Testa, Daniel Dworakowski et al. · arXiv (Cornell University) · 2016 · 3.1K citations · Full text
Equality of Opportunity in Supervised Learning
Moritz Hardt, Eric Price, Nathan Srebro · arXiv (Cornell University) · 2016 · 1.9K citations · Full text