Context-Sensitive MTL Networks for Machine Lifelong Learning.

Abstract

Context-sensitive Multiple Task Learning, or csMTL, is presented as a method of inductive transfer that uses a single output neural network and additional contextual inputs for learning multiple tasks. The csMTL method is tested on three task domains and shown to produce hypotheses for a primary task that are significantly bet-ter than standard MTL hypotheses when learning in the presence of related and unrelated tasks. A new measure of task relatedness, based on the context input weights, is shown to have promise. The paper also outlines a ma-chine lifelong learning system that uses csMTL for se-quentially learning multiple tasks. The approach satis-fies a number of important requirements for knowledge retention and inductive transfer including the elimina-tion of redundant outputs, representational transfer for rapid but effective short-term learning and functional transfer via task rehearsal for long-term consolidation.