2007 · 101 citations · 9 references
ComputingComputer ScienceArtificial IntelligenceMachine LearningNatural Language ProcessingMulti-task LearningOptimization For Machine LearningPattern RecognitionSupport Vector MachinesMultitask LearningOrdinal RegressionOrdinal Regression ModelsProtein Family ClassificationSecond-order Cone ProgrammingTask Relatedness Modeling
When we have several related tasks, solving them simultaneously is shown to be more effective than solving them individually. This approach is called multi-task learning (MTL) and has been studied extensively. Existing approaches to MTL often treat all the tasks as uniformly related to each other and the relatedness of the tasks is controlled globally. For this reason, the existing methods can lead to undesired solutions when some tasks are not highly related to each other, and some pairs of related tasks can have significantly different solutions. In this paper, we propose a novel MTL algorithm that can overcome these problems. Our method makes use of a task network, which describes the relation structure among tasks. This allows us to deal with intricate relation structures in a systematic way. Furthermore, we control the relatedness of the tasks locally, so all pairs of related tasks are guaranteed to have similar solutions. We apply the above idea to support vector machines (SVMs) and show that the optimization problem can be cast as a second order cone program, which is convex and can be solved efficiently. The usefulness of our approach is demonstrated through simulations with protein super-family classification and ordinal regression problems. 1
9
Applications of second-order cone programming
Miguel Sousa Lobo, Lieven Vandenberghe, Stephen Boyd et al. · Linear Algebra and its Applications · 1998 · 2.4K citations
Regularized multi--task learning
Theodoros Evgeniou, Massimiliano Pontil · 2004 · 1.5K citations
Convolution kernels on discrete structures
David Haussler · 1999 · 1.1K citations
CSDP, A C library for semidefinite programming
Brian Borchers · Optimization methods & software · 1999 · 515 citations