Self-Paced Curriculum Learning

TLDR

Curriculum learning (CL) and self‑paced learning (SPL) are learning regimes that progress from easy to complex samples, but CL relies on a fixed curriculum derived from prior knowledge while SPL adapts to the learner yet ignores prior knowledge, making each vulnerable to the quality of prior knowledge or overfitting. This work identifies the missing link between CL and SPL and introduces a unified self‑paced curriculum learning (SPCL) framework. SPCL is formulated as a concise optimization problem that jointly incorporates prior knowledge available before training and the learner’s progress during training. Experimental results on two tasks demonstrate that SPCL outperforms both CL and SPL.

Abstract

Curriculum learning (CL) or self-paced learning (SPL) represents a recently proposed learning regime inspired by the learning process of humans and animals that gradually proceeds from easy to more complex samples in training. The two methods share a similar conceptual learning paradigm, but differ in specific learning schemes. In CL, the curriculum is predetermined by prior knowledge, and remain fixed thereafter. Therefore, this type of method heavily relies on the quality of prior knowledge while ignoring feedback about the learner. In SPL, the curriculum is dynamically determined to adjust to the learning pace of the leaner. However, SPL is unable to deal with prior knowledge, rendering it prone to overfitting. In this paper, we discover the missing link between CL and SPL, and propose a unified framework named self-paced curriculum leaning (SPCL). SPCL is formulated as a concise optimization problem that takes into account both prior knowledge known before training and the learning progress during training. In comparison to human education, SPCL is analogous to "instructor-student-collaborative" learning mode, as opposed to "instructor-driven" in CL or "student-driven" in SPL. Empirically, we show that the advantage of SPCL on two tasks.