A theory of the learnable

TLDR

Humans can acquire new concepts without explicit programming. The paper defines learning as knowledge acquisition without explicit programming and proposes a computational methodology to study it. It consists of selecting an information‑gathering mechanism, a learning protocol, and exploring the concept class learnable in polynomial steps. Algorithmic complexity limits the concepts learnable in polynomial time, and the results provide principles for designing realistic learning systems.

Abstract

Humans appear to be able to learn new concepts without needing to be programmed explicitly in any conventional sense. In this paper we regard learning as the phenomenon of knowledge acquisition in the absence of explicit programming. We give a precise methodology for studying this phenomenon from a computational viewpoint. It consists of choosing an appropriate information gathering mechanism, the learning protocol, and exploring the class of concepts that can be learnt using it in a reasonable (polynomial) number of steps. We find that inherent algorithmic complexity appears to set serious limits to the range of concepts that can be so learnt. The methodology and results suggest concrete principles for designing realistic learning systems.