Classifier Fitness Based on Accuracy

TLDR

Classifier strength in many systems predicts future payoff and serves as fitness for the genetic algorithm. The study examines XCS, where classifier fitness is based on prediction accuracy, and demonstrates its suitability for reinforcement learning that requires state generalization. XCS runs the genetic algorithm within match‑set niches and evaluates fitness by prediction accuracy. XCS produces a complete, accurate mapping from inputs and actions to payoff predictions, evolves maximally general classifiers under an accuracy criterion, and is well suited for reinforcement learning requiring state generalization.

Abstract

In many classifier systems, the classifier strength parameter serves as a predictor of future payoff and as the classifier's fitness for the genetic algorithm. We investigate a classifier system, XCS, in which each classifier maintains a prediction of expected payoff, but the classifier's fitness is given by a measure of the prediction's accuracy. The system executes the genetic algorithm in niches defined by the match sets, instead of panmictically. These aspects of XCS result in its population tending to form a complete and accurate mapping X × A → P from inputs and actions to payoff predictions. Further, XCS tends to evolve classifiers that are maximally general, subject to an accuracy criterion. Besides introducing a new direction for classifier system research, these properties of XCS make it suitable for a wide range of reinforcement learning situations where generalization over states is desirable.