Ant Colony Optimization Incorporated With Fuzzy Q-Learning for Reinforcement Fuzzy Control

Abstract

This paper proposes the design of fuzzy controllers by ant colony optimization (ACO) incorporated with fuzzy-Q learning, called ACO-FQ, with reinforcements. For a fuzzy inference system, we partition the antecedent part <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a</i> <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">priori</i> and then list all candidate consequent actions of the rules. In ACO-FQ, the tour of an ant is regarded as a combination of consequent actions selected from every rule. Searching for the best one among all combinations is partially based on pheromone trail. We assign to each candidate in the consequent part of the rule a corresponding <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Q</i> -value. Update of the <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Q</i> -value is based on fuzzy-Q learning. The best combination of consequent values of a fuzzy inference system is searched according to pheromone levels and <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Q</i> -values. ACO-FQ is applied to three reinforcement fuzzy control problems: (1) water bath temperature control; (2) magnetic levitation control; and (3) truck backup control. Comparisons with other reinforcement fuzzy system design methods verify the performance of ACO-FQ.