Multi-criteria Reinforcement Learning

Abstract

"Fe consider multi-criteria sequential decision making problems,,,,here the vector-valued evaluations arc cOluparcd by it given, fixed total ordering. Conditions for the optimality of stationary policies and the BelllUan optimality eqnation arc given for a special, hut importrmt cla...,s of problems when the evaluation of policies can be computed for the criteria independently of each other. The i:utalysi:::; requirel:> special care as the Copolo)?;.v introduced b,y ' pointwise convergence and the order-Lopology introduced by the preference order are in genera.l incompa.tible. Reinforce IHcnt lcarning algorithms are proposed and analyzed. Prclilninar�y computer experiments confirm the validity of the derived a.lgorithms. These type of multi-criteria problems are most useflll when there are several optimal soluUons l.o a problem and one \\vants to choose the one among lhese \\vhich is optilnal according to another fixed criterion. Possible application in robotics ancl repeated games are outlined. 1