Explainable reinforcement learning for powertrain control engineering

Abstract

In this paper we demonstrate a practical post-hoc approach for explainable reinforcement learning (RL) in vehicle powertrain control. The goal is to exploit the advantages of RL yet obtain a solution that is feasible to implement in safety-critical control engineering problems. This means finding a solution that balances optimal product design with the required engineering effort, while maintaining the transparency necessary for safety-critical applications. Our method is based on initially training a neural network based RL policy and converting it into a look-up table, using a decision tree (DT) as an intermediary. The DT is limited to a certain depth, resulting in a look-up table of manageable size that can be directly tested, implemented and evaluated by control engineers. In order to evaluate this approach, a set of RL expert policies were used to train DTs with increasing depth, showing the regions where the DT solution can outperform benchmarks while still remaining small enough to translate to a manageable look-up table. Our approach involves standard Python libraries, lowering the barrier for implementation. This approach is not just relevant to powertrain control, but offers a practical approach for all regulated domains which could benefit from application of RL.