Physics of Fluids · 2019 · 209 citations · 44 references
ComputingComputational EngineeringComputer ScienceComputational Fluid DynamicsFluid DynamicsFluid MechanicsDeep LearningParallel ComputationParallel Numerical MethodsReinforcement LearningActive ControlActive Flow ControlCfd SimulationDeep Learning AccelerationDeep Reinforcement LearningNumerical Flow SimulationNumerical SimulationReynolds Number
Deep Reinforcement Learning (DRL) has recently been proposed as a methodology to discover complex active flow control strategies [Rabault et al., “Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control,” J. Fluid Mech. 865, 281–302 (2019)]. However, while promising results were obtained on a simple 2-dimensional benchmark flow at a moderate Reynolds number, considerable speedups will be required to investigate more challenging flow configurations. In the case of DRL trained with Computational Fluid Dynamics (CFD) data, it was found that the CFD part, rather than training the artificial neural network, was the limiting factor for speed of execution. Therefore, speedups should be obtained through a combination of two approaches. The first one, which is well documented in the literature, is to parallelize the numerical simulation itself. The second one is to adapt the DRL algorithm for parallelization. Here, a simple strategy is to use several independent simulations running in parallel to collect experiences faster. In the present work, we discuss this solution for parallelization. We illustrate that perfect speedups can be obtained up to the batch size of the DRL agent, and slightly suboptimal scaling still takes place for an even larger number of simulations. This is, therefore, an important step toward enabling the study of more sophisticated fluid mechanics problems through DRL.
44
Mastering the game of Go without human knowledge
David Silver, Julian Schrittwieser, Karen Simonyan et al. · Nature · 2017 · 9.2K citations
Engineering, Educational Psychology, Cognitive Psychology +7
Speech recognition with deep recurrent neural networks
Alex Graves, Abdelrahman Mohamed, Geoffrey E. Hinton · 2013 · 8.8K citations
TensorFlow: a system for large-scale machine learning
Martı́n Abadi, Paul Barham, Jianmin Chen et al. · Operating Systems Design and Implementation · 2016 · 6.4K citations
Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates
Shixiang Gu, Ethan Holly, Timothy Lillicrap et al. · 2017 · 1.5K citations