Computer Methods in Applied Mechanics and Engineering · 2024 · 15 citations · 43 references
Natural SciencesComputer SciencePhysicsComputational Fluid DynamicsFluid DynamicsMachine LearningDeep LearningMultiscale MechanicsNavier-stokes EquationsNeural ModelingNeural NetworksPartial Differential EquationsConvolutional Neural NetworksNavier-stokes SolversNonlinear Partial Differential EquationsTime-stepping SchemesTransformer Models
Developing fast surrogates for Partial Differential Equations (PDEs) will accelerate design and optimization in almost all scientific and engineering applications. Neural networks have been receiving ever-increasing attention and demonstrated remarkable success in computational modeling of PDEs, however; their prediction accuracy is not at the level of full deployment. In this work, we utilize the transformer architecture, the backbone of numerous state-of-the-art AI models, to learn the dynamics of physical systems as the mixing of spatial patterns learned by a convolutional autoencoder. Moreover, we incorporate the idea of multi-scale hierarchical time-stepping to increase the prediction speed and decrease accumulated error over time. Our model achieves similar or better results in predicting the time-evolution of Navier–Stokes equations compared to the powerful Fourier Neural Operator (FNO) and two transformer-based neural operators OFormer and Galerkin Transformer. The code and data are available on https://github.com/BaratiLab/MST_PDE.
43
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · 2023 · 76.3K citations · Full text
Maziar Raissi, Paris Perdikaris, George Em Karniadakis · Journal of Computational Physics · 2018 · 16.9K citations · Full text