On the use of Fourier Features-Physics Informed Neural Networks (FF-PINN) for forward and inverse fluid mechanics problems

Abstract

Physics Informed Neural Networks (PINN), a deep learning tool, has recently become an effective method for solving inverse Partial Differential Equations (PDEs) where the boundary/initial conditions are not well defined and only noisy sparse measurements sampled in the domain exist. PINN, and other Neural Networks, tends to converge to the low frequency solution in a field that has multiple frequency scales, this is known as spectral bias. For PINN this happens when solving PDEs that exhibit periodic behavior spatially and temporally with multi frequency scales. Previous studies suggested that Fourier Features-Neural Networks (FF-NN) can be used to overcome the spectral bias problem. They proposed the Multi Scale-Spatio Temporal-Fourier Features-Physics Informed Neural Networks (MS-ST-FF-PINN) to overcome the spectral bias problem in PDEs solved by PINN. This has been evaluated on basic PDEs such as Poisson, wave and Gray-Scott equations. In this paper we take MS-ST-FF-PINN a step further by applying it to the incompressible Navier-Stokes equations. Furthermore, a comparative analysis between the PINN and the MS-ST-FF-PINN architectures solution accuracy, the learnt frequency components and the rate of convergence to the correct solution is included. To show this three test cases are shown (a)-Forward time independent double-lid-driven cavity, (b)-Inverse time independent free surface estimation of Kelvin wave pattern, and (c)-Inverse 2D time-dependent turbulent Von Karman vortex shedding interaction downstream of multiple cylinders. The results show that MS-ST-FF-PINN is better at learning low and high frequency components synchronously at early training iterations compared to the PINN architecture that does not learn the high frequency components even after multiple iteration numbers such as the Kelvin wave pattern and the Karman vortex shedding cases. However, for the third test case, the MS-ST-FF-PINN architecture showed a discontinuity for the temporal prediction of the pressure field due to over-fitting.

Keywords

PINN Fourier Features Neural Network incompressible flow spectral bias deep learning FEM CFD free surface OpenFOAM Volume of Fluid

Get full access to this article

View all access options for this article.

References

Raissi

Perdikaris

Karniadakis

. Physics informed deep learning (part i): Data-driven solutions of nonlinear partial differential equations. arXiv preprint arXiv:171110561. 2017.

Raissi

. Deep hidden physics models: deep learning of nonlinear partial differential equations. J Mach Learn Res 2018; 19(1): 932–955.

Raissi

Yazdani

Karniadakis

. Hidden fluid mechanics: learning velocity and pressure fields from flow visualizations. Science 2020; 367(6481): 1026–1030.

Rahaman

Baratin

Arpit

, et al. On the spectral bias of neural networks. PMLR 2019; 97: 5301–5310.

Tancik

Srinivasan

Mildenhall

, et al. Fourier features let networks learn high frequency functions in low dimensional domains. Adv Neural Inf Process Syst 2020; 33: 7537–7547.

Williamson

Govardhan

, et al. Vortex-induced vibrations. Annu Rev Fluid Mech 2004; 36(1): 413–455.

Hamada

Furth

. Ground effect on current energy harvesting from a freely-oscillating circular cylinder at low Reynolds number. In: International conference on offshore mechanics and arctic engineering, 2021, vol. 85192, p.V009T09A003. New York: American Society of Mechanical Engineers.

Lin

. Regular and irregular wave impacts on floating body. Ocean Eng 2012; 42: 93–101.

Wang

Perdikaris

. On the eigenvector bias of Fourier feature networks: from regression to solving multi-scale PDEs with physics-informed neural networks. Comput Methods Appl Mech Eng 2021; 384: 113938.

10.

Meng

Mao

, et al. DeepXDE: A deep learning library for solving differential equations. SIAM Rev 2021; 63(1): 208–228.

11.

Raissi

Perdikaris

Karniadakis

. Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J Comput Phys 2019; 378: 686–707.

12.

Baydin

Pearlmutter

Radul

, et al. Automatic differentiation in machine learning: a survey. J Mach Learn Res 2018; 18: 1–43.

13.

Abadi

Agarwal

Barham

, et al. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:160304467, 2016.

14.

Paszke

Gross

Massa

, et al. Pytorch: an imperative style, high-performance deep learning library. Adv Neural Inf Process Syst 2019; 32.

15.

Bishop

. Neural networks and their applications. Rev Sci Instrum 1994; 65(6): 1803–1832.

16.

Nwankpa

Ijomah

Gachagan

, et al. Activation functions: Comparison of trends in practice and research for deep learning. arXiv preprint arXiv:181103378, 2018.

17.

Glorot

Bengio

. Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the thirteenth international conference on artificial intelligence and statistics. JMLR Workshop and Conference Proceedings, 2010, pp.249–256.

18.

Kingma

. Adam: A method for stochastic optimization. arXiv preprint, arXiv:14126980. 2014.

19.

Schöberl

. C++11 implementation of finite elements in NGSolve. Institute for analysis and scientific computing. Vienna: Vienna University of Technology, 2014. pp.30.

20.

Brezzi

Falk

. Stability of higher-order Hood–Taylor Methods. SIAM J Numer Anal 1991; 28(3): 581–590.

21.

Sallam

Feng

Stason

, et al. Computer vision techniques for floating structures experimental analysis: a heaving buoy case study. 2022. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4140113

22.

Walsh

Hancock

Hines

, et al. Wave-measurement capabilities of the surface contour radar and the airborne oceanographic lidar. Johns Hopkins APL Tech Dig 1987; 8(1): 74–81.

23.

Salmon

. Introduction to ocean waves. San Diego, CA: Scripps Institution of Oceanography, University of California, San Diego, 2008.

24.

Kim

Van

Kim

. Measurement of flows around modern commercial ship models. Exp Fluids 2001; 31(5): 567–578.

25.

Shi

Yuan

. Investigation of the ship–seabed interaction with a high-fidelity CFD approach. J Mar Sci Technol 2021; 26(3): 931–946.

26.

Deshpande

Anumolu

Trujillo

. Evaluating the performance of the two-phase flow solver interFoam. Comput Sci Discov 2012; 5(1): 014016.

27.

Jasak

Jemcov

Tukovic

, et al. OpenFOAM: A C++ library for complex physics simulations. In: International workshop on coupled methods in numerical dynamics, vol. 1000, 2007, pp.1–20. IUC Dubrovnik Croatia.

28.

Holzmann

. Mathematics, numerics, derivations and Openfoam®. Loeben: Holzmann CFD, 2016.

29.

Gisen

. Generation of a 3D mesh using snappyHexMesh featuring anisotropic refinement and near-wall layers. In: ICHE 2014. Proceedings of the 11th international conference on hydroscience & engineering, 2014, pp.983–990.

30.

Leiteritz

Pfluger

. How to avoid trivial solutions in physics-informed neural networks. arXiv preprint, arXiv:211205620, 2021.

31.

Piomelli

. Large-eddy simulation: achievements and challenges. Prog Aerosp Sci 1999; 35(4): 335–362.

32.

Liu

Nocedal

. On the limited memory BFGS method for large scale optimization. Math Program 1989; 45(1–3): 503–528.

33.

Zhang

Chen

, et al. Efficient mini-batch training for stochastic optimization. In: Proceedings of the 20th ACM SIGKDD international conference on knowledge discovery and data mining, 2014, pp.661–670.

34.

Kandel

Castelli

. The effect of batch size on the generalizability of the convolutional neural networks on a histopathology dataset. ICT Express 2020; 6(4): 312–315.