PINN (Physics-Informed Neural Networks)

This project attempts to solve Duffing equation using PINN and numerical methods.

Project goals

• Implement PINN method for solving Cauchy problem of Duffing equation
• Compare results with numerical methods
• Research relation between behaviour of system and its parameters

Task formulation

Duffing equation
$\frac{d^2x}{dt^2}+\delta \frac{dx}{dt} + \alpha x + \beta x^3 = \gamma \cos(\omega t), \quad t>0$

Cauchy data of the problem - $x(0) = x_0, \frac{dx}{dt}(0) = v_0$

Parameters of equation - $\alpha, \beta, \gamma, \delta, \omega$

Numerical methods

3 numerical methods were implemented:

• Euler method (first-order numerical procedure)
• Trapezoidal method (second-order numerical procedure)
• Runge-Kutta method (fourth-order numerical procedure)

Numerical methods were tested on the following differential equation with analytic solution - $\ddot x + \alpha x = 0 \left(x_0 = 1, v_0 = 0, t_0 = 0\right)$

Log-log plot of numerical methods errors

PINN

For approximating equations whose solution is unknown, conventional types of neural networks are not applicable - it is impossible to calculate the error of prediction.

PINN is a type of neural network for approximating differential equations that has a special error function - it is calculated by substituting the predicted values into the original differential equation, taking into account Cauchy data of the equation.

$LOSS = \frac{1}{N} \sum_{i=1}^{N} (x''_i + \delta x'_i + \alpha x + \beta x_i^3 - \gamma \cos \left( \omega t_i \right))) + \lambda \left(\left(x(0) - f(0)\right)^2 + \left(x'(0) - f'(0)\right)^2 \right)$

Neural network configurations

There were 3 configurations of neural network that were tested during research.

Training of models for tests was performed for 256 epochs.

Configuration №1

| | Layer 1 | Layer 2 | Layer 3 | Layer 4 | Layer 5 | Layer 6 | |:--------------------|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:| | Neurons | 32 | 32 | 32 | 32 | 32 | 1 | | Activation function | sin | sin | tanh | tanh | sin | id |

| Max error | MSE | |:---------:|:------:| | 0.068 | 1.8e-7 |

Configuration №2

| | Layer 1 | Layer 2 | Layer 3 | Layer 4 | Layer 5 | Layer 6 | |:--------------------|:-------:|:-------:|:----------:|:-------:|:-------:|:-------:| | Neurons | 32 | 32 | 32 | 32 | 32 | 1 | | Activation function | sin | tanh | softplus | tanh | sin | id |

| Max error | MSE | |:---------:|:-------:| | 0.059 | 1.39e-7 |

Configuration №3

| | Layer 1 | Layer 2 | Layer 3 | Layer 4 | Layer 5 | Layer 6 | |:--------------------|:-------:|:-------:|:----------:|:-------:|:-------:|:-------:| | Neurons | 64 | 64 | 64 | 64 | 64 | 1 | | Activation function | sin | tanh | softplus | tanh | sin | id |

| Max error | MSE | |:---------:|:-------:| | 0.054 | 1.18e-7 |

Research of parameters

By setting some parameters of the Duffing equation to zero, it is transformed into the equations of already known oscillatory systems.

Thus, it is possible to identify influence of the equation parameters on the behavior of the oscillator using said method.

$\alpha$ parameter

For given parameters $\beta = \delta = \gamma = \omega = 0$, the Duffing equation describes simple harmonic oscillations - $\ddot x + \alpha x = 0$.

We can conclude that $\alpha$ defines linear stiffness of oscillator.

Plots and phase portraits

| $\alpha = 1$ plot | $\alpha = 1$ phase portrait | |:-----------------------------------------------------:|:-------------------------------------------------------------------------:| | | |

| $\alpha = 0$ plot | |:-----------------------------------------------------:| | |

| $\alpha = -3$ plot | $\alpha = -3$ phase portrait | |:-------------------------------------------------------:|:---------------------------------------------------------------------------:| | | |

$\delta$ parameter

For given parameter $\beta = 0$, the Duffing equation describes damped oscillations - $\ddot x + \delta \dot x + \alpha x = 0$.

We can conclude that $\delta$ represents degree of oscillation damping.

Plots

| $\delta = 0.5$ plot | |:---------------------------------------------------------:| | |

| $\delta = 0.03$ plot | |:-----------------------------------------------------------:| | |

$\beta$ parameter

For given parameter $\gamma = 0$, the Duffing equation describes non-linear dynamic oscillation system - $\ddot x + \delta \dot x + \alpha x + \beta x^3 = 0$.

We can conclude that $\beta$ represents degree of non-linearity of driving force.

Plots and phase portraits

After some research, it was discovered that if $\beta$ parameter is non-null, PINN has serious problems with approximating Duffing equation.

This is due to the fact that this coefficient has a decisive influence on the possibility of the system transitioning to a state of dynamic chaos, and also affects the non-linear properties of the system.

| $\beta \neq 0$ plot | $\beta \neq 0$ phase portrait | |:------------------------------------------------------------:|:--------------------------------------------------------------------------------:| | | |

$\beta$ parameters also impose problems on numerical methods, but since we can specify arbitrary precision and choose different methods decent approximation is still possible.

Stability of solutions

Although $\beta$ parameter severely complicates equation, we can still research its behaviour using knowledge from Stability theory.

Hooke's law for linear oscillation system - $F = - \alpha x$. Transformed version of that law - $F = - \alpha x - \beta x^3 = -x \left(\alpha - \beta x^2 \right)$.

Now it is possible to identify equilibrium points - points at which $F = 0$.

Zero is always an equilibrium point; but if $\alpha \lt 0$ or $\beta \lt 0$ with big enough x value, then zero is unstable point, otherwise it is asymptotically stable. If there is a real solution for the $\left(\alpha - \beta x^2 \right) = 0$ equation, then that solution is an asymptotically stable point.

| $\alpha \lt 0$, $\beta \gt 0$ phase portrait | |:---------------------------------------------------------------------------------------------------------------:| | |

| $\alpha \gt 0$, $\beta \gt 0$ phase portrait | |:------------------------------------------------------------------------------------------------------------------:| | |

$\gamma$ and $\omega$ parameters

For given parameter $\delta = 0$, the Duffing equation describes driven oscillation system - $\ddot x + \alpha x + \beta x^3 = \gamma \cos \left( \omega t \right)$.

We can conclude that $\gamma$ represents amplitude of periodic driving force and $\omega$ represents angular frequency of a periodic driving force.

Plots

| $\gamma = 1.5$, $\omega = 1.2$ plot | |:--------------------------------------------------------------------------------------:| | |

| $\gamma = 0.5$, $\omega = 0.5$ plot | |:--------------------------------------------------------------------------------------:| | |

General form of the equation

When all parameters are non-null ($\frac{d^2x}{dt^2}+\delta \frac{dx}{dt} + \alpha x + \beta x^3 = \gamma \cos(\omega t), \quad t>0$), we can research behaviour of Duffing equation by exami