Top Qs
Timeline
Chat
Perspective

Neural differential equation

Equation in machine learning From Wikipedia, the free encyclopedia

Remove ads

Neural differential equations are a class of models in machine learning that combine neural networks with the mathematical framework of differential equations.[1] These models provide an alternative approach to neural network design, particularly for systems that evolve over time or through continuous transformations.

The most common type, a neural ordinary differential equation (neural ODE), defines the evolution of a system's state using an ordinary differential equation whose dynamics are governed by a neural network: In this formulation, the neural network parameters θ determine how the state changes at each point in time.[1] This approach contrasts with conventional neural networks, where information flows through discrete layers indexed by natural numbers. Neural ODEs instead use continuous layers indexed by positive real numbers, where the function represents the network's state at any given layer depth t.

Neural ODEs can be understood as continuous-time control systems, where their ability to interpolate data can be interpreted in terms of controllability.[2] They have found applications in time series analysis, generative modeling, and the study of complex dynamical systems.

Remove ads

Connection with residual neural networks

Summarize
Perspective

Neural ODEs can be interpreted as a residual neural network with a continuum of layers rather than a discrete number of layers.[3] Applying the Euler method with a unit time step to a neural ODE yields the forward propagation equation of a residual neural network:

with ℓ being the ℓ-th layer of this residual neural network. While the forward propagation of a residual neural network is done by applying a sequence of transformations starting at the input layer, the forward propagation computation of a neural ODE is done by solving a differential equation. More precisely, the output associated to the input of the neural ODE is obtained by solving the initial value problem

and assigning the value to .

Remove ads

Universal differential equations

Summarize
Perspective

In physics-informed contexts where additional information is known, neural ODEs can be combined with an existing first-principles model to build a physics-informed neural network model called universal differential equations (UDE).[4][5][6][7] For instance, an UDE version of the Lotka-Volterra model can be written as[8]

where the terms and are correction terms parametrized by neural networks.

Remove ads

References

See also

Loading related searches...

Wikiwand - on

Seamless Wikipedia browsing. On steroids.

Remove ads