Top Qs
Timeline
Chat
Perspective
Neural differential equation
Equation in machine learning From Wikipedia, the free encyclopedia
Remove ads
Neural differential equations are a class of models in machine learning that combine neural networks with the mathematical framework of differential equations.[1] These models provide an alternative approach to neural network design, particularly for systems that evolve over time or through continuous transformations.
The most common type, a neural ordinary differential equation (neural ODE), defines the evolution of a system's state using an ordinary differential equation whose dynamics are governed by a neural network: In this formulation, the neural network parameters θ determine how the state changes at each point in time.[1] This approach contrasts with conventional neural networks, where information flows through discrete layers indexed by natural numbers. Neural ODEs instead use continuous layers indexed by positive real numbers, where the function represents the network's state at any given layer depth t.
Neural ODEs can be understood as continuous-time control systems, where their ability to interpolate data can be interpreted in terms of controllability.[2] They have found applications in time series analysis, generative modeling, and the study of complex dynamical systems.
Remove ads
Connection with residual neural networks
Summarize
Perspective
Neural ODEs can be interpreted as a residual neural network with a continuum of layers rather than a discrete number of layers.[3] Applying the Euler method with a unit time step to a neural ODE yields the forward propagation equation of a residual neural network:
with ℓ being the ℓ-th layer of this residual neural network. While the forward propagation of a residual neural network is done by applying a sequence of transformations starting at the input layer, the forward propagation computation of a neural ODE is done by solving a differential equation. More precisely, the output associated to the input of the neural ODE is obtained by solving the initial value problem
and assigning the value to .
Remove ads
Universal differential equations
Summarize
Perspective
In physics-informed contexts where additional information is known, neural ODEs can be combined with an existing first-principles model to build a physics-informed neural network model called universal differential equations (UDE).[4][5][6][7] For instance, an UDE version of the Lotka-Volterra model can be written as[8]
where the terms and are correction terms parametrized by neural networks.
Remove ads
References
See also
External links
Wikiwand - on
Seamless Wikipedia browsing. On steroids.
Remove ads