Backpropagation
Optimization algorithm for artificial neural networks / From Wikipedia, the free encyclopedia
Dear Wikiwand AI, let's keep it short by simply answering these key questions:
Can you list the top facts and stats about Backpropagation?
Summarize this article for a 10 year old
In machine learning, backpropagation is a gradient estimation method used to train neural network models. The gradient estimate is used by the optimization algorithm to compute the network parameter updates.
![]() | This article may be in need of reorganization to comply with Wikipedia's layout guidelines. The reason given is: Inconsistent use of variable names and terminology without images to match. (August 2022) |
It is an efficient application of the chain rule to neural networks.[1] It is also known as the reverse mode of automatic differentiation or reverse accumulation, due to Seppo Linnainmaa (1970).[2][3][4][5][6][7][8] The term "back-propagating error correction" was introduced in 1962 by Frank Rosenblatt,[9][1] but he did not know how to implement this, even though Henry J. Kelley had a continuous precursor of backpropagation[10] already in 1960 in the context of control theory.[1]
Backpropagation computes the gradient of a loss function with respect to the weights of the network for a single input–output example, and does so efficiently, computing the gradient one layer at a time, iterating backward from the last layer to avoid redundant calculations of intermediate terms in the chain rule; this can be derived through dynamic programming.[10][11][12] Gradient descent, or variants such as stochastic gradient descent,[13] are commonly used.
Strictly the term backpropagation refers only to the algorithm for computing the gradient, not how the gradient is used; but the term is often used loosely to refer to the entire learning algorithm – including how the gradient is used, such as by stochastic gradient descent.[14] In 1986 David E. Rumelhart et al. published an experimental analysis of the technique.[15] This contributed to the popularization of backpropagation and helped to initiate an active period of research in multilayer perceptrons.