Timeline of machine learning

This page is a timeline of machine learning. Major discoveries, achievements, milestones and other major events in machine learning are included.

Overview

Decade	Summary
pre- 1950	Statistical methods are discovered and refined.
1950s	Pioneering machine learning research is conducted using simple algorithms.
1960s	Bayesian methods are introduced for probabilistic inference in machine learning.[1]
1970s	'AI winter' caused by pessimism about machine learning effectiveness.
1980s	Rediscovery of backpropagation causes a resurgence in machine learning research.
1990s	Work on Machine learning shifts from a knowledge-driven approach to a data-driven approach. Scientists begin creating programs for computers to analyze large amounts of data and draw conclusions – or "learn" – from the results.[2] Support-vector machines (SVMs) and recurrent neural networks (RNNs) become popular.[3] The fields of computational complexity via neural networks and super-Turing computation started.[4]
2000s	Support-Vector Clustering[5] and other kernel methods[6] and unsupervised machine learning methods become widespread.[7]
2010s	Deep learning becomes feasible, which leads to machine learning becoming integral to many widely used software services and applications. Deep learning spurs huge advances in vision and text processing.
2020s	Generative AI leads to revolutionary models, creating a proliferation of foundation models both proprietary and open source, notably enabling products such as ChatGPT (text-based) and Stable Diffusion (image based). Machine learning and AI enter the wider public consciousness. The commercial potential of AI based on machine learning causes large increases in valuations of companies linked to AI.

Timeline

Year	Event type	Caption	Event
1763	Discovery	The Underpinnings of Bayes' Theorem	Thomas Bayes's work An Essay Towards Solving a Problem in the Doctrine of Chances is published two years after his death, having been amended and edited by a friend of Bayes, Richard Price.[8] The essay presents work which underpins Bayes' theorem.
1805	Discovery	Least Square	Adrien-Marie Legendre describes the "méthode des moindres carrés", known in English as the least squares method.[9] The least squares method is used widely in data fitting.
1812		Bayes' Theorem	Pierre-Simon Laplace publishes Théorie Analytique des Probabilités, in which he expands upon the work of Bayes and defines what is now known as Bayes' Theorem.[10]
1843	Visionary	Visionary Pioneer	Ada Lovelace Lovelace's most significant relationship was with Charles Babbage, the inventor of the Analytical Engine, which is considered the first conceptual blueprint for a modern computer. [11] Lovelace's vision extended beyond Babbage's own understanding of his machine. She saw the Analytical Engine as more than a calculator; she believed it could process and manipulate any form of symbolic data, such as music or text. This early vision of machines processing more than just numbers laid the groundwork for the development of symbolic AI and machine learning. [12] Her contributions included what is now considered the first algorithm designed to be executed by a machine, making her the world's first computer programmer. Lovelace's understanding of the computational potential of machines continues to influence modern technologies like artificial intelligence. [13]
1913	Discovery	Markov Chains	Andrey Markov first describes techniques he used to analyse a poem. The techniques later become known as Markov chains.[14]
1943	Discovery	Artificial Neuron	Warren McCulloch and Walter Pitts develop a mathematical model that imitates the functioning of a biological neuron, the artificial neuron which is considered to be the first neural model invented.[15]
1950		Turing's Learning Machine	Alan Turing proposes a 'learning machine' that could learn and become artificially intelligent. Turing's specific proposal foreshadows genetic algorithms.[16]
1951		First Neural Network Machine	Marvin Minsky and Dean Edmonds build the first neural network machine, able to learn, the SNARC.[17]
1952		Machines Playing Checkers	Arthur Samuel joins IBM's Poughkeepsie Laboratory and begins working on some of the first machine learning programs, first creating programs that play checkers.[18]
1957	Discovery	Perceptron	Frank Rosenblatt invents the perceptron while working at the Cornell Aeronautical Laboratory.[19] The invention of the perceptron generated a great deal of excitement and was widely covered in the media.[20]
1963	Achievement	Machines Playing Tic-Tac-Toe	Donald Michie creates a 'machine' consisting of 304 match boxes and beads, which uses reinforcement learning to play Tic-tac-toe (also known as noughts and crosses).[21]
1967		Nearest Neighbor	The nearest neighbour algorithm was created, which is the start of basic pattern recognition. The algorithm was used to map routes.[2]
1969		Limitations of Neural Networks	Marvin Minsky and Seymour Papert publish their book Perceptrons, describing some of the limitations of perceptrons and neural networks. The interpretation that the book shows that neural networks are fundamentally limited is seen as a hindrance for research into neural networks.[22]
1970		Automatic Differentiation (Backpropagation)	Seppo Linnainmaa publishes the general method for automatic differentiation (AD) of discrete connected networks of nested differentiable functions.[23][24] This corresponds to the modern version of backpropagation, but is not yet named as such.[25][26][27][28]
1976	Discovery	Transfer Learning	Stevo Bozinovski and Ante Fulgosi introduced transfer learning method in neural networks training. [29] [30]
1979		Stanford Cart	Students at Stanford University develop a cart that can navigate and avoid obstacles in a room.[2]
1979	Discovery	Neocognitron	Kunihiko Fukushima first publishes his work on the neocognitron, a type of artificial neural network (ANN).[31][32] Neocognition later inspires convolutional neural networks (CNNs).[33]
1981	Achievement	Learning to recognize 40 patterns	Stevo Bozinovski showed an experiment of neural network supervised learning for recognition of 40 linearly dependent patters: 26 letters, 10 numbers, and 4 special symbols from a computer terminal. [34]
1981		Explanation Based Learning	Gerald Dejong introduces Explanation Based Learning, where a computer algorithm analyses data and creates a general rule it can follow and discard unimportant data.[2]
1982	Discovery	Recurrent Neural Network	John Hopfield popularizes Hopfield networks, a type of recurrent neural network that can serve as content-addressable memory systems.[35]
1982	Discovery	Self Learning	Stevo Bozinovski develops a self-learning paradigm in which an agent does not use an external reinforcement. Instead, the agent learns using internal state evaluations, represented by emotions. He introduces the Crossbar Adaptive Array (CAA) architecture capable of self-learning. [36][37]
1982	Achievement	Delayed reinforcement learning	Stevo Bozinovski solved the challenge of reinforcement learning with delayed rewards. Using the Crossbar Adaptive Array (CAA) he presented solutions of two tasks: 1) learning path in a graph 2) balancing an inverted pendulum.[38]
1985		NETtalk	A program that learns to pronounce words the same way a baby does, is developed by Terry Sejnowski.[2]
1986	Application	Backpropagation	Seppo Linnainmaa's reverse mode of automatic differentiation (first applied to neural networks by Paul Werbos) is used in experiments by David Rumelhart, Geoff Hinton and Ronald J. Williams to learn internal representations.[39]
1988	Discovery	Universal approximation theorem	Kurt Hornik [de] proves that standard multilayer feedforward networks are capable of approximating any Borel measurable function from one finite dimensional space to another to any desired degree of accuracy, provided sufficiently many hidden units are available.
1989	Discovery	Reinforcement Learning	Christopher Watkins develops Q-learning, which greatly improves the practicality and feasibility of reinforcement learning.[40]
1989	Commercialization	Commercialization of Machine Learning on Personal Computers	Axcelis, Inc. releases Evolver, the first software package to commercialize the use of genetic algorithms on personal computers.[41]
1992	Achievement	Machines Playing Backgammon	Gerald Tesauro develops TD-Gammon, a computer backgammon program that uses an artificial neural network trained using temporal-difference learning (hence the 'TD' in the name). TD-Gammon is able to rival, but not consistently surpass, the abilities of top human backgammon players.[42]
1995	Discovery	Random Forest Algorithm	Tin Kam Ho publishes a paper describing random decision forests.[43]
1995	Discovery	Support-Vector Machines	Corinna Cortes and Vladimir Vapnik publish their work on support-vector machines.[44]
1997	Achievement	IBM Deep Blue Beats Kasparov	IBM's Deep Blue beats the world champion at chess.[2]
1997	Discovery	LSTM	Sepp Hochreiter and Jürgen Schmidhuber invent long short-term memory (LSTM) recurrent neural networks,[45] greatly improving the efficiency and practicality of recurrent neural networks.
1998		MNIST database	A team led by Yann LeCun releases the MNIST database, a dataset comprising a mix of handwritten digits from American Census Bureau employees and American high school students.[46] The MNIST database has since become a benchmark for evaluating handwriting recognition.
2002	Project	Torch Machine Learning Library	Torch, a software library for machine learning, is first released.[47]
2006		The Netflix Prize	The Netflix Prize competition is launched by Netflix. The aim of the competition was to use machine learning to beat Netflix's own recommendation software's accuracy in predicting a user's rating for a film given their ratings for previous films by at least 10%.[48] The prize was won in 2009.
2009	Achievement	ImageNet	ImageNet is created. ImageNet is a large visual database envisioned by Fei-Fei Li from Stanford University, who realized that the best machine learning algorithms wouldn't work well if the data didn't reflect the real world.[49] For many, ImageNet was the catalyst for the AI boom[50] of the 21st century.
2010	Project	Kaggle Competition	Kaggle, a website that serves as a platform for machine learning competitions, is launched.[51]
2011	Achievement	Beating Humans in Jeopardy	Using a combination of machine learning, natural language processing and information retrieval techniques, IBM's Watson beats two human champions in a Jeopardy! competition.[52]
2012	Achievement	Recognizing Cats on YouTube	The Google Brain team, led by Andrew Ng and Jeff Dean, create a neural network that learns to recognize cats by watching unlabeled images taken from frames of YouTube videos.[53][54]
2012	Discovery	Visual Recognition	The AlexNet paper and algorithm achieves breakthrough results in image recognition in the ImageNet benchmark. This popularizes deep neural networks.[55]
2013	Discovery	Word Embeddings	A widely cited paper nicknamed word2vec revolutionizes the processing of text in machine learnings. It shows how each word can be converted into a sequence of numbers (word embeddings), the use of these vectors revolutionized text processing in machine learning.
2014	Achievement	Leap in Face Recognition	Facebook researchers publish their work on DeepFace, a system that uses neural networks that identifies faces with 97.35% accuracy. The results are an improvement of more than 27% over previous systems and rivals human performance.[56]
2014		Sibyl	Researchers from Google detail their work on Sibyl,[57] a proprietary platform for massively parallel machine learning used internally by Google to make predictions about user behavior and provide recommendations.[58]
2016	Achievement	Beating Humans in Go	Google's AlphaGo program becomes the first Computer Go program to beat an unhandicapped professional human player[59] using a combination of machine learning and tree search techniques.[60] Later improved as AlphaGo Zero and then in 2017 generalized to Chess and more two-player games with AlphaZero.
2017	Discovery	Transformer	A team at Google Brain invent the transformer architecture,[61] which allows for faster parallel training of neural networks on sequential data like text.
2018	Achievement	Protein Structure Prediction	AlphaFold 1 (2018) placed first in the overall rankings of the 13th Critical Assessment of Techniques for Protein Structure Prediction (CASP) in December 2018.[62]
2021	Achievement	Protein Structure Prediction	AlphaFold 2 (2021), A team that used AlphaFold 2 (2020) repeated the placement in the CASP competition in November 2020. The team achieved a level of accuracy much higher than any other group. It scored above 90 for around two-thirds of the proteins in CASP's global distance test (GDT), a test that measures the degree to which a computational program predicted structure is similar to the lab experiment determined structure, with 100 being a complete match, within the distance cutoff used for calculating GDT.[63]

See also

• History of artificial intelligence
• Timeline of artificial intelligence
• Timeline of machine translation