1970 ⟶ Linnainmaa Publishes Reverse Mode Automatic Differentiation

Seppo Linnainmaa publishes the reverse mode of automatic dif...
Timelines Logo
Year
1676
1970
2015

📐 Leibniz's Derivation of the Chain Rule

Leibniz derived the Chain rule. The rule is used by AI to train neural networks, for example the Backpropagation algorithm uses the Chain rule.⟶
Leibniz's Derivation of the Chain Rule (1676)
CalculusMathematicsChain RuleNeural NetworksBackpropagationMachine LearningOptimization
GermanyGermany

🧮 Linnainmaa Publishes Reverse Mode Automatic Differentiation

Seppo Linnainmaa publishes the reverse mode of Automatic differentiation. This method became later known as Backpropagation, and is heavily used to train Artificial neural networks.⟶
Linnainmaa Publishes Reverse Mode Automatic Differentiation (1970)
BackpropagationAutomatic DifferentiationNeural NetworksMachine LearningDeep LearningOptimizationTraining AlgorithmsSeppo Linnainmaa
FinlandFinland

➕ Highway and Residual Networks Developed for Deep Learning

Two techniques were developed concurrently to train very deep networks: Highway network, and the Residual neural network (ResNet). They allowed over 1000-layers-deep networks to be trained.⟶
Highway and Residual Networks Developed for Deep Learning (2015)
Deep LearningNeural NetworksHighway NetworksResidual NetworksResNetTraining Techniques