🤖 AI Summary
This paper addresses the fragmentation of AI historiography, the obscuration of neural networks’ intellectual origins, and the marginalization of cybernetics in mainstream narratives. It proposes a unified historical reconstruction framework centered on the concept of “credit assignment.” Through rigorous historical document analysis, interdisciplinary knowledge graph construction, and scholarly provenance tracing, the study systematically traces the mathematical and technical lineage—from the 17th-century chain rule and 19th-century linear regression to the first implementation of deep learning in 1965—thereby correcting widespread textbook misconceptions and reaffirming cybernetics’ foundational role in modern AI. The resulting contribution is the most comprehensive chronology of deep learning to date (as of 2022), documenting over one hundred pivotal events, rigorously attributing original contributions, and embedding hundreds of hyperlinked authoritative sources. This chronology has been published as a core chapter in an academic monograph on AI.
📝 Abstract
Machine learning (ML) is the science of credit assignment: finding patterns in observations that predict the consequences of actions and help to improve future performance. Credit assignment is also required for human understanding of how the world works, not only for individuals navigating daily life, but also for academic professionals like historians who interpret the present in light of past events. Here I focus on the history of modern artificial intelligence (AI) which is dominated by artificial neural networks (NNs) and deep learning, both conceptually closer to the old field of cybernetics than to what's been called AI since 1956 (e.g., expert systems and logic programming). A modern history of AI will emphasize breakthroughs outside of the focus of traditional AI text books, in particular, mathematical foundations of today's NNs such as the chain rule (1676), the first NNs (linear regression, circa 1800), and the first working deep learners (1965-). From the perspective of 2022, I provide a timeline of the—in hindsight—most important relevant events in the history of NNs, deep learning, AI, computer science, and mathematics in general, crediting those who laid foundations of the field. The text contains numerous hyperlinks to relevant overview sites from my AI Blog. It also debunks certain popular but misleading historic accounts of deep learning, and supplements my previous deep learning survey which provides hundreds of additional 29/12/2022, 12:21 Annotated history of modern AI and deep neural networks https://people.idsia.ch/~juergen/deep-learning-history.html 2/75 references. Finally, to round it off, I'll put things in a broader historic context spanning the time since the Big Bang until when the universe will be many times older than it is now. The present piece is also the draft of a chapter of my upcoming AI book. Disclaimer. Some say a history of deep learning should not be written by someone who has helped to shape it—"you are part of history not a historian." I cannot subscribe to that point of view. Since I seem to know more about deep learning history than others, [DL1-2] I consider it my duty to document and promote this knowledge, even if that seems to imply a conflict of interest, as it means prominently mentioning my own team's work, because (as of 2022) the most cited NNs are based on it. Future AI historians may correct any eraspecific potential bias.