How AI Has Progressed Over 70 Years

Mike's Notes

A solid overview of AI's long history.

Resources

• https://www.preprints.org/manuscript/202608.1992
• https://epoch.ai/publications/compute-trends
• https://en.wikipedia.org/wiki/Transistor_count
• https://ourworldindata.org/grapher/computer-chess-ability
• https://github.com/computer-chess-index/cci
• https://chessbenchllm.onrender.com/
• http://arxiv.org/abs/https://onlinelibrary.wiley.com/doi/pdf/10.1609/aimag.v27i4.1904
• https://doi.org/10.1609/aimag.v27i4.1904
• https://doi.org/10.1017/CBO9780511819346.028
• http://arxiv.org/abs/2009.06489
• http://www.incompleteideas.net/IncIdeas/BitterLesson.html
• https://doi.org/10.1109/IJCNN55064.2022.9891914
• http://arxiv.org/abs/2111.15366
• http://arxiv.org/abs/2104.12871
• https://www.blog.ajabbi.com/2026/01/richard-sutton-father-of-rl-thinks-llms.html
• https://www.blog.ajabbi.com/2023/12/john-henry-holland.html

References

• Reference

Repository

• Home > Ajabbi Research > Library > Authors > Carlos Gershenson
• Home > Ajabbi Research > Library > Subscriptions > Complexity Digest
• Home > Handbook > 

Last Updated

17/09/2026

How AI Has Progressed Over 70 Years

By: Mario Franco, Zeinab Davoudmanesh, Sean P. Maley, Fernanda Sánchez-Puig, Carlos Gershenson

PrePrint Press: 27/08/2026

Mario Franco: Systems Science and Industrial Engineering, Binghamton University, Binghamton, NY, USA.

Zeinab Davoudmanesh: Systems Science and Industrial Engineering, Binghamton University, Binghamton, NY, USA

Sean P. Maley: Mathematics Department, Finger Lakes Community College, Canandaigua, NY, USA 

Fernanda Sánchez-Puig: Facultad de Ciencias, Universidad Nacional Autónoma de México, Mexico City, Mexico

Carlos Gershenson: Systems Science and Industrial Engineering, Binghamton University, Binghamton, NY, USA.

...

Abstract

Seventy years of artificial intelligence are usually told as a long preamble followed by a revolution beginning around 2012. We organize the period differently, around a question the field has answered differently at different times: what kind of thing is intelligence, such that a machine could have it? Read that way, the human contribution does not withdraw as systems learn more; it relocates, and mostly to places our instruments do not record. Whether the recent acceleration is a change in kind or a change in budget is, we suspect, the more interesting question, and not one that benchmark curves can settle.

Keywords:

artificial intelligence; machine learning

1. Introduction

The Dartmouth workshop of 1956, which launched artificial intelligence as a formal discipline [1], brought expectations of rapid progress. Progress did occur, but not uniformly: seven decades of AI development have included periods of acceleration, two major funding contractions known as the “AI winters” and a new race to the bottom as models are becoming increasingly more powerful. Measuring this trajectory is essential for researchers, policymakers, and the broader public.

This opinion piece aims to put into perspective seventy years of artificial intelligence, usually told as a long preamble followed by a revolution beginning around 2012. However, we consider it more interesting to organize it around the question the field has answered differently at different times: what kind of thing is intelligence?

2. From Cold-Logic to Large Black Boxes

Any periodization of a seventy-year field is a convenience rather than a finding. The boundaries below are drawn where the dominant answer to a single question changed: what kind of thing is intelligence, such that a machine could have it?

The first era took intelligence to be a matter of logic, search and heuristics. It predates the field’s own founding: McCulloch and Pitts described a logical calculus of neural activity in 1943, and Turing’s 1950 paper set the terms of the discussion before the Dartmouth workshop of 1956 supplied a name. The Logic Theorist (1956) and the General Problem Solver (1957) assumed that reasoning reduced to symbol manipulation and that adequate search would do the rest. The assumption was not unreasonable, but the search spaces grew faster than the machines. This periodization inadequacy is evident from the start: Rosenblatt’s perceptron (1957) learned its weights from examples rather than having them written down, and Samuel’s checkers player (1959) improved by playing against itself. Both belong to an answer the field would not take up in earnest for another thirty years. Minsky and Papert’s 1969 analysis and the Lighthill report of 1973 closed off much of the funding, producing the first winter [2,3].

The second era argued that the missing ingredient was not search but knowledge: what a program could not derive, it could be given. DENDRAL (1964), MYCIN (1979) and later XCON (1980) demonstrated that narrow expertise could in fact be encoded, and by the mid-1980s, expert systems were a commercial industry with national programmes. Knowledge, however, is expensive. Rules had to be elicited one at a time from experts who could not always articulate what they knew, the resulting systems failed unpredictably at the edges of their coverage, and the specialized hardware market collapsed in 1987, producing a second winter [4].

The third era moved from providing to learning the knowledge. Many popular bioinspired methods enter into stage around this time: genetic algorithms and genetic programming, particle swarm optimization, etc. Particularly, backpropagation — having been derived independently several times before Rumelhart and colleagues brought it to the field’s attention in 1986– made neural network approaches practical. Progress was real but modest, limited by the data available and by the hardware that happened to exist [5].

The fourth and present era begins when raw, unadultered computational power became increasingly more accessible; intelligence here is quantity, and the answer is more of it, please [6]. AlexNet won the 2012 ImageNet competition with a top-5 error of 15.3%, against roughly 26% for the best entry still relying on hand-crafted features; three years later, residual networks reported 3.57% for an ensemble. AlphaGo followed in 2016. Nothing in the underlying mathematics was fundamentally new; it had finally been given enough hardware and data. What nobody anticipated is that the recipe had no obvious stopping point. The transformer in 2017 removed the last structural obstacle to enlarging it, and early GPT models turned it into a strategy: capability became something purchased rather than designed, and scale became the field’s explanation of first resort. As summarized in Figure 1, through the first three eras, training compute roughly tracked available hardware, doubling every two years or so, which is what one expects when researchers use whatever machines exist. From around 2010 it grew by a factor of four to five per year, an acceleration that hardware improvement does not explain and that investment does [7].

Figure 1. Growth in AI training compute from 1950 to 2026 compared with the trajectory predicted by Moore’s Law. Line graph of AI training compute versus year on a logarithmic scale. Training compute increases gradually from the 1950s through 2000s, then rises much more rapidly after the mid-2010s. A dashed reference line shows the trajectory expected from Moore’s Law, illustrating that training compute grows faster than Moore’s Law around the 2010s. Training cost data taken from Epoch AI and transistor count data from Wikipedia.

3. Progress Told by Milestones

Consider the following selected systems, across five different domains, shown in Table 1. Early systems were explicitly told what to do: six rules and a small dictionary for translating Russian, a script for holding a conversation, chemistry written out longhand by chemists. Later ones were told almost nothing, and did considerably more. Effort that once went into rules went into designing features, then into labelling a million photographs, then into assembling corpora, filtering them, and keeping the machines that consume them running. None of this appears in a table of systems, for the same reason the expert systems appear in no curve of training compute:  the work moved somewhere the instrument does not read.

Table1.Selected illustrative systems by domain,1952–2025.

Domain	Year	System	Relevance	Source
Games	1959	Samuel’s checkers	First program to get better at something by practicing	Samuel 1959
	1991	TD-Gammon	Reached near-professional backgammon from the rules only	Tesauro 1991
	1997	Deep Blue	Beat the world chess champion by examining two hundred million positions a second	Campbell 2002
	2016	AlphaGo	Beat arguably the best Go player in a game with more legal positions than there are atoms in the universe	Silver 2016
	2017	AlphaZero	Taught itself chess, shogi and Go to superhuman level in a day, with no human games to copy	Silver 2017
	2024	SIMA	Generalist AI agent for 3D virtual settings	SIMA 2024
Vision	1958	Perceptron Mark I	A machine that learned to see shapes	Rosenblatt 1958
	1979	Neocognitron	First appearance of the design behind every modern vision system	Fukushima 1979
	1998	LeNet-5	Read the handwriting on cheques commercially	LeCun 1998
	2012	AlexNet	Surpassed hand-crafted methods on image classification	Krizhevsky 2012
	2015	ResNet	Surpassed human performance on image classification	He 2016
	2026	RF-DETR	Real-time object detection, segmentation and keypoint detection	Robinson 2026
Speech	1952	Audrey	First system to recognize spoken digits	Davis 1952
	1976	Harpy	First system to recognize a 1,000-word vocabulary of connected speech	Lowerre 1976
	1989	Sphinx	First speaker-independent system	Lee 1989
	2012	DNN acoustic models	Neural networks surpased traditional methods	Hinton 2012
	2026	RosettaSpeech	Zero-shot speech-to-speech translation	Zheng 2026
Language	1954	Georgetown–IBM	Translated sixty Russian sentences (and promised the rest within five years)	Hutchins 2004
	1966	ELIZA	First system to get attributed human-like feelings	Weizenbaum 1966
	2003	NPLM	Words are transformed from symbols to positions in a space (embeddings)	Bengio 2003
	2017	Transformer	Proposed the architecture behind the LLM revolution	Vaswani 2017
	2020	GPT-3	Turned scale from an engineering detail into a research programme	Brown 2020
	2025	DeepSeek-R1	Learned to work through a problem step by step from reward alone	DeepSeek 2025
Science	1969	DENDRAL	First system to do professional scientific work	Buchanan 1969
	1976	MYCIN	First drug-recommender system (never deployed)	Yu 1979
	2020	AlphaFold 2	Solved the protein folding problem, a critical problem in biology	Jumper 2021
	2025	AlphaProof	First system to reach medal level at the International Mathematical Olympiad, with machine-checkable proofs	Hubert 2025

Chess is probably one of the few continuous records in existence that measures machine progress in a specific task, mostly thanks to hobbyists keeping the score. Progress is summarized in Figure2. Attempts to play chess by machine go a long way back, predating the formal discipline itself by four decades, and showing that the ambition to build thinking machines is older than any of the ways we have found to measure it. El Ajedrecista played a rook endgame in 1912 and Turochamp was executed by hand in 1948; Belle earned the grade of master in 1983, and in 1997 Deep Blue beat the reigning human world champion. Overall, model performance has been roughly improving by an average of 40 to 50 points each year, approximately; with only a visible slowdown in recent years (present models are believed to be near optimal play). Hand-tuned heuristics, purpose-built silicon, self-play networks and the current hybrids all sit on the same slope; techniques varied but progress was roughly constant, whatever it cost to build the system behind. Present language models asked to play are improving rapidly and arrive at a level dedicated programs passed in the late 1970s, though 54 of the 105 evaluated models fell below the benchmark’s random-mover anchor. In other words, extremely impressive systems, capable of discussing advanced physics and adept programming, lose to an agent that simply picked random valid moves. Nonetheless, they are the first thing in seventy years that is not on the line.

Figure 2. Chess playing strength, 1965–2026. Open circles are strengths inferred from games against human opponents, with error bars; AlphaZero, is placed from its match against Stockfish 8. Solid line indicates the SSDF year-end leader, calibrated against human play in 2000. Dashed line indicates Stockfish releases, hardware fixed. Grey dots are all engine releases in the Computer Chess Index, on present hardware. Orange dots are language models at release, rated by Glicko-2 against engine anchors. Elo for LLM models has no true zero; anchor for random legal play sits at 400. Vertical distances between series are indicative only (rating pools were never anchored to one another). Sources: SSDF via Our World in Data; Computer Chess Index; LLM Chess Benchmark.

4. The Current Frontier

Four things converged around 2012, and the acceleration is usually attributed to some combination of them. 1. Compute became abundant and then cheap enough to waste: CUDA made a generation of graphics hardware programmable by people who were not graphics programmers. 2. A handful of architectural changes — residual connections above all, together with better optimizers — removed the practical ceilings on neural networks depth and length. 3. Curated data reached a scale no individual laboratory could have assembled, first in domain-specific datasets, like ImageNet, and later in the web itself. 4. Expenditure entusiasm, less often listed, opened doors to computational scales far beyond any research budget of the previous decades. Which of these was necessary and which merely coincident is not an easy question to answer.

What followed has a recognizable shape. A benchmark is proposed as a measure of something machines cannot do; within a few years, systems approach or pass the human figure attached to it; a harder benchmark replaces it, and the previous one is retired to the appendix of model reports. The target is abandoned and a harder one takes its place: chess gave way to Go, then to imperfect information games, like poker or StarCraft, each chosen because the previous one had fallen to methods that did not generalize. The format is kept and the difficulty raised: MMLU gave way to GPQA and then to Humanity’s Last Exam; the same short, checkable questions, asked of progressively narrower expertise. Or the authors redesigned the benchmark, as ARC-AGI has now done twice, around the failures the previous one exposed, which is the field conceding the pattern in advance. What is being measured has changed along with them. The benchmarks of the last decade posed questions with answers; the ones now being written pose tasks with a duration — repair this repository, complete this workflow, operate this software unsupervised for an hour. The quantity of interest is no longer accuracy but how long a system can be left alone before it goes wrong, which is harder to score and harder to game. Whether this measures intelligence better or merely measures usefulness better is not a question the field has settled.

Thus, a section titled “the current frontier” is obsolete before it is printed, and ours will be too: the systems and scores we name here were superseded during the writing of this piece. Each one was built to be just out of reach, so each is passed shortly after it is built, and the sequence measures how quickly the field can construct new measures of its own inadequacy [8,9].

5. Discussion and Implications

Seventy years of artificial intelligence are usually told as a long preamble and a revolution. What the record shows instead, we argue, is a field that has changed its answer about four times to a single question, and a set of instruments that only began recording partway through. Intelligence was taken to be search, knowledge, learning from examples, and now something bought. Each answer produced real successes, and each ended not because it was refuted but because it met a cost it could not pay [10]: search spaces grew faster than machines, rules had to be elicited one expert at a time, and learning waited two decades for data and hardware. The divisions we have drawn here are conveniences — Deep Blue is a first-era system operating in the third, and the expert systems that occupied a decade appear in no curve of training compute at all, because they do not train. A history assembled only from the quantities we can plot will report that little happened before 2012.

The most consistent movement across the whole period is not in the systems but in what was asked of the people building them. From writing the solution, to designing the features, to labelling the examples, to supplying nothing but a corpus that already existed. The effort moved — into curation, filtering, evaluation, and the operation of machines whose cost is now the field’s dominant fact — and it moved somewhere our instruments cannot reach so easily.

What remains open is whether any of this constitutes a change in kind. Training compute now grows four to five times a year, while hardware doubles every two, and benchmark curves cannot settle what that means. We suspect the honest position is that the field has become enormously more expensive, that it has become correspondingly more capable, and that these are not yet known to be the same claim. Furthermore, we can only speculate what might be the limits to both of these.

References

1. McCarthy, J.; Minsky, M.L.; Rochester, N.; Shannon, C.E. A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence: August 31, 1955. AI Magazine 2006, 27, 12–14, [https://onlinelibrary.wiley.com/doi/pdf/10.1609/aimag.v27i4.1904]. https://doi.org/ 10.1609/aimag.v27i4.1904. 2. 3. 4. 5. 6. 
2. Minsky, M.; Papert, S. Perceptrons: An Introduction to Computational Geometry; MIT Press: Cambridge, MA, 1969.
3. Lighthill, J. Artificial Intelligence: A General Survey. In Artificial Intelligence: A Paper Symposium; Science Research Council: London, UK, 1973.
4. Nilsson, N.J., Speed Bumps. In The Quest for Artificial Intelligence: A History of Ideas and Achievements; Cambridge University Press: Cambridge, UK, 2010; chapter 24, pp. 305–330. https://doi.org/10.1017/CBO9 780511819346.028.
5. Hooker, S. The Hardware Lottery. CoRR 2020, abs/2009.06489, [2009.06489].
6. Sutton, R.S. The Bitter Lesson. http://www.incompleteideas.net/IncIdeas/BitterLesson.html, 2019. Incomplete Ideas (blog), 13 March 2019. © 2026 by the author(s). Distributed under a Creative Commons CC BY license.Preprints.org (www.preprints.org)  |  NOT PEER-REVIEWED  |  Posted: 27 August 2026 doi:10.20944/preprints202608.1992.v1 6 of 6 7. 8. 9.
7. Sevilla, J.; Heim, L.; Ho, A.; Besiroglu, T.; Hobbhahn, M.; Villalobos, P. Compute Trends Across Three Eras of Machine Learning. In Proceedings of the 2022 International Joint Conference on Neural Networks (IJCNN), 2022, pp. 1–8. https://doi.org/10.1109/IJCNN55064.2022.9891914.
8. Raji, I.D.; Bender, E.M.; Paullada, A.; Denton, E.; Hanna, A. AI and the Everything in the Whole Wide World Benchmark. CoRR 2021, abs/2111.15366, [2111.15366].
9. Chollet, F. On the Measure of Intelligence, 2019, [arXiv:cs.AI/1911.01547]. 10.
10. Mitchell, M. Why AI is Harder Than We Think. CoRR 2021, abs/2104.12871, [2104.12871]. 

Disclaimer/Publisher’s Note:

The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.