Survival of the Fittest

John McCarthy first coined the term “artificial intelligence” for a 1956 summer research project at Dartmouth University. From that prominent beginning, he became the most influential of the early advocates vying to define AI’s research agenda.

The stated purpose of the summer project—which McCarthy organ­ized with Shannon, Marvin Minsky, and Nathan Rochester—was to proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it. An attempt will be made to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves.¹¹

McCarthy and his colleagues wanted to translate complex tasks into precise computational problems, and thereby to evolve computers into arenas previously reserved for humans. They had framed the problem brilliantly. They recognized that computers were creations capable of evolution, growth, and self-improvement—if only someone could teach them the precise steps needed to evolve, to grow, and to improve.

McCarthy’s careful early ministrations and innovations nurtured AI throughout its infancy. But alas, he failed to appreciate the true com­plexity of evolution. His 1956 proposal also asserted that: “we think that a significant advance can be made in one or more of these problems if a carefully selected group of scientists work on it together for a summer.”¹² This estimate was way, way, way off. The agenda outlined in that proposal continues to command the attention of research scien­tists today—and appears likely to do so throughout the foreseeable future.

McCarthy apparently failed to learn from that early faux pas. In 1969, he and Patrick Hayes published an influential article insisting that prob­ability theory was “epistemologically inadequate” for dealing with the challenges of AI.¹³ In lay terms, McCarthy and Hayes argued that the branch of mathematics specifically devoted to helping people make deci­sions in uncertain environments was inappropriate for translating uncer­tain human environments into computational terms. They advised their followers to avoid the only known branch of mathematics capable of solving the problems that they were trying to solve. Many heeded their advice. For close to two decades, orthodox AI insisted that intelligence— both natural and artificial—was strictly a matter of manipulating “symbols,” and that numeric, mathematical, or quantitative reasoning was inapropos.

Fortunately, not everyone followed that orthodoxy. Expert systems researchers were AI pragmatists who managed to get the basics of soft­ware evolution right despite AI’s ban on mathematics. They followed in Shannon’s footsteps by translating specific, narrow, not-strictly- computational tasks into mathematical models. The tasks that they chose turned out to be much less complex than chess; they investigated banal­ities like diagnosing cardiopulmonary diseases or prospecting for molyb- denum.¹⁴ Their basic approach was simple. They identified genuine human experts, interviewed them, and built huge databases of the lessons that they learned during these interviews. They encoded their entire data­base as a collection of “if... then... else...” rules, and generated some surprisingly insightful conclusions. While this original formulation was much too simplistic to capture more than a few special problems, it did demonstrate another important point about software evolution. Computing organisms require education to grow. This lesson echoes what Berliner learned along the way from correspondence chess to Deep Blue: General principles and strategic acuity reach their limits in a vacuum. Growth, evolution, and eventual excellence also require detailed information.

In addition to just raw information, though, growth and evolution require mechanisms for reasoning, particularly in the presence of uncer­tainty. And the scientific discipline devoted to uncertain reasoning is none other than statistics, which McCarthy and Hayes had banned from AI. Nevertheless, and despite the ban, various probabilists and statisticians continued to explore the role that their tools could play in translating conceptual tasks into computational ones—thereby evolving the trans­lation frontier upward.

Their work arrived at AI from two different directions. One school rejected the claimed inadequacy of probability theory outright. Its members demonstrated that the best way to evolve computers is to trans­late tasks involving uncertain reasoning into the only mathematical lan­guage capable of manipulating uncertain quantities in a logical manner: probability theory. This work, most of which applied Bayesian approaches to probabilistic modeling and decision making, drew together scholars of computer science, management science, operations research, cognitive psychology, behavioral economics, and statistics.¹⁵ Their systems combined the knowledge-intensive approach of expert systems with the more flexible and powerful structured interviewing techniques of decision analysis. These systems, and the principles behind them, began to emerge from the labs and enter the commercial arena in the late 1980s.¹6

A second school developed a set of neural networks capable of “learn­ing” statistical relationships;¹⁷ Star Trek’s writers were so enamored of this work that they credited Lt. Commander Data’s android intelligence to his neural network. And though neural networks have yet to approach Trekkie dreams, they did make a huge contribution to software’s evolu­tion. They demonstrated how a system could learn to reason while acquiring knowledge. Suppose that a software system began by making arbitrary connections—along the lines of directing all Simon and Garfunkel fans toward Mary J.

Eventually, even the priests of orthodox AI had to admit that they’d been wrong. They incorporated numeric reasoning back into their work. The innovations of expert systems, Bayesian decision analysis, and neural networks, had done more than fundamentally reshape the faith. They had also made major contributions to the evolution of the translation frontier by translating task after task from imprecise English into precise computation—even if the computational outputs were occasionally wrong.

The tenacity of these unorthodox scholars had brought down the ruling order, in the research world’s version of survival of the fittest. As the 1990s dawned, Al’s leading lights realized how to help software evolve in promising, important directions. That realization set the stage for some important evolutionary advances. The Internet would soon bring these newly evolved delights to our desktops and living rooms, and we would soon come to marvel at our noble ancestors who had somehow managed to navigate the unwired world of the pre-information age.