Mutation of a Sector

Integration, translation, and modular design long have been critical to the information sector. That slow evolutionary process mutated to give us the full-blown Internet only about a decade ago.

The spread of computing and networks throughout the 1980s and early 1990s wasn’t restricted to the research world. At least two other important sectors also witnessed rampant computerization: offices and homes. Office automation began with selected administrative, account­ing, and word-processing capabilities, then progressed to place a PC on every desk, and eventually interconnected all of those computers via a local area network (LAN). Companies with multiple locations began to network these LANs together to create private, often nationwide (or even global) networks. During this same period, many homes acquired their first PCs, primarily to perform business-like tasks—home finances, writing, homework, etc.—but soon discovered that simply owning a computer created an unexpected set of opportunities.

The 1990s thus began with widespread computing connected to a number of separate, often proprietary, networks. Two further innova­tions were necessary before that world could evolve into its current form. Perhaps the single greatest weakness of the research-oriented Internet lay in indexing. As a matter of technology, any user on a network could access any file residing anywhere on the network. As a practical matter, there was no systematic way for users to know which files were avail­able, what information they contained, or where they were located. Put another way, the natural science on both sides of the divide was fine; what we lacked was one of Simon’s carefully designed interface artifacts. In 1989 Tim Berners-Lee, a researcher at the European Laboratory for Particle Physics (CERN) in Geneva filled that gap. He developed an indexing scheme that gained rapid and widespread acceptance.²²

Berners-Lee’s system combined a text-formatting system called HTML (the Hypertext Markup Language), a communication standard called HTTP (the Hypertext Transfer Protocol), and an addressing scheme to locate “Web sites” called a URL (Universal Resource Locator). This com­bination essentially broadcast every file’s content and location to every user on the network. Broadcasts are only valuable, however, in the pres­ence of receivers. These receivers, known as browsers, allowed users to locate files indexed by URL. The combination of browsers and search engines—programs that search files’ keyword lists and return their URLs—enabled full-scale navigation of the Internet. Navigability soon made the original Internet so popular that previously proprietary net­works felt compelled to join. They too adopted URLs, enabled keyword indexing, and revised their network protocols to be compatible with those of the ARPA-originated Internet.

Of course, Berners-Lee didn’t create the Web tabula rasa. He built upon many layers of research contributions to provide some critical missing pieces. He himself recognized that Vannevar Bush—head of the U.S. Office of Scientific Research during World War II—foresaw some­thing like the Web as early as 1945. Bush had hoped to build the Memex, a machine that would combine binary coding, instant photography, and other technologies either available or foreseeable in the 1940s, to index and to cross-reference texts stored as microfilm. Twenty years later, Ted Nelson coined the term “hypertext” and predicted that computers would soon allow people to write in this nonlinear format. His futuristic machine of 1965, Xanadu, would publish all of the world’s information as hypertext.

But the basic ideas underlying hypertexts and hyperlinks predate even Memex and Xanadu—by more than a few centuries. Journalist Jonathan Rosen demonstrated how the arcane techniques of biblical exegesis embodied in the Babylonian Talmud presaged the Web long ago, in the great Jewish academies of what is now Iraq, in the centuries before the Arab invasion.

I have often thought, contemplating a page of the Talmud, that it bears a certain uncanny resemblance to a home page on the Internet, where nothing is whole in itself but where icons and text boxes are doorways through which visitors pass into an infinity of cross-referenced texts and conversations. Consider a page of the Talmud. There are a few lines of Mishnah, the conversation the Rabbis con­ducted... stemming from the Bible Underneath these few lines being the

Gemarah, the conversation later Rabbis had about the conversations earlier Rabbis had in the Mishnah Running in a slender slip down the inside of the

page is the commentary of Rashi, the medieval exegete, commenting on both the Mishnah and the Gemarah, and the biblical passages (also indexed elsewhere on the page) that inspired the original conversation.

Rosen’s parallels between Talmudic cross-referencing and hypertext are instructive. Not because huge numbers of Web surfers are likely to access the Talmud—though they could²⁴—but rather because they illus­trate an important point about technological development and innova­tion. Talmudic study developed slowly, over the course of centuries. While the discussions themselves included many cross-references, the true challenge arose in designing an appropriate layout. How could printers assemble all of these commentaries-upon-commentaries in a manner that successive generations of scholars could access? It took a certain amount of ingenuity, but they did it. They devised a layout that anyone could follow. Anyone, that is, with suitable Talmudic training. No wonder “exegesis” sounds arcane! The beauty of the Web is that it made this type of nonlinear indexing available to the broad public. And while some still find the Web a daunting place filled with strange dead ends, pop-up ads, irrelevant links, frustrating searches, and unwelcome solicitations, the training necessary to bring them up to speed falls far short of the training needed to educate a Talmudic scholar. First the Inter­net and then the Web democratized a form of nonlinear thinking that was, for ages, the sole province of an esoteric priesthood.

It was Berners-Lee’s contributions that made that final step possible: HTML, HTTP, and URL made casual exegesis a reality for millions—so casual, in fact, that we relabeled it “surfing.” They also completed the Internet’s basic plumbing. The PC revolution had placed computers on every desktop, in all but the least-affluent American homes, and in many public places. The networking revolution had ensured that most of these computers were at least equipped to join a network.

As it turned out, the Internet puzzle was still one crucial piece shy of a commercial explosion. The newly complexioned Internet gave busi­nesses and consumers a fundamentally new, low-cost way to exchange information. But relatively few companies felt the need to rush into the online world, and relatively few consumers clamored for such access; total goods sold on the Internet during 1996 reportedly fell below $3 billion. The source of this casual disregard was obvious. Once again, the initial research focus of the Internet had colored the way that the tech­nology had matured—and the features that excited technologically ori­ented researchers (a.k.a., geeks) didn’t necessarily excite typical companies and consumers.

Early browsers contained inelegant textual interfaces useful for the retrieval of text and data files. These browsers became popular among academic researchers, college students, and technophiles but failed to capture the imagination of the public at large.²⁵ In 1993, a group of stu­dents at the University of Illinois developed Mosaic, the first platform­independent, user-friendly, graphics-enabled browser. In 1994, several key members of the Mosaic team (led by Marc Andreessen) moved to the Silicon Valley and joined forces with tech-industry veterans (notably Jim Clark, the founder of Silicon Graphics)²⁶ to form a company dedi­cated to the improvement and commercialization of Mosaic: Netscape.

Not only was Netscape’s IPO a precursor of the bubble to come,²⁷ but its flagship product, Navigator—a modular design sitting atop estab­lished translation frontiers, in the spirit of Brooks—motivated the first serious analysis of the new interfaces to the Internet—a design analysis in the spirit of Simon. Cusumano and Yoffie studied the people and ideas that drove Netscape’s stratospheric rise. They interviewed many of Netscape’s key employees, studied the company’s strategic coups and missteps, and contemplated the meaning of Competing on Internet Time.²⁹

But Netscape’s strategic maneuvering was only part of their analysis. The other—and ultimately the more important—part of the “competi­tion” came out of Redmond, Washington. On Pearl Harbor Day— December 7, 1995—Bill Gates announced that Microsoft was redefining itself around the Internet, redesigning all of its software products to be Web-centric, and developing a browser that would be free forever. The news that Microsoft “got” the Internet spread like wildfire through the tech-investment community; some routes to the Internet might no longer pass through Netscape’s browser. Netscape soon lost half of its market cap. Microsoft had fired the opening shots of the “browser wars” and set the stage for the trial that would soon share the front pages with the Netscape-inspired bubble.

The browser wars redefined the terrain of network-connectivity providers. As late as 1995, the network world consisted of the research- oriented Internet and a number of separate private and/or proprietary commercial networks. Netscape’s Navigator, and soon thereafter Microsoft’s Internet Explorer, made the Internet too attractive for anyone to sustain a stand-alone proprietary model. By the end of 1996, even the largest proprietary networks recognized that the Internet age mandated a new business plan. AOL linked its previously proprietary network to the Internet, adopted the advertising slogan “The Internet and More,” began offering some of its proprietary content to nonmembers, and changed its fee structure from a base-plus-usage system to a flat fee for unlimited use.²⁹

Meanwhile, as AOL labored to redefine home connectivity and content provision in the Internet age, the browser wars raged—until Netscape conceded in late 1998. Netscape’s leaders realized that they lacked the resources to compete with Microsoft’s tactics—many of which violated the antitrust laws and reminded tech investors why it’s good to be a monopolist—and sought a richer partner. AOL acquired Netscape, and Microsoft’s Internet Explorer became the dominant gateway to the Inter­net. The true irony in Netscape’s defeat, though, was not its choice of adoptive parents, but rather its timing. Netscape’s demise as a stand­alone entity in November 1998 marked the first failure of an inevitable Internet monopolist—at almost precisely the moment that investors launched the next wave of Internet IPOs to stratospheric valuations.

But those tales, exciting as they were, only obscured the real trends that had finally mutated into a vibrant information sector. Every time that AI translates a new task from the qualitative world of humans to the computational world of machines, the information sector’s potential grows. Every time that software engineering implements that translation as a software product, the information sector’s actual reach grows. Soft­ware’s evolutionary stage thus defines the information sector’s size, shape, scope, and importance.

That evolution has been rapid and profound. From its humble begin­nings as a tool that allowed select government scientists to crunch large numerical problems, the information sector has evolved into the domi­nant communication platform of the global economy. It has shaken our beliefs about correspondence, about reference libraries, about financial services, about shopping, and about the availability, utility, and ubiquity of information—both critical and frivolous. And it’s about to take over the world of entertainment.

Napster’s peer-to-peer (P2P) file sharing has already shaken the music industry’s self-image. Jazz greats Louis Armstrong and Ella Fitzgerald never thought of themselves as “content creators.” Grateful Dead lyri­cist John Perry Barlow and Metallica drummer Lars Ulrich very clearly do. Ulrich testified in front of the Senate Judiciary Committee about the need to protect his bit strings; he described P2P as “old fashioned traf­ficking in stolen goods.”³⁰ Barlow rose to Napster’s defense. He con­tended that P2P opened exciting new vistas for artists, enhanced their ability to serve their fan bases, and increased the economic welfare of artists, though likely at the expense of the record labels and production companies: “I do not believe that the kid in Ohio is injuring my eco­nomic interests by sharing my music with another fan in Los Angeles, Tokyo, or Dublin. Deadheads have been sharing our songs with each other for decades, and it did nothing but increase the demand for our work.”³¹ Barlow put his activism where his mouth is by helping to found the cyberlibertarian Electronic Frontier Foundation.³²

That sort of evolutionary encroachment relates to the artificial science. A series of scientific queries first introduced to allow Shannon’s computer to play chess evolved in a matter of decades to enable kids around the world to steal from Metallica while enriching the Dead. It also framed a critical strand of the information sector. The browser wars and the P2P wars reappear later, in chapters 5 and 7, respectively. For the moment, though, they both illustrate the roles that the artificial science of soft­ware evolution and its practical counterpart of software engineering played in the information sector’s emergence from the ivory tower and onto the front pages.