THE ARPANET ERA AND THE EMERGENCE OF TCP/IP

Whereas creating a stable, integrated field of telephony required a top-down strategic approach, the task of creating a unified data communication field was altogether dif­ferent.

ARPANET was one of the most successful instances of sustainable government invest­ment in infrastructure (Leiner et al., 1997). Unlike the privately funded public switched telephone network, the ARPANET was publicly financed through the Advanced Research Project Agency (ARPA) to the amount of $833 million over four years (Pelkey, 2014, Chapter 6). The government’s willingness to back such an untried venture reflected the Cold War climate of the time. In October 1957 the Soviet Union launched Sputnik, raising US fears that the US prowess in science and defense technology was at risk (Abbate, 1994, p. 8). In response, President Eisenhower prioritized research and devel­opment, resulting in the 1958 establishment of ARPA within the US Department of Defense (ibid.; Hafner and Lyon, 1996). Tasked with testing innovative technologies such as packet switching, distributed computing, and the interconnection of heterogeneous computer systems, the aim was to enhance productivity and innovation by allowing for greater resource sharing (Bolt, Beranek and Newman, Inc., 1981, pp.

Despite the magnitude of the effort, ARPA was not organized from the top down, with a strict hierarchy and standardized roles and practices. Instead, seeking to capture the benefits of diversity, ARPA was organized in an open, loosely coupled, decentralized fashion, so that roles and relationships were not set a priori but rather negotiated in an ongoing practice (Hauben, 1997; Ryan, 2010). Thus, ARPA did not conduct research on its own. Instead, it established a distributed division of labor, funding major computer research centers with grants of $500 000 to $3 million to work on various aspects of inter­related computer research (Hafner and Lyon, 1996, p. 46). The first four centers were at the University of California, Los Angeles, Stanford Research Institute, the University of California, Santa Barbara, and the University of Utah. In addition, ARPA contracted with the firm Bolt, Beranek and Newman (BBN) to build the communications layer and administer the system. By addressing different, but related, aspects of research, each center complemented the others (Hauben, 1997).

With the research centers in place, ARPA coordinated its efforts through informal working groups comprised of representatives from each center. These centers pooled their diverse resources, and developed a framework for building a network of heterogene­ous computers and researchers that could work together and share resources. Their goal was to create a low-level communication platform, upon which higher communication functions might be built (Abbate, 1994; Hafner and Lyon, 1996; Hauben, 1997).

The principal investigators (PIs) from the computer centers were at first reluctant to engage in collaboration (Abbate, 1994, p. 53). Nonetheless, a core group of PIs com­mitted to pursuing the ARPANET vision. Together they formed the Network Working Group (NWG), which assembled periodically ‘to develop software for host comput­ers and discuss early experiences with the network’ (ibid., p.

A new technology - packet switching - was adopted to carry out the functions of the sub-network. This technology allowed messages to be routed through the sub-network via a common ‘message’ protocol, where they were broken up into packets that could then be transmitted separately through the network, taking different paths depending on which were free at the moment. On reception at the host computer, the packets would be recombined in the correct sequence. Packet switching was well suited to handling the ‘bursty’ traffic associated with data transmission. Moreover, it was faster, cheaper, and more accurate than circuit-switched telephony, and as such it was ideal for real-time interactive communication (ibid., p. 44). This was evermore true as the cost of switching in relationship to transmission significantly declined (MacKie-Mason and Varian, 1994, pp. 4-5).

ARPA’s network - the ARPANET - evolved into three layers: a sub-network, or communication layer that comprised the packet-switching IMPs; a host layer, labeled the Network Control Center, which provided end-to-end communication between host computers as well as a universal interface for user service; and an applications layer that fed data to the Network Control Program (NCP) where it was packaged and sent to the local IMP (Abbate, 1994, pp. 45, 58). Once this architecture was established, build­ing the network was relatively straightforward: by 1969, the sub-network was put into place, while the host and application protocols followed in the next two years (ibid., p.

Creating a social network linking researchers at the computer research centers was somewhat more difficult. Despite the potential network benefits, a culture of sharing was slow to take hold. Given the limited resources at each computer center, researchers were hesitant to expend their efforts collectively. Moreover, demand was low because available applications were limited, a problem that the NWG sought to rectify by developing new software (ibid., p. 63). Over time, however, ARPA’s informal, decentralized, consensus­based management style and participatory organizational culture promoted a collective approach to problem solving (ibid.; Passy, 2003). The researchers’ common background in computer science played an important role in this regard (Hauben, 1997). Because this new field was so small, many of its top researchers were acquainted with, if they did not personally know, one another (Abbate, 1994, p. 77; Hafner and Lyon, 1996). Moreover, engaging in many of the same problems, they shared much in common (Abbate, 1994, p. 80; Hauben, 1997). Equally important, their ‘hacker’ culture emphasized openness, experimentation, and freewheeling, interactive engagement, as exhibited in the reitera­tive ‘Request for Comment Process’ (Hauben, 1997; Hauben and Hauben, 1997; Ryan, 2010, p. 35). This shared culture not only fostered participation, it also generated the social capital required for collaboration and knowledge sharing to take place (Coleman, 1988; Tsai and Ghoshal, 1998; Inkpen and Tsang, 2005, p. 151). Hence, it was only as a last resort that the ARPA directors had to use their financial leverage to gain compliance (Abbate, 1994, p. 79).

Such synergies and positive externalities continued to expand the ARPANET as experimental applications, such as remote interactive login, file transfer protocol, and electronic mail were developed by users, refined by the NWG, and put into place. In 1971, ARPANET ran at only 2 percent of its capacity (Rosenzweig, 1998, p.

Efforts were also underway to port ARPANET to packet radio and satellite technol­ogy. However, to transfer data packets across these diverse networks required redesigning the NCP protocol (Russell, 2006, p. 50). Between 1973 and 1978, four reiterations took place. The final version, designed to accommodate voice transmission, divided the origi­nal NCP protocol into two layers, TCP (Transmission Control Protocol) and IP (Internet Protocol) (Cerf, 2004, pp. 2-5). Accordingly, the TCP breaks messages into streams of packets at the source, and reconfigures them when they reach their destination. In turn, the IP addresses the messages, ensuring that the packets are routed across multiple nodes and even across multiple networks with different standards (Sterling, 1993).

TCP/IP’s open, non-proprietary protocol engendered numerous externalities that moved the ARPANET up the diffusion curve. This growth was propelled by externalities that stemmed not only from the extension of the TCP/IP across networks of all types, but also from the growing availability of low-cost computers, and the creation of an extended user base that was engaged in network applications development (Rosenzweig, 1998; Ryan, 2010). In fact, so great was the ARPANET’s growth it soon overwhelmed ARPA’s ability to take it forward (Russell, 2006, p.

Thus, in 1972, ARPA sought out private-sector actors to assume responsibility for the network. There were no bidders, however (Rosenzweig, 1998). Skeptical of ARPA’s venture from the start, and fearing that data networking might undermine its own telephone monopoly, AT&T had emphatically shied away from any offers of collaboration (Abbate, 1994; Hafner and Lyon, 1996, pp. 65-6; Ryan, 2010, p. 18). And the computer industry, although by now realizing the market for data communication, was threatened by open standards (Ryan, 2010, p. 44). To maintain the industry’s integrated businesses intact, IBM and DEC (the Digital Equipment Corporation) sought to develop proprietary data communication standards of their own. With no alternatives in sight, the ARPANET was turned over to the Defense Communication Agency (DCA) in 1975 (Abbate, 1994). This transfer did not tighten the reigns on the ARPANET, as one might have expected; instead, the DCA made the open standard, TCP/IP, mandatory for all ARPANET users (Russell, 2006). In 1983, access was extended further, when the military cordoned off its segment, MILNET, but maintained TCP/IP as the ultimate, connecting link.

By all measures, the ARPANET was a remarkable, innovative feat. Its success stemmed not only from having public funding in its initial stage, but also from its unique institu­tional milieu, which engendered a community of practice (Wenger, 1998) that encouraged trial and error, learning by doing, and participatory feedback (Brown and Duguid, 2001). The architecture of the development process was itself exceptionally important. As characterized in social network theory, both the ARPA process and the ARPANET took the form of a small world network (Watts and Strogatz, 1998; Watts, 1999; Buchanan, 2002; Uzzi et al., 2007; Borgatti et al., 2009), which is optimal for innovation to take place (Burt, 1992; Uzzi, 1997; Burt, 2005; Fleming and Marx, 2006). As Uzzi and others have described it, a small world network is characterized by cohesive clusters composed of strong ties that provide the trust and social capital required for collective action, and weak ties emanating from these clusters that assure that new and diverse information is made available. In ARPA’s case the research centers constituted the cluster of strong ties, whereas the NWG allowed for interconnection based on weak ties.

The broader institutional environment also favored ARPANET. Government invest­ment, sparked by international crises, allowed ARPA to experiment as well as operate on a trial and error basis without having to preoccupy itself with profit-making and other market criteria. Similarly, cordoned off in the Department of Defense, ARPA research­ers could operate without external stakeholders and rivals, thereby establishing a com­munity with a common narrative and script, which fostered innovation and knowledge sharing. At the same time, ARPA benefited from external system-wide advances, such as those in computer technology and computer science, which generated many of the exter­nalities that fueled the diffusion of ARPANET.

ARPA, which spanned the worlds of academics, computer scientists, defense special­ists, and industry practitioners, constituted an organizational field, with its own internal modes of operation. However, unlike the field of telephony, which was highly structured and deeply embedded in society, the ARPA field was both loosely coupled as well as lightly embedded in its surroundings. As a result, in contrast to AT&T, which was unable to adapt to its changing environment, ARPA’s flexible structure allowed it to coevolve when its very success brought about major changes in the landscape. Moreover, the open, interactive culture developed during this period generated the modus operandi for the period that followed (Ryan, 2010, p. 33).

26.5