RISK REDUCTION BY MARKET SOLUTIONS AND GOVERNANCE CHOICE: OPPORTUNITIES AND LIMITATIONS

There are two perspectives on governance and algorithms: governance by algorithms refers to the above-mentioned power of technology and the ability of algorithms to shape society (Just and Latzer, 2015); governance of algorithms refers to the practices to control, shape and regulate algorithms (Saurwein et al., 2015).

This section discusses justifications, opportunities and limitations for the govern­ance of algorithmic selection. From a public interest point of view, governance should reinforce benefits and minimize risks. Benefits and risks are tightly interlinked, because risks are central barriers for the exploitation of potential benefits. Accordingly, a ‘risk­based approach’ (Black, 2010) examines the risks and explores the opportunities and limitations to reduce them. There are various arrangements to reduce risks and increase the benefits of algorithmic selection, ranging from market mechanisms at one end, to command and control regulation by state authorities at the other (Latzer et al., 2002, 2003). In between there are several additional governance options: self-organization by individual companies; (collective) industry self-regulation; and co-regulation - regula­tory cooperation between state authorities and the industry.

19.8.1 Market Solutions: Risk Reduction Strategies by Consumers, Content Providers and Suppliers

Not all risks of algorithmic selection necessarily call for regulation. Risks may also be reduced by (voluntary) changes in the market conduct of consumers, suppliers of algo­rithmic services and by providers of the content that is processed by algorithms.

Consumers and providers of content may refrain from using problematic services, switch to other service providers or make use of technologies to protect themselves against risks. There are, for instance, technical self-help solutions for consumers in the case of censorship, bias and privacy violations, for example tools for anonymization and de-personalization of services. Content providers could avoid violations of copyright by using robots.txt files. In areas like search, recommendation and filtering, a digital arms race is observable, where market participants are trying to avoid disadvantages by using content optimization strategies (Wittel and Wu, 2004; Jansen, 2007). Insights from behavioral economics applied to the Internet of Things might help in understanding the motivations and practice of market participants for using technological design for self-help (Fleisch, 2010).

However, there are also several limitations to self-help for consumers and content providers. Algorithmic applications often work without explicit consent and opt-out possibilities, for example from state and company surveillance programs. Switching service providers requires the existence of alternative services, but several markets are highly concentrated. If there are hardly any alternative suppliers, the switching oppor­tunities are limited. For consumers, information asymmetries often make the risks of algorithmic selection barely visible, hence a direct motivation for consumer reaction is missing. Moreover, the usage of algorithmic services is mostly a low-cost situation for users, because advertising revenues finance services. The absence of costs decreases the incentives to switch to lower-risk alternatives. Finally, behavioral economics points out that even if costs and risks are detectable, consumers often do not carefully cal­culate the precise costs and benefits of their decisions (bounded rationality). Instead they rely on cognitive biases, which do not always increase their long-term benefits, for example regarding the self-protection of privacy (Acquisti and Grossklags, 2005; Brown, Chapter 12 this volume).

Suppliers of algorithmic selection services may counter risks by product innova­tions, that is, with new services or technological modifications of established ones. In such a case, the reduction of risks is part of the business strategy. There are, for example, services that aim at avoiding bias and violations of privacy and copyright in the first place. Some news aggregators’ business models integrate content providers, who receive compensation (e.g., nachrichten.de). Other algorithmic services do not collect user data (e.g., the search engine DuckDuckGo). Services such as ConsiderIt, Reflect and OpinionSpace are designed to avoid filter bubbles and bias and integrate elements of serendipity (Munson and Resnick, 2010; Schedl et al., 2012; Resnick et al., 2013).

19.8.2 Self-organization by Individual Companies

Individual suppliers of algorithmic services may reduce risks or strengthen their account­ability by means of ‘self-organization’. Typical measures are principles and standards, which reflect the public interest, internal quality assessment and ombudsmen at the cor­porate level. The commitment to self-organization is often part of a broader corporate social responsibility (CSR) strategy. From an economic point of view the purpose is to increase a company’s reputation or to avoid reputation losses.

Suppliers of algorithmic services can commit themselves to certain ‘values’ (Introna and Nissenbaum, 2000), such as search neutrality or the minimum principle for data collection, for instance (Langheinrich, 2001; Cavoukia, 2009). Ethics boards may be an option for issues with ethical implications such as software development or interferences with user experience. For risks such as censorship, discrimination, bias and manipula­tion, companies may further adopt principles and internal quality control. Qualified personnel are essential for quality assessment and conflict resolution. For big data, in­house algorithmists have been suggested to oversee big-data operations, and who would be the first points of contact for people who feel harmed by an organization’s big-data predictions (Mayer-Schonberger and Cukier, 2013).

However, several potential barriers may inhibit voluntary measures at company level. Self-organization depends on incentives, that is, benefits and costs for the company. However, the benefits of high standards of data protection (Hustinx, 2010; London Economics, 2010) and of the disclosure of the codes/algorithms may be limited. Disclosure would increase transparency, but also the danger of manipulation and imita­tion, resulting in the ‘transparency dilemma’ (Rieder, 2005; Bracha and Pasquale, 2008; Granka, 2010). The willingness for self-restrictions also depends on reputational factors. High levels of public attention on well-known companies in B2C markets may promote self-organization in the public interest. Google, for instance, runs an ethics board at the company level (Lin and Selinger, 2014). Low public awareness of companies in B2B markets, such as data brokers (e.g., Acxiom, Corelogic and Datalogix; see FTC, 2014) reduces the reputational sensitivity and therefore also the preconditions for voluntary self-organization. Finally, the suitability of self-organization depends on the type of risk. It is not suitable, for example, for reducing problems like market concentration and transformations of cognitive capabilities.

19.8.3 Self-regulation by the Industry

Self-regulation refers to collective self-restrictions of an industry in order to pursue public objectives. Typical instruments are codes of conduct, industry standards, quality seals and certification bodies, ombudsman schemes and ethics committees.

There are sectoral initiatives of self-regulation in the advertising industry (e.g., USA, Europe), the search engine market (e.g., Germany), social online networks (e.g., Europe) and in the domain of algo trading. These initiatives deal with risks such as violations of privacy and copyright, manipulation and controllability.

At the same time, there are reasons why self-regulation for algorithmic selection has not yet been comprehensively applied and suggestions as to how the conditions could be improved. Algorithmic selection is applied in a wide range of sectors. Due to the large number and the heterogeneity of the branches involved a common overall self-regulatory initiative is unlikely. In order to get a grip on fragmentation the establishment of a pro­fession of ‘algorithmists’ and special professional rules and ethics have been suggested (Meyer-Schonberger and Cukier, 2013). However, there are additional factors that inhibit self-regulation. For instance, self-regulation is more likely to occur in mature industries with like-minded market players. But some of the markets are rather new (e.g., algo­rithmic content production) and often the developers of algorithmic solutions want to challenge established players and do not voluntarily comply with older industry schemes. Minimum standards that apply to all market participants would then have to be intro­duced by statutory regulation. In particular, self-regulation is not suitable in cases where there is a sharp divergence between public and private interests, and where damage in the case of regulatory failure would be high (Latzer, 2007; Saurwein, 2011).

19.8.4 Co-regulation and State Regulation

The limitations of market mechanisms and self-regulation in reducing the risks can provide reasons and justifications for state intervention in algorithmic selection. Typical instruments of state intervention are: command and control regulation, incentives by subsidies/funding and taxes/fees, soft law and information measures.

In practice there are several examples of state influence in the domain of algorithmic selection, and regulations are related to particular risks rather than to a certain sector or a special technology. There are command and control regulations for violations of privacy and copyright, freedom of expression and fair competition. For example, in Europe, the Privacy Protection Directive (95/46/EC, Art. 15) protects people against automated individual decisions on certain personal aspects such as performance at work, creditworthiness, reliability and conduct. In the area of privacy protection the develop­ment of privacy-enhancing technologies (PETs) is funded by the EU, and some have even suggested introducing a data fee/tax in order to decrease the economic incentives for data collection (Lanier, 2013; Collin and Colin, 2013). Co-regulation has been established with the safe harbor principles and with data protection certification schemes and seals of quality. Another area of ongoing regulatory debate is search. Due to concerns regard­ing fair competition, Google was the subject of investigations by US and European competition authorities, because competitors claimed that a Google search gives undue preference to the company’s other services. Some regulatory suggestions for the search engine market aim at increased transparency and controllability by public authorities (e.g., algorithm disclosure requirements), while others propose cutting the barriers to market entry (Schulz et al., 2005). A publicly funded ‘index of the web’ (Lewandowski, 2014) or user data sets (Argenton and Prufer, 2012) are suggested as common resources in order to enhance market contestability, facilitate market entry and promote competition. Altogether, state intervention is multi-faceted in the area of data protection, and there are many suggestions for regulating searches. But state intervention does not apply to all the problems of algorithmic selection. As for risks such as bias, uncontrollability and effects on cognitive capabilities, for instance, there are hardly any measures or suggestions for state intervention via regulation. In some of these areas it might be helpful to promote consumer awareness (governance by information), enhance user media literacy and stimulate conscious usage and self-protection abilities. Since algorithmic selection also involves ethical concerns, political actors may consider the appointment of ethical com­mittees with broad stakeholder involvement to deal with conflicting values. It is evident that not all types of risk are suited to state intervention and when it comes to regulatory choice one also has to bear in mind the disadvantages of state regulation as compared to self-regulation, for example higher regulatory costs to the state, lower regulatory flex­ibility, and lower industry commitment to comply with regulations (Latzer et al., 2002; Bartle and Vass, 2005).

19.9