Abstract
Much of the history of our disciplines can be understood as successions of paired concepts—concepts that distill central intellectual tensions of each new age. One thinks of Thomas Hobbes’s juxtaposition of life in the state of nature versus citizenship under established sovereignty; or Ferdinand Tönnies’s Gemeinschaft and Gesellschaft; or Émile Durkheim’s mechanical versus organic solidarity; or Robert Park’s contrast between interactive, mutually stimulating publics versus aggregates of isolated, uncommunicating individuals. Such dualisms—many more could readily be mentioned—point both to analytical distinctions and, often, value choices that preoccupy generations of thinkers. My point here is not to cast such highly charged pairs as competitors for scarce intellectual space. All are valid and valuable so long as they capture analytical choices that continue to make a difference for the conduct of our work.
The dualism at issue here has to do with the role of computing—and of sophisticated mass manipulation of personal information more generally—in the emerging world of the future. What should our attitude be, as analysts and as advocates, toward the new structures and practices emerging from the seemingly endless computerization of one phase of life after the next? The newest alternatives have yet to acquire snappy designations like Gemeinschaft and Gesellschaft. But the themes they invoke will be intuitive to any observer with a sociological turn of mind: Where do these new directions in human relations come from? Does creation of vast repositories of personal information as bases for dealing with the people concerned reflect a new and higher wisdom on the part of the data-holding organizations? Or do the slick organizational performances realized by credit card companies, dating services, employment placement agencies, the IRS, and other authoritative bodies simply represent a triumph of expediency on the part of those organizations—perhaps at the expense of those whose lives they shape?
Considering Nexus: A Brief History of Information Networks from the Stone Age to AI, by Yuval Noah Harari, and The Ordinal Society, by Marion Fourcade and Kieran Healy, side by side offers a productive avenue for thinking through these questions. Both books are subtle enough to avoid embracing the extreme version of dualist positions. Yet both lean toward skepticism about the compatibility of what one might call computing opportunism with various longer-standing values. And, spoiler alert: both are outstanding works, learned in the references and imaginative in the scenarios that they evoke for the future. Taking their deepest ideas seriously will keep any reader from drowning in the gush of technological euphoria that has flooded many early commentaries on these subjects.
The three authors bring to their works backgrounds nearly as diverse and eclectic as their subject matter. Yuval Harari, born in Israel in 1976, achieved early eminence as a public intellectual with a global portfolio. His works have reportedly been translated into some 65 languages. French sociologist Marion Fourcade, born in 1968, is Professor of Sociology at the University of California-Berkeley. She is the author of many studies on personal information in consumer credit. Kieran Healy, born in Ireland in 1973, is widely published on matters relating to the sale of blood and bodily organs. Thus a tantalizing prospect: two outstanding works by three accomplished figures originating from three political cultures, addressing some of the most compelling issues in social change of our generation.
Anyone seeking to classify Harari’s professional profile faces serious challenges. He began academic life as a miliary historian, but his restless attention quickly carried him across disciplinary boundaries. I would be tempted to bracket him as an intellectual historian, given his attention to the role of ideology and public philosophy in the unfolding of large-scale historical change. Yet he often seems to strike a stance of agnosticism toward attempts to elevate any one model of historical causation over others.
He is above all a devotee of the Big Picture in historical analysis. Nexus, after all, at nearly five hundred pages, bears the subtitle A Brief History of Information Networks from the Stone Age to AI. The range of explanatory possibilities pursued here is accordingly wide indeed. For example:
[w]e are creating an entirely new kind of information network, without pausing to reckon with its implications. It emphasizes the shift from organic to inorganic information networks. The Roman Empire, the Catholic Church, and the U.S.S.R. all relied on carbon-based brains to process information and make decisions. The silicon-based computers that dominate the new information network function in radically different ways. For better or worse, silicon chips are free from many of the limitations that organic biochemistry imposes on carbon neurons. Silicon chips can create spies that never sleep, financiers that never forget, and despots that never die. [p. xxxi]
“Don’t sell the steak,” goes a classic dictum of the U.S. marketing industry; “sell the sizzle.” With a writer like Harari, one can always count on plenty of sizzle, and he serves it up with verve and gusto. But does he really mean that contemporary systems of surveillance and control are necessarily more severe in the regimes that they enforce because of the physical materials constituting the memory units?
I hope not.
Most of Nexus is far more closely reasoned than the passage quoted above. For example, Harari pinpoints the role of “user engagement” in the Facebook algorithms that guided news flows in the early days of Myanmar’s ethnic warfare. This variable was supposed to index the ability of Facebook coverage to keep listeners glued to their radios without stint. Reliance on it apparently led to programming directed at Buddhist audiences consisting of a steady stream of hate-filled broadcasts against the minority Rohingya. The ultimate result, according to Harari, was a poisonous climate of opinion that ultimately contributed to the deaths of between 7,000 and 25,000 Muslims, along with the expulsion of an additional 730,000 from Myanmar. Confronting numbers like these, he writes, demonstrates “how decisions made by nonhuman intelligence are already capable of shaping major historical events. We are in danger of losing control of our future. A completely new kind of information network is emerging, controlled by the decisions and goals of an alien intelligence” (p. 200).
But do we dare transfer responsibility in matters like these from those who set crucial technologies to work to the technologies themselves? Harari apparently means to take something very close to this position. Perhaps he does this to support a larger vision that he seeks to uphold—that we are in an inadvertent transition to a world dominated by amoral mechanisms that order human affairs without answering to any higher authority. But I see no reason to deny the role of human agency in this account. Approving operations like the one described above, which made the amplification and repetition of hateful messages all but predictable, surely resulted from some form of human decision-making. Blaming an algorithm strikes me as tantamount to attributing motives to entities that have no goals.
Fourcade and Healy’s The Ordinal Society differs in some obvious ways from Harari’s Nexus. It focuses more on strictly American material—particularly the evolution of the information industry in this country since the middle of the last century—and it is more explicitly sociological. But what one might term the “analytical DNA” of the two works could qualify them as siblings. Both are articulate and highly responsible reactions to thematic tensions of our times: do we accept the modes of thought and organization that appear imposed by the information technologies that we adopt, or do we approach such demands with the skepticism warranted by invading aliens?
Fortunately, neither work embraces an unqualified position of computing-as-the-moral-guide-to-the-future or computing-as-Pandora’s-box-for-the-twenty-first-century. Still, skepticism, in both books, does have the upper hand. Fourcade and Healy write, “We live in an ordinal society . . . oriented toward, justified by, and governed through measurement . . . . In domain after domain, it is changing the overall distribution of opportunity, the everyday experience of status, and the nature of economic competition. In its wake, our moral intuitions about merit and personal worth are changing too” (pp. 1–3).
Evidently Harari is not the only social scientist moved to extremes of generalization by developments like those under discussion here. No thoughtful observer of current social change could or should miss the conclusion that computerization is triggering profound rearrangements of human affairs. But acknowledging this much simply gets us started in a much more demanding project—that of assessing what concrete changes can be expected to result from these processes and what counterforces are apt to mobilize against them.
Fourcade and Healy describe ordinality as a pervasive force in contemporary culture that transforms social reality by shaping treatments of claims, and claimants for, desirable and scarce resources. In an unexpected way, their vision recalls Émile Durkheim’s conviction that basic social structures like kinship set down intellectual structures that pervade the thought-ways of entire cultures. But for these authors, the origins of the category systems in question lie in the prevailing practices for distributing claims on scarce resources—notably, in our own quantitative and calculative world, the propensity to rate human groups or populations in terms of their relative desirability, profitability, popularity, or, it would seem, virtually any other desirable (or undesirable) characteristic. Classic examples here include credit scores, qualifications for university admissions, likelihood that a tax return will show serious underreporting, or likelihood of a convict’s becoming a criminal recidivist.
Driven by quest for profit, public acclaim, or any number of more or less laudable motives, organizations ranging from dating services to credit card companies have found in computing methods for exploiting even the smallest slices of personal data as bases for invidious distinctions. Or—what may be even more disturbing—these organizations are held to have drawn this conclusion, whether warranted or not. The net result, the authors contend, is a world where consumers are constantly being pitted against one another, based on the data trails that no one can avoid leaving in a world where nearly every interaction seems to be computerized.
One can imagine, from the elaboration of these dynamics, profound and far-reaching changes in America’s consumer economy. Instead of sellers awaiting orders from buyers, organizations with things to sell—anything from airline tickets to in-home massages to psychotherapy—could be empowered to “shop” for buyers. Based on increasingly available consumer information concerning past purchases, internet browsing, financial situations, and all the other personal data now harvested in bulk from the internet, one could create ordinal hierarchies of consumers who are just at the point of readiness to make a particular sort of purchase. Starting at the top of the list of susceptibility, and relying on the most up-to-date contact information, credit status, and (who knows?) data on the consumer’s current mood swing, the marketer would have a decisive advantage.
Carried out consistently, a strategy like this could eliminate considerable costs arising from inadequate access to personal data on the consumer. The trouble, of course, is that the results would spell disaster for privacy.
One of the most penetrating discussions in The Ordinal Society focuses on recent controversies over the standings of American law schools. It goes without saying that American universities and their law schools constitute a hierarchy. But it was apparently not until 1987 that the news magazine U.S. News and World Report attempted to formalize this hierarchy in a system of ratings. The reactions of participants in this hierarchy were complex and contradictory. Publicly, spokespeople for the various institutions generally welcomed transparency into what had long been obscure forms of performance. Off the record, it seems, reactions were more complex. To oversimplify only slightly, few observers seemed ready to attest to the value of ratings as they were applied to their own institutions. Yet none of the institutions involved found it easy to withdraw from the process of being rated. In short, the U.S. News scheme proved to be one of those social processes that no player seemed to enjoy taking part in, but hardly anyone could stand to leave.
Fourcade and Healy offer detailed analyses of the conflicting motives ascribed to law school deans and other engaged parties in this still-ongoing process. To these, let me add one of my own, a friendly amendment of sorts: The very creation and existence of a formal rating system of this once unranked population created tensions and pressures that had not previously existed. With the creation of the rankings, it became untenable (though reasonable) to insist that the virtues and demerits of the institutions concerned were too diffuse to lend themselves to meaningful encapsulation in a single score. Once the rankings were a fait accompli, attempts by any interested party to minimize or qualify their significance sounded like futile excuse-making—and nearly any commentator would be judged an interested party. In short, in a quantitative, ratings-oriented world, demand for access to the rankings would be all but irresistible. If in the land of the blind the one-eyed man is king, in judgments where precision is inherently elusive, a succinct statistic even of highly contestable origins will eclipse more multifaceted forms of evaluation.
The authors of these works certainly need to be ready for many challenges to their arguments. What they probably need not worry about are complaints that the subject matter does not deserve the attention it receives here. Anyone accustomed to dealing with large organizations will have noticed changes in their tools for handling information and changes in the social structures and processes for mobilizing those tools. Reviewing the changes that have transpired in just the last half-century, the one point on which observers of all stripes must agree is that they are nowhere near having run their course. Indeed, we may still be nearer the beginning of the computer revolution than the end.
So what responses are we prepared to make to the various mind-rattling possibilities that these three authors have set before us? There is no reason why we should simply declare defeat in the face of scenarios envisaged in these two works. One legal response worthy of the challenge could lie in creation of some form of property right over commercial exploitation of data on one’s own life—a right that could also entail responsibility for negligent or destructive uses of software that one creates or controls. Such a new form of property might begin to create defensible interests in, and influence over, the fate of what is becoming one of the most consequential forms of information.