AI,the Future of Science,and the Monkey’s Paw

In his 1902 horror story “The Monkey’s Paw,” the English author W. W. Jacobs imagines a mummified monkey’s paw that grants its owner three cursed wishes that would each be fulfilled in the worst possible way. “Be careful what you wish for,” the parable’s epigraph reads, “you may get it.” Recently, scientists and technologists, wishing for the bounties of technological progress, have embraced AI and its vast possibilities. But what will we, the public, get as scientists press forward with their wish?

Academic and industry scientists have, for the most part, successfully insulated themselves from today’s AI criticism. Some avidly advocate for AI regulation, ethics, and alignment—until it negatively impacts their funding sources. Others wave off critical colleagues as overselling AI’s existential threat, arguing that unnecessary oversight could slow down innovation. Google’s chief scientist, Jeff Dean, offered an intentionally vague bromide that neatly encapsulated the mindset of industry scientists, who, by 2021, accounted for 42 percent of all U.S. PhD degree recipients: “We're continually learning to understand emerging risks while also innovating boldly.” Universities, too, are pouring millions into building their AI faculties while tightening belts across other academic units. After all, research does not stop just because vague, unspecified risks may emerge somewhere down the road.

As a computational and digital sociologist who uses AI to study AI, I have been asking how we should understand scientists’ role in an era of radical AI deregulation and pursuit of technology supremacy—what sociologist Tina Law, in an accompanying article in this issue, calls a hegemony-based approach to AI.

Science inadvertently helped surveillance companies sanitize their language while evading ethical questions about how they collected their data and how the resulting technologies would be used.

Part of the problem is a lack of public understanding about what AI scientists really do. Much of AI criticism focuses on obvious bad actors: a profit-driven tech industry, a polarizing and hard-charging White House, and an opaque, over-scaling, black-box technology. However, contrary to the belief that scientists are unfortunate victims of the commercial success of their ideas, we have played an active role in facilitating and legitimating the deregulation of AI over the past decade.

Facial recognition companies around the world have actively capitalized on their relationships with the scientific community to legitimize an industry that profits from selling human facial data and mass surveillance technologies.

A provocative recent study in Nature reveals how AI research by big tech firms and elite universities has created what Pratyusha Ria Kalluri and colleagues call the “surveillance AI pipeline.” Through an analysis of research papers and downstream patents in computer vision—a core area of AI research that enables machines to perceive objects and motion—they found that the seemingly neutral scholarship of scientists has served industry interests in two critical ways. Their research has not only become the primary intellectual source for technical ideas designed to empower mass-surveillance technology like facial recognition, but has also provided the obfuscating language that transforms the controversial use of human body parts (like the face) and motions (such as one’s gait) into more generic and harmless terms like “objects” or “moving objects.” In this way, science inadvertently helped surveillance companies sanitize their language while evading ethical questions about how they collected their data and how the resulting technologies would be used.

OPEN IN VIEWER

Tech companies around the world have actively capitalized on their relationships with the scientific community to legitimize an industry that profits from selling mass surveillance technologies.

iStockPhoto // alice-photo

In a working paper presented at the Academy of Management, I found that facial recognition companies around the world have actively capitalized on their relationships with the scientific community to legitimize an industry that profits from selling human facial data and mass-surveillance technologies. Their algorithms, they claim, have become an essential research tool—the monkey’s paw—for scientists to accelerate AI research by automating the creation of large image datasets that once required intensive human labor. Universities, in turn, have embraced the companies’ language to demonstrate that they are AI-savvy organizations. In this way, facial recognition companies successfully deflect moral responsibility and redirect public attention away from themselves as producers and promoters of mass surveillance.

AI scientists have not only inadvertently helped deregulate AI for surveillance; we have also contributed to the obfuscation of the human labor behind it, sometimes called ghost work. Sociologists have documented how the AI industry relies on the cheap labor of an army of gig and precariously employed data workers to maintain the day-to-day operations of AI systems. While gig platforms and AI companies appear to be primarily responsible for creating this new form of precarious labor, my research shows how scientists were the ones who reinvented data work as a deskilled and dispersible job even before platforms existed. To get more high-quality training data, computer scientists used their laboratories to experiment with different ways to transform data curation, once an expert task reserved for scientists, into a microtask that nearly anyone could do outside the laboratory for much less pay. To paraphrase Bruno Latour, we might say, “Give me a laboratory, and I will proletarianize the world.”

While computer scientists have historically been central to the development of AI, they are not the only ones who have become part of the surveillance state and America’s push for technological dominance. As scientific disciplines turn to big data, a new generation of computational scientists like me has emerged as an important constituency in AI deregulation. We promote the use of AI to extract information from human data. We compete for federal grants designed to maintain U.S. dominance in AI. We often teach students not to reject these technologies, but to adopt them. When scientists do not fully understand the broader implications of our role in AI, we risk losing touch with our own power and sense of right and wrong as professional knowledge creators for human society. And without a public understanding of science’s role in society, policymakers and industry leaders could easily frame calls for AI regulation as anti-science and anti-innovation—again, what Tina Law calls a hegemony-based approach—rather than as urgently needed efforts to reduce social harms.

What recommendations can a sociology of AI offer a public concerned about the rapid and still unregulated advances of AI technologies?

First, a human-centric, sociological perspective on science and technology matters more than ever in the age of AI deregulation. Sociologists have produced important research on the politics behind machine learning datasets, big data’s limited ability to address health inequality, and gender bias in image recognition systems. And even more importantly, sociology offers a counter-narrative against the dominant one that casts scientific knowledge as neutral and technological change as unregulatable. Yes, when knowledge is reduced to a tool of competition and dominance, scientists can inadvertently contribute to the commodification of your data and eventually let AI run wild. Yes, when studying human facial features carries unprecedented risk for mass surveillance, science can—and should—be bounded by ethical and regulatory constraints. And yes, it is time for the public to equip itself with sociological knowledge about AI and to fight back.

Second, scientists need the public’s support to prevent science from being further weaponized by the AI industry. Slashing public funding and forcing scientists to compete for industry money may sound like a great idea—efficient, flexible, and less burdensome on taxpayers. But as Naomi Oreskes, a historian of science at Harvard University, recently argued in Science, history suggests only “publicly funded science can be explicitly focused on public needs.” And not just through the federal government. We need closer partnerships between scientists and the public. Universities must become more transparent and accountable and do more for the public than for institutional funders. At the end of the day, it is a science funded and trusted by the public that will serve as the strongest safeguard against the danger of letting AI run wild.

Finally, if you are a student who aspires to pursue a career in the tech industry or a university administrator who aspires to educate the next generation of leaders in science and technology: take your humanities and social science courses seriously. Although many universities do require STEM majors to take general education courses and some have swiftly created new courses dedicated to AI ethics, the reality is that many students think these courses are there to hand out easy As and boost their GPAs, rather than prepare them for a future of profound technological risks. It is not just a failure to plan and coordinate between programs or to update the curriculum to reflect the fast-changing technological reality. It is also the result of students internalizing the institution’s contempt for the supposedly “less profitable” departments and “less employable” sciences. Universities can create as many ethics courses as they want, but if they continue to position themselves as suppliers of a loyal workforce for the AI industry, we will produce only the next unwitting owner of the monkey’s paw.