Sensing the human: biometric surveillance and the Japanese technology industry

Abstract

This article examines the Japanese biometrics industry and its discourse, with a focus on the language of biometric ‘sensing’ that has shaped its development over the past two decades. Rooted in the ubiquitous computing boom of the early 2000s, the language of sensing reimagines biometric technology as a mediator between the digital and the human, laying the foundation for biometric surveillance’s expansion into everyday settings such as retail ones. In these newer settings, biometric surveillance is promoted as a means for collecting data on human affect and behavior to be used for marketing and other applications. I argue that this growing ambiguity of biometric surveillance re-articulates a convergence between production and consumption, while it also informs safe society discourses and the shifting role of embodiment within digital culture.

Keywords

biometrics facial recognition Japan Japanese technology industry sensor technology surveillance ubiquitous computing

This article examines the Japanese biometrics industry and its development over the past two decades, with a focus on the language of ‘sensing’ that has come to mark much of its discourse. In doing so, it seeks to highlight a dominant force shaping biometric surveillance at present. When Amazon and other US giants declared in response to the killing of George Floyd that they would no longer supply facial recognition technology to law enforcement, press accounts noted that such promises were largely meaningless, since these companies are not the primary supplier of facial recognition systems (e.g. Pisani and O’Brien, 2020). In contrast, the less well-known Japanese electronics company NEC is estimated to be one of the largest suppliers of facial recognition technology globally, with customers ranging from police departments in the US to border security agencies around the world (Gershgorn, 2020). Notably, one of NEC’s algorithms figured in part in a 2020 incident, in which the Michigan State police wrongfully arrested Robert Julian-Borchak Williams after its facial identification system fingered him for a crime he did not commit. As Williams himself protested to the police upon his arrest, he bore little resemblance to the suspect, beside the fact that both of them were Black (Hill, 2020).

Law enforcement and security applications, however, represent just one part of NEC and other Japanese companies’ biometric offerings, which in recent years have begun to incorporate biometric surveillance in everything from customer analytics at shopping centers to employee training apps. I argue that key to this expanded vision of biometrics’ applicability has been a reappraisal of the technology that, beginning in the early 2000s, sought to frame it as a form of sensor or ‘sensing’ [senshingu] technology that might mediate between the human and the digital. This epistemological shift, I demonstrate, has allowed the Japanese technology industry to reimagine biometrics in increasingly ambitious terms, while it has catalyzed a push for the technology’s adoption across a broad range of contexts, from factories and airports to hotels and shopping centers around the world. Across each of these settings, biometrics producers tout the flexibility of their platforms, which can zero in on a single individual or collect data on broad classes of consumers for marketing and other purposes. Throughout, the same potential for racial bias and profiling seen in the Robert Julian-Borchak Williams case remains. While the pivot to the discourse of sensing might serve to obscure the racial and other biases constitutive of the biometric project, it ultimately extends, within a global context, the already prevalent forms of racialized surveillance within the everyday by automating, as Browne (2015) notes, surveillance’s underlying anti-Black and racist impulses.

Rooted in mid-20th century discourse on factory automation and applied at present to both industrial and retail settings, the language of biometric sensing in Japan reflects an intertwinement of production and consumption, whose history and present state has been well documented (e.g. Liu, 2004). The growing reach of biometric surveillance likewise mirrors the forms of data surveillance now prevalent on digital platforms, which have been documented and analyzed in examples including Zuboff (2015), Birchall (2017), Helles and Flyverbom (2019), and Langlois and Elmer (2019). As Turow (2017) demonstrates, however, present-day online data surveillance practices should be understood as extending a longer history of customer data collection rooted in the practices of brick-and-mortar retailers, which stretch back to the early 20th century. Similarly, Gekker and Hind (2020: 1417) highlight how online practices of data surveillance have begun to merge with those in physical space in what they identify as ‘infrastructural surveillance’ – a new stage of surveillance in which ‘[p]rivacy is . . . not threatened after the fact, but is constrained as a matter-of-fact’.

Much the same can be said of the new forms of biometric surveillance being promoted by Japanese technology firms and others around the world: as they spread into everyday settings from retail to street-side, they offer few chances to opt-out or interrogate the algorithms by which they sort and quantify the surveilled. Addressing this expansion of facial recognition into daily life within the context of its use on social media, Mann (2019: 41, 46) aptly notes that ‘the photographic image of the face has long been torn between its representational mode and its biometric calculability’. Mann expands upon this observation to argue that facial recognition extends the dual role of the facial image with an ultimate aim to ‘link an image to a real and embodied person’. The examples I will discuss, however, suggest a trend away from the strictly identificatory in some applications of biometrics. Namely, the ‘sensing’ style biometrics I address paint a portrait of biometric surveillance that is less concerned with identifying and disciplining the individual body, than with treating the surveilled body as a source of raw data to be quantified and processed in ways whose connection to the originary body are opaque at best.

In what follows, I begin with an examination of the intersection of biometric research and the ubiquitous computing boom in the early 2000s in Japan. This intersection, I contend, propelled biometrics’ reappraisal as a form of human sensing, setting the stage for more ambitious applications of biometric surveillance over the following decade. I then turn to recent examples of biometrics-based products and platforms being brought to market by Japanese technology companies and marketed around the world. Here, I highlight the parity, created by biometric surveillance, between sites of production and consumption: functionalized by the same or similar technologies, biometric surveillance in both industrial and retail settings reveal a unifying ‘safe society’ ideology, which reflects both biometrics’ security heritage as well as sensing’s discursive roots in ubiquitous computing and, before that, in cybernetics. Finally, I conclude by considering the emergent impacts of biometric surveillance under the ambiguous reach of sensing, as well as its reinforcement of long-standing biases and norms.

Although I will focus on the case of the Japanese technology industry, many of the features of biometric surveillance that I discuss can be seen elsewhere. In Canada, for example, the mall operator Cadillac Fairview recently came under fire for surreptitiously installing facial surveillance in kiosks to collect data on visitors, mirroring a use of facial recognition which will be seen in the Japanese examples below (Rieger, 2018). Importantly, by raising the latter, it is expressly not my objective to perpetuate the techno-orientalism which frequently accompanies discussions of contemporary Japan and East Asia. Indeed, the story of facial recognition and biometrics is a global one, given that the technologies have historically been developed across national boundaries. In my discussion of the Japanese context, I aim to instead foreground their transnational aspect and thus decenter an all-too-familiar focus on the US technology industry.

Biometrics and the ubiquitous computing boom

In 2013, Japan’s National Institute of Information and Communication Technologies (NICT) announced plans to test an experimental facial recognition system at the JR Osaka rail station in April of the following year. Under the plan, the NICT would deploy 92 video cameras throughout the station and the connected Osaka Station City buildings – a vast complex including department stores, as well as a hotel and fitness center. Using facial recognition, the system would track the movement of over 82,000 daily visitors, to be used in potential disaster response scenarios, such as earthquakes, fires, and flooding (National Institute of Information and Communication Technologies, 2013). The trial never took place, at least on the scale originally proposed. As summarized by the NICT’s own review of the incident, entities including the press, civic groups, and the Osaka city government expressed concerns regarding privacy rights and the potential for data leaks (Eizōsensā shiyō daikibojisshōjikken kentō iinkai, 2014: 4). The NICT eventually relented and agreed to redesign the trial to monitor only during nighttime hours and in restricted areas accessed by station employees whose permission would be secured beforehand (Kiyoshima, 2014).

Despite this failure to launch, the trial highlights the pervasiveness of the language of sensing within Japanese biometrics discourse. Throughout NICT’s (2013) official announcement of the test, words such as ‘surveillance camera’ or even simply ‘camera’ were absent, and in their place, the organization used the more anodyne term ‘image sensors’ (eizō sensā) to describe the 92 cameras that would be used during the trial. This substitution of ‘sensor’ for ‘camera’ might seem a ham-handed attempt to preempt privacy concerns. However, the NICT was not alone in its use of the term ‘image sensor’. Foreshadowing the NICT trial, a 2009 article on facial recognition and privacy in the Journal of the Japanese Society for Artificial Intelligence similarly argued that it would be more productive to approach modern digital cameras as a type of sensor, since they constitute ‘optical signal receptors’ (kōgaku shingō juyōki) akin to infrared motion detectors that one might encounter in a public bathroom (Kakusho et al., 2009: 196–197). At present, the Japanese technology industry continues to echo this language. The electronics company OMRON (2020), for example, markets its facial recognition platform OKAO Vision – a platform it promotes for a variety of uses from marketing and sales analysis to smart TVs – as an ‘image sensing technology’ [gazō senshingu gijutsu].

This language of sensing resonates with a broader discourse on biometrics within the Japanese technology industry that frames the technology as a mediator between the human and the digital. This discourse is exemplified by NEC, whose motto declares that it is ‘orchestrating a brighter world’ and whose discussions of biometrics emphasizes lofty goals which encompass and extend well beyond security. An introductory blurb on the Japanese-language website for its biometrics research unit notes that the company has ‘led the field of biometrics for over 40 years’, and that its fingerprint, facial, and iris recognition technologies have a range of possible applications that include ‘promoting health, improving workplace efficiency and even marketing’. As the website summarizes the field: ‘Because biometrics is a technology that digitizes the information of a person’s exterior and interior, it is ultimately a technology that connects to an “understanding of people”’ (NEC, 2019a).

The notion of ‘digitizing’ the human has been a prominent one in Japanese technology discourse for some time. Notably, the first decade of the 21st century in Japan witnessed a flurry of interest in ubiquitous computing – a paradigm focused on embedding computing within everyday spaces and which, in the Japanese context, catalyzed interest in biometrics as a sensing technology that could translate the human into the digital. A term and concept originally coined by Mark Weiser at the Xerox PARC lab in the late 1980s, ubiquitous computing prophesied a new stage of digital technologies in which they would become so widespread and pervasive in daily life that they would become practically invisible there, much like writing or printing (Weiser, 1991). Understood in Japan to be a precursor of the Internet of Things, ubiquitous computing found particularly fertile ground there during the 2000s, as academic researchers, corporations and government bureaucrats perceived in the paradigm a new lifeline for Japanese industry (Kanasugi, 2004; Koshizuka, 2016: 1024–1025). Reflecting this interest, the Japanese government rolled out its u-Japan initiative in 2005 as a successor to the earlier e-Japan broadband expansion strategy (Ministry of Internal Affairs and Communications, 2005).

Japanese investment in biometrics research stretches back to the 1970s. NEC included a facial detection demonstration at the 1970 Osaka Expo (Gates, 2011: 25–26), and in the 1980s, it began providing automated fingerprint recognition systems to clients around the world, including police departments in the United States (NEC, 2020a). Although I focus on ubiquitous computing’s intersection with biometrics in Japan, it is important to note that the industry’s development from the 2000 onwards built upon these pre-existing strengths. Furthermore, renewed attention in Japan to biometrics during the first decade of the new century mirrored a phenomenon playing out around the world during these years. Against the backdrop of the US-led war on terror, biometrics rose to the fore around the world as an up-and-coming technology that could aid in border security and other counterterrorism efforts. Although these were not a primary focus of Japanese biometrics during these years, many researchers appeared aware of the significance of the US context, likely reflecting the nature of Japan’s technology industry as an export-focused one (Seto, 2004: 34, 2014: 77).

Yet, even when verification and authentication represented a major focus of biometrics research in the early 2000s in Japan, interest in ubiquitous computing remained a dominant influence. Namely, numerous researchers recognized that ubiquitous computing meant the ubiquitous flow of data, and so they perceived biometric technologies as prime candidates for securing access to personal data (Seto, 2014: 78). A 2002 article in Hitachi’s inhouse journal Hitachi hyōron noted, for example, that in a ‘ubiquitous information society’, a variety of information including ‘personal information, credit card information, and paid content’ flows across networks, requiring ways to authenticate users so as to verify their access rights to that information. One solution the article proposed was a finger vein recognition system under development at Hitachi (Kinoshita et al., 2002: 8). (The following year, Hitachi applied for a patent on the technology (Miura et al., 2009)). Similarly, a 2003 article in Fujitsu’s technical journal on the company’s biometrics work focused on the technologies’ use in authentication applications. The article noted that fingerprint and palm reading systems offered the most accurate mode of verification; it speculated, however, that facial and voice recognition then under development might soon offer a more ‘natural’ means of verifying an individual’s identity, often unbeknownst to the individual (Mori et al., 2003).

Biometrics’ security potential furthermore complemented an underlying safe society ideology that ran throughout much of the fanfare surrounding ubiquitous computing. In addition to hopes that it would revive Japan’s economy, a significant portion of the government and industry rhetoric on ubiquitous computing lauded its potential to realize a 21st Japanese society focused on safety, peace-of-mind, and comfort/convenience (qualities frequently represented in Japanese, respectively, as anzen, anshin, and kaiteki) – rhetoric that continues to the present in technology discourse in Japan, as evidenced by the Japanese government’s current Society 5.0 initiative, which inherits many of the priorities and ideological features of uJapan before it (e.g. Cabinet Office, 2020). As biometric technologies became more widely available and efficient at the turn of the century, they provided a good fit to this broader safe society theme: they would not only secure individuals’ data; by extension, they would secure a coming ubiquitous future by serving as checks against possibilities, such as data leaks and abuse, that might undermine it.

Yet, even when accounts focused on biometrics’ security role in the early 2000s, the heady language of sensing and understanding the human was already apparent. Andō (2003) – a researcher at the University of Tokyo and frequent commentator on ubiquitous computing – noted that with the spread of on-demand services, the era of ubiquitous computing had already begun to arrive and that, for these and other future services to succeed, ‘technology will be required that can identify [the user and the service provider] in the same way two old friends recognize each other when meeting face-to-face. . .’ That technology, he continued, was biometrics, since they could identify and determine individual access rights through the use of ‘information specific to each individual human, such as fingerprints, faces, signatures and DNA’. He concluded: ‘Biometrics are in fact the foundational technology of a safe and vibrant network society’ (pp. 266–267).

Andō’s discussion of biometric technology emphasized its role as a bridge between the digital and the human – a view of biometrics resonant with the broader understanding of ubiquitous computing in Japan, as well as Weiser’s own definition. For Weiser, ubiquitous computing offered the antithesis of paradigms such as virtual reality, which immersed the user in the world and idiom of the computer. For ubiquitous computing to make itself invisible, it would instead have to translate computing into a human idiom and make its use nearly effortless (Weiser, 1991: 94). As Japanese researchers understood this imperative, ubiquitous computing’s focus on everyday environments predicated a central role for sensor technology in general. Under this vision of ubiquitous computing, networked arrays of sensors embedded throughout public and private spaces would gather information on humans and human environments to be digitized and thus made legible to computers. As a 2005 Hitachi hyōron article put it, the networks and the sensor nodes that formed them (and more specifically Hitachi-produced sensor nodes) would provide the ‘interface with the world of reality’ by ‘collect[ing] in real time information on the environment, things and people’ (Ishizaki et al., 2005: 628). Similarly, a 2006 white paper on information and communication technologies (ICT), published by Japan’s Ministry of Internal Affairs and Communications, accorded to sensors and sensor networks a central role in realizing ubiquitous computing’s promise of a safe society. The white paper noted that the Ministry was actively supporting sensor network research, with the expectation that sensors would bolster ICT’s ability to supplement ‘a variety of societal and economic activities such as medicine and social welfare, crime prevention and security, disaster prevention, and the response to environmental risks’ given their ability to ‘recognize . . . the circumstances of people and things as well as their surrounding environment’ (Ministry of Internal Affairs and Communications, 2006: 221–222).

If such discussions of ubiquitous computing leading to a safe and well managed society appear vaguely reminiscent of mid-20th century cybernetic discourse – particularly of Norbert Wiener’s more grandiloquent discussions of the cybernetic paradigm – this is no coincidence. Not a few Japanese researchers and commentators placed their utopic hopes for ubiquitous computing squarely within a history of industrial technology, in which sensing technology played a key role. Andō (2003) framed biometrics and other contemporary sensor technologies as rooted in a history of factory automation that led the way in Japan’s postwar industrial expansion. Sensors, he noted, represented a ‘key technology’ that had ‘strengthened the competitiveness of postwar Japanese industrial products’ and subsequently led to improved industrial safety and product quality. Sensor technology had, until recently, been limited to the controlled environment of the factory, but the coming revolution in sensor technology, he predicted, would be led by sensors embedded in the less controlled environments of everyday life. This next generation of sensors would be ones that people ‘carry with them or attach to their bodies’ or that would be distributed throughout everyday spaces and even be capable of autonomous movement. Notably though, the benefits of the new sensor technologies would echo those that had been limited to the factory floor. According to Ando, they would ‘make both industry and society prosperous, increasing safety and convenience. . .’ (p. 263) – a safety-based equivalence between factory and non-factory that, as will be seen, continues to animate present-day applications of biometric surveillance.

Other 2000s era accounts of biometrics and ubiquitous computing drew similar lineages, connecting contemporary sensor technology and its possibilities to a technological heritage that extended back to mid-20th century automation. Ogata (2004) – a researcher at OMRON – traced modern sensor technology through three stages beginning with factory automation in the 1950s, continuing with the widespread adoption of information technology in the 1960s and 1970s, and extending through the rise of ubiquitous computing at the beginning of the 21st century. He argued that with early industrial automation, factory sensors focused primarily on ‘things’ through the monitoring of physical properties such as temperature and pressure. The information technology revolution began to introduce sensors into spaces outside of the factory, but these sensors remained focused on things and information, such as traffic on public roads. Ubiquitous computing, according to Ogata, had introduced a new stage of sensing technology in which it had begun to pervade spaces from the public to the private and ‘take as its object “the person.”’ Facial recognition in particular assumed a privileged position in Ogata’s description of this latest stage, as it would be able to identify people, estimate age and sex, discern facial expressions, and so forth (pp. 452–454).

Although neither Andō nor Ogata directly cite Wiener, their placement of sensors in a timeline extending back to postwar automation appeared positioned to do so implicitly. The cyberneticist had been well received in Japan during the 1950s and 60s, and his theories had been influential not only in the country’s drive toward automation but also amongst intellectuals who saw in his feedback-based control mechanisms a key to a better managed society (e.g. Asahi Shinbun, 1956). It is therefore worthwhile in this context to note the role that sensors played in Wiener’s cybernetics. Wiener (1954) argued that ‘sense organs’, based on already extant instruments such as photoelectric cells and thermometers, would be essential in the coming age of factory automation, since they would provide the feedback data for ‘computing machines’ which would in turn direct adjustments via effectors, extending the role feedback mechanisms had played in industrial production since the 18th-century Watt steam engine and its centrifugal governor. The main improvement that the new ‘sense organs’ would add to these older feedback mechanisms was in its digitization of feedback data. Whereas the centrifugal governor throttled steam being let into the engine valves via a mechanical link between the rotation of the governor and the valve it would close as it spun faster, the updated sensor equipment of the automated factory would translate readings into ‘a pattern of consecutive digits’, allowing for more complex calculations and feedback mechanisms (pp. 152, 156–157).

As I will suggest in the remainder of this article, the convergence between ubiquitous-based biometrics research and a longer history of industrial technology in Japan offers more than mere historical footnote. Instead, it highlights how biometrics’ subsequent foray into the ‘real world’ of everyday, human environments extended and adapted, rather than broke from, that industrial tradition, particularly as they functioned to create an equivalence between products and the human agents that used and consumed them. Notably, numerous researchers and commentators on ubiquitous computing characterized biometric sensors, both implicitly and explicitly, as a complementary technology to the RFID (radio frequency identification) that represented a primary research focus of 2000s ubiquitous computing work. Within this expanded vision of biometrics’ role, they would mirror RFID by identifying and tracking human agents in the same manner RFID technologies were being developed to track products from factory floor to site of consumption (Hashimoto and Yamamoto, 2006: 278–279). For example, Seto (2004) argued that, for ubiquitous computing to succeed, a central problem would be how ‘cyberspace distinguishes between things and people’. According to Seto, the answer was clear cut: in their merger with ‘real space’, ubiquitous systems would use RFID to distinguish and identify objects, while they would use biometrics to identify the human agents that used them (p. 535).

More broadly, numerous accounts discerned a parallel between RFID and biometrics’ roles as mediators between human environments and digital systems. In a 2004 discussion of developments in facial recognition the technology magazine Computer & Network LAN noted that ‘. . .if RFID could be said to be the bridge from the digital world to the real world, automated recognition of facial information could be said to be the bridge from the real world to the digital’ (Kuramasu, 2004: 8). As an example of this, the article detailed the work of the roboticist Matsumoto Yoshio who developed a system during the early 2000s that could extrapolate individual line-of-sight and facial expression, using stereo cameras to extract three-dimensional facial feature information, such as the position of eye sclera and mouth edges (see also Ido et al., 2006). Such research anticipated the developments of the past decade, in which Japanese companies and others have moved to incorporate, within their products, biometrics’ increasingly granular capabilities to monitor human agents.

Monetizing biometric sensing

In their characterization of biometrics as a new stage of sensor technology that would mediate between the human and the digital, accounts from the early 2000s foreshadowed how facial recognition and other biometric technologies would evolve over the next two decades. During this time, biometrics research and product development in Japan has intersected with newer trends in computing, while advances in deep learning have brought substantial improvements to the field of computer vision in general. Even before AlexNet’s win in the 2012 ImageNet contest – an event that, as in North America, led to interest across Japan’s computer vision community in deep learning (Okatani, 2014) – facial recognition’s ability to discern facial expression and attention through line-of-sight had improved substantially. Similarly, capabilities such as whole-body detection had already begun to make possible the analysis of movement and gesture (Kawade et al., 2011). Since then, facial and other image recognition technology’s integration within surveillance camera systems have begun to realize the Japanese ubiquitous computing community’s vision of an everyday environment saturated with smart sensing devices – a ubiquitous sensing network that provides massive amounts of data to further train deep learning systems while dovetailing with Japanese companies’ IoT and Smart City products. Reminiscent of the utopic aspirations of ubiquitous computing (as well as cybernetics before it), these latter products seek to manage urban spaces in part through the data provided by biometrics and other forms of surveillance.

Against this backdrop, facial recognition and other biometric technologies’ security applications remain a focus for Japanese companies. NEC has played a leading role here, having developed and brought to market numerous applications for its biometrics technologies through a coterie of brands, including its NeoFace facial recognition platform and BIO-IDiom biometric authentication suite. As the company’s official PR would have it, the sky is the limit when it comes to biometric surveillance. The same NEC facial recognition technology used in passport checkpoints in airports across Japan and around the world, including New York’s JFK, has also been used at concert and event venues to verify ticket holders’ identity (Hayase and Tsukahara, 2019: 56–57; Iwaki, 2014; NEC, 2016). At an airport hotel in New Delhi, NEC facial recognition systems stay on the lookout for individuals on criminal watchlists, at the same time that they alert staff of the arrival of VIP guests, demonstrating, according to an article in the company’s NEC Technical Journal, how NEC’s facial recognition technology can be used to both ‘ensure guest safety and improve customer service’ (Imaoka, 2016: 37). In partnership with the convenience stores chains 7-Eleven and Lawson, the company has also begun testing unstaffed and low-staffed pilot stores where facial recognition assists in contact-free shopping (NEC, 2019b, 2020b).

NEC has furthermore found numerous ways to monetize its promise to sense and decode the human interiority through biometrics, particularly within the context of retail. According to company literature, NeoFace-enabled cameras placed at store entrances and near digital advertising can be used to collect data, such as the age and gender of customers and passersby – characteristics which facial recognition algorithms determine by the analysis of features including bone structure and wrinkling (Kawade et al., 2011: 1536). Facial recognition systems can also collect data on customer behavior at sales registers, as can customers’ line-of-sight and flow patterns throughout the store. Furthermore, workers’ and customers’ ‘degree of smiling’ can be quantified, the company and its research suggest, so as to provide data on stores’ customer service and satisfaction (Kuroda et al., 2019: 47–48; NEC, 2021; Sakamoto and Takaya, 2014: 56–57). Notably and of perhaps greatest concern, NEC claims that similar expression and gesture recognition capabilities can be deployed to automatically profile and track individuals about to engage in criminal activity using machine learning based detection of ‘suspicious behavior’, such as loitering (Liu and Nishimura, 2019).

NEC is not alone in these efforts. Numerous companies offer smaller, more limited applications and devices: NTT and the startup Earth Eyes, for example, market an ‘AI Guardman’ camera system that purports, in echoes of NEC, to detect potential shoplifters by automatically detecting suspicious behavior (NTT Gijutsu janaru, 2018). Other companies likewise integrate biometric surveillance into larger platforms. In 2016, Fujitsu rolled out its ‘GREENAGES Citywide Surveillance’ suite – a platform that, similarly marketed worldwide, combines cloud computing, deep learning, facial recognition, and other biometrics-based surveillance. Now in its third iteration, GREENAGES provides a broad range of services from security to marketing applications. According to Fujitsu, it can detect unauthorized access to restricted areas within buildings and other facilities, while at the same time it can monitor and manage customer flow patterns in shopping malls and restaurants. Fujitsu (2020a) furthermore boasts that their AI and facial recognition technology can analyze the success or failure of digital signage in public spaces by measuring crowds’ attention and even breaking down the demographic composition of a given crowd. The company has likewise pushed GREENAGES’ applicability during the COVID-19 pandemic and recently announced a partnership with the city of Kawasaki to test the platform at evacuation facilities where it might monitor and control crowding (Fujitsu, 2020b).

Japanese companies’ deployment of biometric surveillance across multiple product lines throws into relief a common focus on the movement, interactions and attitudes of bodies – a focus that extends back to biometrics’ earlier intersection with ubiquitous computing in Japan and the parallels that researchers drew between biometric sensing and RFID. In the context of recent applications of biometrics, this focus seeks to monitor movement on an aggregate scale by collecting data through what is often referred to in the companies’ literature as jinryū kaiseiki, or ‘human flow analysis’. The companies propose to use the data to intervene in those flows and manage them. In an aim consistent with the companies’ various Smart City strategies, they propose that the data be used in the design of more efficient building and store layouts, rationalizing transit schedules, and so forth. In an echo of earlier cybernetics, many of these schemes visualize the loop between biometric data and intervention in the terms of a feedback loop, in which smart systems nudge human actors toward certain behaviors through the remodeling of environmental cues and even the presentation of feedback data.

A prime example here is Hitachi’s Ekishi-Vision, a flow visualization program that the company developed for the Tokyo-area Tōkyū Railways. Available within Tokyu’s smartphone app, Ekishi allows users to view real-time images of crowding at the rail line’s ticket gates. These images are created via body recognition algorithms that detect the number and position of bodies and then convert them into simplified icons which are superimposed on previously captured still images. Hitachi’s (2018) stated aspiration for the product is to alleviate crowding on public transportation, by encouraging users of the app to take trains at less congested times. Transit notably represents a major target for biometrics and other smart city applications: In late 2019, Osaka Metro began testing facial recognition-based ticket gates by vendors including OMRON and Toshiba, with an aim of rolling out fully functional gates across its network by 2024 (Satō, 2019). In these and other examples, industry and bureaucratic literature justifies the use of biometric surveillance as a means to rationalize human flows and promote seamless and convenient interactions with technology.

The evolution of safety and security as primary watchwords across Japanese companies’ biometrics-enabled platforms is furthermore striking. Here, Hitachi is once again instructive, as it boasts that its consumer and civilian-oriented surveillance technologies can improve quality of life while providing the foundation of a safe, ‘peace-of-mind’ society (e.g. Fukuda et al., 2020; Iwasaki and Nakamura, 2020). Notably, such characterizations resonate with earlier ones, seen in the context of ubiquitous computing, that biometrics would not only secure a safe version of the ubiquitous society, they would provide a cognate role to the factory-floor sensors, which improved industrial safety and quality control in the context of mid-20th century factory automation.

Nor should it be overlooked that the same technologies, which Hitachi and others propose to promote a safe society, likewise still find application on the factory floor. At its Omika Factory in Ibaraki Prefecture, for example, Hitachi utilizes its image recognition technology alongside RFID tags to provide, as the company website boasts, ‘a commanding view of the movement of “people” and “things.”’ In a feedback loop depicted on the website as a virtuous circle of ‘sense’, ‘think’, and ‘act’, managers can efficiently monitor the flow of numerous workers and products throughout the production line; hidden bottlenecks, Hitachi (2020) furthermore suggests, can be pinpointed and fixed. Indeed, the language of security often goes hand-in-hand with flow monitoring and management in the context of such official literature, highlighting the reach of the discourse and its associated platforms from production to consumption.

Sensing between discipline and control

From point of production to point of purchase, facial and body recognition algorithms stand to become increasingly pervasive, as they attend to individual bodies at the same time that they amass data on crowds and classes of individuals. As they do so, they pivot from singling out worker error and criminal intent to aggregating consumer data and visitor flow patterns. It is significant in this context that Robert Julian-Borchak Williams – the man misidentified in part by an NEC algorithm – was arrested in connection with a shoplifting incident. Notably, the facial recognition algorithm was run on surveillance after the fact rather than as a result of active monitoring (i.e., the cameras in question appear to have not been directly equipped with facial recognition) (Hill, 2020). However, as more active biometric surveillance is adopted in retail settings, incidents like the Williams case can be expected to proliferate.

Building upon Frantz Fanon’s notion of epidermalization, Browne (2015: 110–115) identifies in biometric surveillance’s inherent biases a new stage of ‘digital epidermalization’. Browne argues that the digitalization of older processes of epidermalization automates societal surveillance of the racialized body, representing ‘[an] exercise of power cast by the disembodied gaze of certain surveillance technologies . . . that can be employed to do the work of alienating the subject by producing a truth about the racial body and one’s identity . . . despite the subject’s claims’. Indeed, Browne’s digital epidermalization can be seen to inform facial recognition across numerous stages of surveillance and categorization. Notably, the technology’s age and gender recognition capabilities discussed earlier operate at a level that is barely skin deep, based as they are on features such as wrinkling and bone structure that ignore the social and cultural forces that establish and maintain these demographic categories.

The emergent fields of emotion and behavior recognition, based on facial and body recognition, raise similar concerns. Products like NTT and Earth Eyes’ AI Guardman, for example, propose to simplistically distinguish customer from criminal based on assumptions regarding the expressions and behaviors appropriate to either class. Turow (2017: 3–7) highlights how retailers’ data collection practices sort customers based on demographic and social background, leading to gaps in the service they can expect to receive in-store. One can expect predictive biometric surveillance to only aggravate these structural inequities. As the Coalition for Critical Technology (2020) wrote in an open letter to Springer Publishing in protest of a paper that claimed to have designed a deep learning network that might predict criminality, the automation of profiling will likely serve more readily to funnel individuals, particular people of color, into categories of criminality based on pre-existing biases baked into systems. The language of sensing, despite its seemingly anodyne nature, works in reality to buffer and obfuscate these very real dangers presented by biometric surveillance.

In a longer history of modern practices of identification, Caplan (2001) notes that while a ‘standardized system of identification’ arose in the 19th century as a means of providing stability to modern nation-states in their management of expanding populations, it was always undermined by a foundational instability, since ‘even in its most controlling and technologized forms it is based on a concept that is itself difficult to stabilize and control’. Namely, identity in its modern sense always carries a ‘dual meaning’ in which it denotes both an identity with oneself but also identity with a group or class to which one is understood to belong (pp. 50–51). Responding to Caplan, Gates (2011) suggests that facial recognition and other biometric technologies might be understood to extend rather than ameliorate this underlying instability, since they have accompanied and catalyzed an explosion of data that is at present recognized to comprise any one individual’s identity. ‘Rather than stabilizing identity’, Gates writes, ‘the proliferation of [data] is making it messier than ever, in turn leading to the perpetual pursuit of new efforts at stabilization’ (p. 17).

Gates’s insights into the growing messiness of data-based identity are apt, as are her observations on biometric surveillance’s role therein. However, one might also speculate that biometrics’ pre-existing biases will catalyze a further tilt toward the broader class-based forms of identification laid out within Caplan’s ‘dual meaning’ of identity, while their recent applications suggest at the same time a blurring of the two poles. On one hand, biometric surveillance reveals a Taylorist focus on the individual body in its attention to and management of worker movement and even affect; on the other, it reveals Post-Taylorist tendencies in its drive to quantify and monetize customer behavior and mood, thus shifting away from the individual body and identity and toward a macroscopic treatment of bodies en masse. When biometrics aid in the regulation of worker movement, they provide glimpses of the factory-floor style surveillance, which Deleuze (1992) declared defunct in his description of an ascendant control society (p. 5). Yet, when they distill the image of bodies in physical space into icons as they do in the case of Hitachi’s Ekishi-Vision, they seem to confirm that the systems of control have prevailed, concerning themselves more with the modulation of data’s flow rather than the discipline of the individual bodies that constitute it.

Ultimately, however, the emergent paradigm of biometric surveillance and the logic of sensing that informs it in the Japanese case confound such easy categorizations, because the body that it surveils does not conform neatly to its more familiar roles within earlier forms of surveillance. In a discussion of biometrics at the beginning of the 21st century, Lyon (2001) noted that the new technologies had precipitated a curious if somewhat counterintuitive turn, rendering the body ‘once again, a source as well as a site of surveillance’ (p. 291). Indeed, biometric surveillance reasserts the centrality of the body. However, in a manner that Lyon could perhaps not conceive at the time, biometrics’ expansion into the ambiguous realm of sensing, as well as their association with computing paradigms like deep learning have caused it to alter the relationship of surveillance to the body: at the same time that biometric technologies monitor the body, they also loosen its indexicality. If facial recognition cameras are to be understood as image sensors and optical signal receptors, the connection between the signals they receive and the bodies from which they emanate shift. No longer traces that simply point back to the presence of a unique body, they become data to be aggregated and reassembled in novel visualizations and representations. Under this emergent regime of biometric surveillance, the individual body is not a primary focus of a panopticon-like, isolating gaze in the Foucauldian sense. However, the body is also not simply lost within a flight to disembodied code in the Deleuzian sense. Contra Deleuze’s prophecy that the passcode would surmount individual identity, the body itself becomes the source of codes within biometrics – codes to be sensed and deciphered before they are processed into new ones.

Supplemental Material

sj-pdf-1-mcs-10.1177_01634437211036996 – Supplemental material for Sensing the human: biometric surveillance and the Japanese technology industry

Supplemental material, sj-pdf-1-mcs-10.1177_01634437211036996 for Sensing the human: biometric surveillance and the Japanese technology industry by David Humphrey in Media, Culture & Society

Footnotes

Funding

The author received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

David Humphrey

References

Andō

(2003) Yubikitasujōhōkankyō ni okeru senshingugijutsu kobetsu no chinōka kara kettowākuka to chishikikyōyū no senshingu gijutsu e. The Transactions of the Institute of Electrical Engineers of Japan 123(8): 263–270.

Asahi Shinbun (1956) Wiinaa hakase kaikenki – Saibanetikkusuteki bunmeiron o kiku. Asahi Shinbun, 13 April, Tokyo evening edition.

Birchall

(2017) Shareveillance: The Dangers of Openly Sharing and Covertly Collecting Data. Minneapolis, MN: University of Minnesota Press.

Browne

(2015) Dark Matters: On the Surveillance of Blackness. Durham, NC: Duke University Press.

Cabinet Office (2020) Society 5.0. Available at: https://www8.cao.go.jp/cstp/society5_0/ (accessed 10 November 2020).

Caplan

(2001) ‘This or that particular person’: Protocols of identification in nineteenth-century Europe. In: Caplan

Torpey (eds) Documenting Individual Identity: The Development of State Practices in the Modern World. Princeton, NJ: Princeton University Press, pp.49–66.

Coalition for Critical Technology (2020) Abolish the #TechToPrisonPipeline. Available at: https://medium.com/@CoalitionForCriticalTechnology/abolish-the-techtoprisonpipeline-9b5b14366b16 (accessed 11 November 2020).

Deleuze

(1992) Postscript on the societies of control. October 59: 3–7.

Eizōsensā shiyō daikibojisshōjikken kentō iinkai (2014) Chōsahōkokusho. Report. 20 October. Available at: https://www.nict.go.jp/nrh/iinkai/report.pdf (accessed 12 November 2020).

10.

Fujitsu (2020a) AI gazōbunseki soryūshon ‘GREENAGES Citywide Surveillance V3’ o hanbaikaishi. Fujitsu, Press release, 12 February. Available at: https://pr.fujitsu.com/jp/news/2020/02/12.html (accessed 29 September 2020).

11.

Fujitsu (2020b) AI o katsuyō shi 3 mitsu risuku o teigen suru shingata koronauirusu wazawai de no hinanjo un’ei no jisshōjikken o jisshi. Fujitsu, Press release, 24 August. Available at: https://pr.fujitsu.com/jp/news/2020/08/24.html (accessed 29 September 2020).

12.

Fukuda

Murakami

Ōkubo

, et al. (2020) Anzen – anshin na shakai no kōchiku ni kōken suru AI gijutsu o mochiita eizōkaiseki soryūshon. Hitachi hyōron 102(3): 400–406.

13.

Gates

(2011) Our Biometric Future: Facial Recognition Technology and the Culture of Surveillance. New York, NY: New York University Press.

14.

Gekker

Hind

(2020) Infrastructural surveillance. New Media & Society 22(8): 1414–1436.

15.

Gershgorn

(2020) Carnival cruises, delta, and 70 countries use a facial recognition company you’ve never heard of. OneZero, 18 February. Available at: https://onezero.medium.com/nec-is-the-most-important-facial-recognition-company-youve-never-heard-of-12381d530510 (accessed 25 June 2020).

16.

Hashimoto

Yamamoto

(2006) Yubikitasushakai o sasaeru RFID tagu kanrenkiki. Fujitsu 57(3): 274–279.

17.

Hayase

Tsukahara

(2019) Baiometorikusu o katsuyō shita hinichijōkūkantaikenkōjō e no torikumi. NEC Technical Journal 71(2): 55–58.

18.

Helles

Flyverbom

(2019) Meshes of surveillance, prediction, and infrastructure: On the cultural and commercial consequences of digital platforms. Surveillance & Society 17(1/2): 34–39.

19.

Hill

(2020) Wrongfully accused by an algorithm. New York Times, 24 June.

20.

Hitachi (2018) Tōkyūdentetsu to Hitachi no kyōsō de, ‘yūzāshiten no sābisu kaihatsu’ o. Hitachi. Available at: https://www.hitachi.co.jp/products/it/lumada/stories/20180316_tokyu/ (accessed: 21 September 2020).

21.

Hitachi (2020) Seisangenba no zentai saitekika. Hitachi. Available at: https://www.hitachi.co.jp/products/it/lumada/cs/00008/index.html (accessed 23 September 2020).

22.

Ido

Matsumoto

Ogasawara

, et al. (2006) Riarutaimu kao – shisenkeisoku shisutemu no kaihatsu to chiteki intafēsu e no ōyō. Jōhōshorigakkai ronbunshi konpyūtabijon to imēji media 47(15): 10–21.

23.

Imaoka

(2016) Riarutaimu kanshi o jitsugen suru dōga kaoninshō gijutsu. NEC Technical Journal 69(1): 34–37.

24.

Ishizaki

Nishizawa

Ōta

, et al. (2005) Yubikitasujōhōshakai no gugenka ni muketa gijutsukaihatsu e no torikumi. Hitachi hyōron 87(7): 627–632.

25.

Iwaki

(2014) Momokuro konsāto ni ‘honninkakuninshisutemu’. ITMedia entāpuraizu, 8 December. Available at: https://www.itmedia.co.jp/enterprise/articles/1412/08/news132.html (accessed 11 November 2020).

26.

Iwasaki

Nakamura

(2020) Shakai shisutemu no jizokutekina kaihatsu to unyō o sasaeru Lumada no purattofōmu. Hitachi hyōron 102(3): 335–340.

27.

Kakusho

Minoh

Mitsugami

(2009) Kamera eizō ni okeru puraibashi taisho no tame no appurōchi. Journal of the Japanese Society for Artificial Intelligence 24(2): 196–201.

28.

Kanasugi

(2004) Yubikitasu shakai ni mukete. The Journal of the Institute of Electrical Engineers of Japan 124(2): 71.

29.

Kawade

Mochimaru

Morishima

(2011) Jisseikai ni manabu eizōgijutsu ~Genjitsu to tenbō~: Kao – jintai media ga hiraku shinsangyō no gazōgijutsu. The Journal of the Institute of Image Information and Television Engineers 65(11): 1534–1544.

30.

Kinoshita

Owashi

Sagawa

, et al. (2002) Yubikitasujōhōshakai ga sōzō suru yutaka na shinseikatsu. Hitachi hyōron 84(11): 5–8.

31.

Kiyoshima

(2014) JR Ōsaka biru no kanshi kamera o tsukatta kaoshikbetsu jikken, han’igentei de saikai e. Nikkei XTECH, 7 November.

32.

Koshizuka

(2016) IoT (Internet of Things) no dōkō to kongo no kadai. Keisoku to seigyo 55(12): 1024–1029.

33.

Kuramasu

(2004) Yubikitasushakai ni muketa jidōninshikigijutsu: Ningen no hyōjō ga motsu yutakana jōhō o jidōninshiki suru ‘kaojōhō no riarutaimu keisoku’. Computer & Network LAN 22(11): 5–8.

34.

Kuroda

Nada

Tezuka

, et al. (2019) Shōrai no ritēru sābisu o sasaeru seitai ninshō gijutsu ni yoru atarashii tempo soryūshon. NEC Technical Journal 71(2): 45–48.

35.

Langlois

Elmer

(2019) Impersonal subjectivation from platforms to infrastructures. Media Culture & Society 41(2): 236–251.

36.

Liu

(2004) The Laws of Cool: Knowledge Work and the Culture of Information. Chicago, IL: University of Chicago Press.

37.

Liu

Nishimura

(2019) Eizō kara fushinsha o kōseido de shiborikomu kōdō patānu no jidōbunrui. NEC Technical Journal 71(2): 87–90.

38.

Lyon

(2001) Under my skin: From identification papers to body surveillance. In: Caplan

Torpey (eds) Documenting Individual Identity: The Development of State Practices in the Modern World. Princeton, NJ: Princeton University Press, pp.291–310.

39.

Mann

(2019) ‘I Am Spartacus’: Individualising visual media and warfare. Media Culture & Society 41(1): 38–53.

40.

Ministry of Internal Affairs and Communications (2005) ‘u-Japan’ no gaiyō. Report, January. Available at: https://www.soumu.go.jp/main_sosiki/joho_tsusin/policyreports/chousa/yubi_pla/pdf/050316_2_03.pdf (accessed 10 November 2020).

41.

Ministry of Internal Affairs and Communications (2006) Jōhōtsūshinhakusho Heisei 180-nenban. Available at: https://warp.da.ndl.go.jp/info:ndljp/pid/258151/www.johotsusintokei.soumu.go.jp/whitepaper/ja/h18/pdf/18honpen.pdf (accessed 17 September 2020)

42.

Miura

Miyatake

Nagasaka

(2009) Kojinninshōsochi. Patent JP4207717B2, Japan.

43.

Mori

Sasaki

Shinzaki

(2003) Baiometorikusuninshōgijutsu. Fujitsu 54(4): 272–279.

44.

National Institute of Information and Communication Technologies (2013) Daikibo fukugōshisetsu ni okeru ICT gijutsu no riyō jisshōjikken o Ōsaka Sutēshon Siti de jissi. Press release. 25 November. Available at: https://www.nict.go.jp/press/2013/11/25-1.html (accessed 12 November 2020).

45.

NEC (2016) NEC to provide facial comparison technology at John F. Kennedy International Airport. Press release. 5 May. Available at: https://www.nec.com/en/press/201605/global_20160505_01.html (accessed 11 November 2020).

46.

NEC (2019a) Baoimetorikusu kenkyūjo: Kenkyūkaihatsu nit suite. Available at: https://jpn.nec.com/rd/labs/biometrics/index.html (accessed 10 November 2020).

47.

NEC (2019b) Sebun-Erebun-Japan: AI to IoT o katsuyō shita, hito to kankyō ni yasashii misedukuri. Video. Available at: https://www.youtube.com/watch?v=ExQeHUbVuZw (accessed 11 November 2020).

48.

NEC (2020a) Biometric authentication: Products & solutions. Available at: https://www.nec.com/en/global/solutions/biometrics/index.html (accessed 10 November 2020).

49.

NEC (2020b) Rōson: Dijitaru gijutsu o katsuyō shita shin’ya shōninka tempo e no chōsen. Video. Available at: https://www.youtube.com/watch?v=C60Gv5piDso (accessed 11 November 2020).

50.

NEC (2021) Gazō ni yoru jinbutsuzōbunseki shisutemu ‘FieldAnalyst.’ Available at: https://www.nec-solutioninnovators.co.jp/sl/fieldanalyst/fa_01/fieldanalyst_for_gate/ (accessed 4 August 2021).

51.

NTT Gijutsu jānaru (2018) Manbikitaisaku o mokuteki toshita NTT Higashinihon no AI kamera yō purattofōmusābisu ‘AI gādoman’ no teikyo. NTT gijutsu jānaru 30(11): 41–43.

52.

Ogata

(2004) Yubikitasukankyōjitsugen no tame no sensa. Shisutemu/Seigyo/Jōhō 48(11): 452–457.

53.

Okatani

(2014) Deep learning. The Journal of the Institute of Image Information and Television Engineers 68(6): 466–471.

54.

OMRON (2020) Gazō senshingu gijutsu apurekēshonrei: Seibetsu – Nenreisuitei. Available at: https://plus-sensing.omron.co.jp/function/attribute (accessed 10 November 2020).

55.

Pisani

O’Brien

(2020) Amazon bans police use of its face recognition for a year. Associated Press, 10 June.

56.

Rieger

(2018) At least two malls are using facial recognition technology to track shoppers’ ages and genders without telling. CBC News, 26 July.

57.

Sakamoto

Takaya

(2014) Seitaininshōgijutsu no sūsei (dai 2 kai) Senshinkoku ni okeru dōkō: Kaoninshōgijutsu o ōyōshita biggudēta – māketingu. Jidōninshiki 27(6): 54–57.

58.

Satō

(2019) Ōsaka Metoro, kaoninshō de gēto o kaihei suru kaisatsuki no jikken. CNET Japan, 11 December.

59.

Seto

(2004) Baiometorikkusuninshōgijutsu no dōkō. Journal of the Society of Instrument and Control Engineers 43(7): 533–538.

60.

Seto

(2014) Baimetorikku ninshōgijutsu no kenkyūkaihatsu to seihinka e no kitai. IEICE Fundamentals Review 8(2): 77–83.

61.

Turow

(2017) The Aisles Have Eyes: How Retailers Track Your Shopping, Strip Your Privacy, and Define Your Power. New Haven, CT: Yale University Press.

62.

Weiser

(1991) The computer for the 21st century. Scientific American 265(3): 94–104.

63.

Wiener

(1954) The Human Use of Human Beings: Cybernetics and Society. Boston, MA: Houghton Mifflin Company.

64.

Zuboff

(2015) Big other: Surveillance capitalism and the prospects of an information civilization. Journal of Information Technology 30(1): 75–89.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.08 MB