From Statistics to Stories: Indices and Indicators as Communication Tools for Social Change

Abstract

The terms ‘indices’ and ‘indicators’ may immediately cause eyelids to droop. How, then, might they serve to impassion publics and, ultimately, promote social change? This paper examines the extent to which indices and indicators can be considered communication tools for social movements and social change. The analysis is based on a 2018 evaluation of one index based in the United States – the Ranking Digital Rights Index, which assesses privacy and freedom of expression in the ICT space – and incorporates interviews with civil society stakeholders. Bringing theory from the fields of journalism and social movements together with the data from the evaluation, the findings suggest indices can serve as useful communication resources for social movements under certain circumstances. In particular, the analysis suggests three communication resources – legitimate information, newsworthy information, and flexible information – that human rights indices are most likely to provide.

Keywords

communication journalism freedom of expression indicators social movements

Introduction

‘Human rights,’ according to the United Nations (1948), relate to “freedom, justice, and peace” and include rights related to physical freedom, marriage, religion, healthcare, privacy, and free speech, among others. While these rights are typically associated with governments and government policy, today it is widely acknowledged that private actors, including private companies, also play a significant role in violating human rights, such as by implementing dangerous working conditions.

One question to be discussed here is whether, and under what circumstances, human rights indices – which numerically score entities on their adherence to human rights – can promote social change. It then considers the circumstances under which these indices are more likely to productively serve as ‘communication tools’ to assist social movement actors in their efforts to change policies and institutions.

These questions are analyzed through an evaluation of the Ranking Digital Rights (RDR) Corporate Accountability Index, a program housed at the U.S.-based think tank New America which ranks 26 of the world's largest information and communication technology (ICT) companies on their freedom of expression and privacy policies. ICTs, including social media sites such as Facebook, are increasingly recognized as potential threats to human rights and democracy (e.g., Fernback & Papacharissi 2007; Maréchal 2015; Whitten-Woodring et al. 2020).

The Index uses companies’ publicly available user disclosure statements to assess adherence to human rights principles related to freedom of expression (e.g., whether a company explains to users when and why users’ content is removed) and to privacy (e.g., whether a company discloses what types of data it collects from users and how the data are used). Its human rights standards are based on the U.N.'s Guiding principles on business and human rights and the Global Network Initiative's Principles and guidelines (Global Network Initiative 2017). The Index, now in its fifth iteration, seeks to change corporate policies to align with these standards.

The analysis, based on internal RDR documents and interviews with relevant stakeholders, suggests that human rights indices may indeed serve as useful communication tools in global social movement toolkits, under certain circumstances. Drawing from literature on journalism and social movements, in addition to the data provided by the current analysis, the findings suggest that indices can potentially provide three needed communication resources to social movements: legitimate information, newsworthy information, and flexible information.

Literature Review

An index is essentially something that points to something else, such as the index at the back of a book. It is composed of indicators which, though ill-defined (Green 2001), are essentially quantifiable ‘things’ that are indicative of less quantifiable ‘things.’ Indicators can be conceived of as proxies or “metaphor[s] for phenomena that are not directly measurable” (Green 2001). Here, the concept of an index refers to a system designed to produce numerical scores across a range of entities (e.g., companies) based upon some set of indicators. More formally, indices of this type are “compound measures that aggregate multiple indicators” (Hawken and Munck 2013: 802).

For example, a website that rates local restaurants may develop scores (or indicators) on a variety of elements, such as food presentation or cleanliness, based on user ratings from one to five. Those scores (indicators) may then be aggregated into an overall indicator of ‘quality’ for each restaurant, which the public can then use to compare restaurants. In this case, the overall set of rated and ranked restaurants could be considered an index, based on indicators relating to the dining experience.

Human-rights- and social-progress-based indices (from here on, ‘HR indices’) work similarly. This section reviews the concept of HR indices, the theory behind them, and their practical and theoretical connections to communication and social movements, in order to shed light on how HR indices could potentially serve as communication tools for social movements.

Indices to Promote Institutional Accountability

Indices relating to social progress and human rights have been around for some time. They are increasingly recognized as “important for designing and assessing policies aiming at advancing the progress of society” and have multiplied over the past forty years or so as technology has provided greater access to information (Stiglitz et al. 2009: para. 3). They are also recognized as communication tools. Indeed, the European Commission (2019), in its guidance on creating indices, states that composite indicators “are ultimately a communication tool” that provide policymakers “with the ‘big picture’ on matters” relevant to social and economic progress.

HR indices have historically focused on country-level social progress and human rights fulfillment (e.g., the Human Development Index or Freedom House's ranking of press freedom) (Green 2001; Merry 2011) but many other types of HR-related indices exist. In the United States, for example, the “School Health Index” tracks, among other things, health and safety protocols in U.S. schools and the “Prison Index” tracks U.S. crime and punishment statistics.

Using indicators to rank private companies is a somewhat new, but growing, practice. Corporate HR indices seek to change behavior by publicly exposing companies that are not adequately protecting human rights. The past two decades have seen a rise in indices ranking global corporations, such as Know the Chain (which ranks apparel companies on their labor practices), the Enough Project (which ranks electronics companies’ use of conflict minerals), and Oxfam's Behind the Brand (which ranks companies on their agricultural policies). There are now more than 600 global rating systems assessing corporate policies and behaviors related to environmental, social, or governance issues (Wong et al. 2019). Typically, these indices are based on internationally accepted standards and measure publicly available disclosure statements. RDR is relatively unique, though not alone, in its focus on rights related to freedom of expression and privacy in the ICT space.

The Allure (and Pitfalls) of Indicators

What these varying versions of HR indices have in common is their attempt to hold powerful entities to account when it comes to policies and behaviors that thwart social progress or threaten human rights. They attempt to take opaque protocols, policies, and behaviors and simplify them to a degree that (ideally) anyone ccould easily see which entities are causing or exacerbating social problems. Groups who choose to could then, in theory, use this information to push policy change or collective action.

Baack (2015: 5), in a discussion of open source coding, argues that the increasing availability of data offers the potential to decrease the power gap between everyday citizens and powerful elites by allowing them to “better understand” the social consequences of elite actions and thus “be more active and engaged.” In this way data can also function as a source of power for groups seeking to create change. Crooks and Currie (2021: 204) coin the term “agonistic data practices” to refer to how community organizers use data to “increase government transparency, agitate for resources” and, ultimately, motivate collective action. Bruno et al. (2014: 200) use the term “statactivism” to describe how social movements may use statistics to “[denounce] a certain state of reality.” In these cases everyday citizens and movement organizers use data to stake a claim on ‘the truth’ – often attempting to use data to convincingly demonstrate that the version of reality offered by elite institutions is invalid or unjust. When turned into indicators and indices, data can take messy phenomena and simplify them in ways that make it easy to see what different institutions are doing and, crucially, to compare them (George 2018).

However, while greater access to data may facilitate social change, there are also worries about the consequences of a world increasingly dependent upon the ‘datafication’ of almost everything. This trend has come with a narrative that data (i.e., numbers) represent an inarguable ‘truth,’ that numbers cannot lie, and that the presentation of data is therefore objective. But of course this is not the case. Data do not emerge, fully formed, from the ether. They are collected, collated, and then presented by human beings with emotions, values, and potentially competing agendas.

This means that data can, at best, present one or more interpretations of ‘the truth.’ Indeed, Porter (1995: 5) argues that an overliance on data actually situates us farther from truth because it “simply evades the deep and important issues.” Similarly Loukissas (2019) notes that by choosing which indicators to use one necessarily places value on some aspects of reality, devalues others, and delimits possible interpretations by outsiders.

In this way indicator choices and definitions often reveal more about the ideologies of their developers than about the social reality they claim to clarify (Merry 2016). GDP, for example, is often used as an indicator for global development, but is embedded with ideologies about what ought to be valued (such as consumption) and what ought to be ignored and therefore devalued (such as environmental degradation or the social contributions of stay-at-home parents) (Fioramonti 2013); costs of a new home calculated by real estate websites consider proximity to good schools but do not consider the potential externalities of gentrification (Loukissas, 2019). Data and indicators therefore must be interpreted with caution as they will always only represent one perspective of ‘truth’ and one small slice of reality.

RDR and its Indicators

RDR is certainly not immune from ideological biases. While a thorough critique of RDR's own ideologies are outside the scope of this paper, they are likely to influence its methodology. For example, the New America Foundation, where RDR is housed, has been categorized as politically left-leaning (influencewatch.org 2021) and has been critiqued for catering to the desires of its corporate and government donors (Dayen 2017). Additionally, the Global Network Initiative, upon whose principles the RDR methodology is based, is a global, multistakeholder group which includes some of the very corporations RDR evaluates, including Google and Facebook.

It is impossible to say how this context might affect the Index but it is worthwhile to note that definitions of human rights concepts such as ‘freedom of expression’ and ‘privacy’ – two of RDR's top concerns – are not self-evident. RDR creates indicators to capture these concepts as it deems fit. For example, within its ‘freedom of expression’ indicators, the Index assesses whether companies notify users when their content is taken down and whether they explain why content is taken down. However, RDR makes no attempt to judge what types of speech ought to be allowed or removed. It is understandable that RDR would not make this fraught judgment call (e.g., what information should qualify as hate speech or disinformation and whether such content should be removed) and would most certainly be accused of ideological bias if it did so. At the same time however, the decision to privilege user notifications over content rules as a way to assess human rights alignment is in itself an ideological decision with potentially worrisome implications if, say, it provides legitimacy to organizations that are transparent in their policies but engage in harmful practices regarding content (such as hate speech).

In sum, HR indices are not panaceas and cannot be viewed as value free. However, this analysis does not seek to assess whether RDR's Index is a good one or to provide insight on whether or not social movements ought to use data and indicators at all. Rather, it explores how social movements might effectively make use of indicators, as one tool in their toolbox, if they see doing so as beneficial to their cause.

With this in mind, the following sections examine how social movements might make use of indicators to further their goals given the media and communication constraints with which they must contend.

Social Movements, Media, and Communication

There is insufficient understanding of the relationship between the fields of communication and social movements (Mattoni and Treré 2014; Obregón and Tufte 2017). As described in this section, communication theory offers insight with regard to what kinds of information are mostly likely to receive media (and particularly news) attention. Social movement theory illuminates what sorts of resources social movements require to communicate their concerns. The following discussion is most applicable to countries with at least a moderately free press system in which news outlets are able to choose what social issues to cover.

Social Movements and Media Coverage

‘Social movements’ here refers to attempts to promote social change or social justice around a particular issue. They can be narrow or broad, local or global, and can include a variety of interconnected individuals and groups.

Media advocacy – getting information about social issues to the public, through the media, in order to spur change – is an important piece of all social movements, but social movements are made up of an array of stakeholders with different goals – only some of which involve the media. Indeed, some social movements prove more successful than others because these stakeholders come together in a “complementary division of labor” that allows the whole social movement to become greater than the sum of its individual parts (Gamson and Wolfsfeld 1993: 121). In order to collaborate efficiently, actors and organizations need to be able to align with each other quickly and on an as-needed basis. These stakeholder networks have become increasingly complex in the age of the internet (Castells 2010).

Social movements may be particularly successful when stakeholders are organized, but at the same time, there is a need to understand social movements and collective action networks as complex, messy, “chaotic” (Carroll and Hackett 2006) or even “wild” (Brunner 2017). This means social movements need to be flexible – using information, events, and opportunities adeptly as movements develop and contexts change. Brunner (2017: 666) offers the concept of “wild public networks” to suggest social movement scholars need to start paying less attention to “things” and more attention to “connections among things.” Her call recognizes the changing and unpredictable nature of relationships between activists, the public, NGOs, researchers, corporations, media, and government.

Media coverage is but one piece of this chaotic puzzle, but a crucial one. The nature of news and information production changes on a seemingly weekly basis, with audiences consuming changing patterns of mainstream, niche, and alternative information in unpredictable combinations; norms regarding how ‘objective’ mainstream news ought to be shifts over time as well (Kelly 2019) which influences which news items get reported and how. Social movements, in order to gain media traction, must have their fingers on the pulse of these changing trends and creatively access these spaces to maximize impact (Billard 2021). Waisbord (2009: 377) refers to the ability to successfully navigate this space as “communication jujitsu.”

Keck and Sikkink (1998: 16) likewise refer to “information politics” or, “the ability to quickly and credibly generate politically usable information and move it to where it will have the most impact” as one of the four main tactics used by social movement networks. Similarly, Jackson (1982) includes “informational activities” such as television interviews as one of the five main strategies in which social movement actors engage. Overall, stakeholders work to mold information into a form that effectively highlights their cause and furthers their particular narrative.

Gaining Media Coverage: The Give and Take Between Journalists and Social Movements. Social movement actors are learning how to best navigate new digital media opportunities, but traditional news outlets continue to be an important avenue for advocacy (Langer and Gruber 2021). Part of the reason for this is that they add political legitimacy to a movement. Social movements typically aim to create change at the policy level and for this to happen discourse around issues needs to occur in politically focused and well respected media spaces.

Indeed, legitimacy and professionalism are cited within social movement literature as key resources for gaining political traction (Andrews and Caren 2010; Gamson and Wolfsfeld 1993). It is acknowledged that a lack of legitimacy makes it difficult for social movements to get messages across, and that stakeholders who are more established and have more authority garner greater media coverage (Andrews and Caren 2010; Wichgers et al. 2021).

The internet has increased opportunities for social movements to get information into the public sphere, but has simultaneously decreased their legitimacy, as they now exist in a world of information overload, misinformation, and disinformation, and thus have to deal with a (rightfully) skeptical public (Garrett 2006). This makes it all the more important that social movement actors look “credible” and not like conspiracy theorists (Garrett 2006: 215). The traditional media gatekeepers offer an aura of legitimacy when they cover stories.

To better understand the circumstances under which social movements do garner news coverage, it is helpful to examine the give and take relationship between social movement actors (the ones producing the information) and media gatekeepers (the ones reproducing it). Gamson and Wolfsfeld (1993: 115) note that often both sides fail to adequately recognize the needs of the other, resulting in unproductive relationships:

“Send my message,” say the activists; “Make me news,” say the journalists. In this dialogue of the deaf, neither activists nor journalists make an effort to understand how the other views their relationship or, better yet, the complex nature of these transactions.

From the journalists’ perspective, there is a need for ‘newsworthy’ material, which typically translates to news that is novel and interesting to readers or relevant to their daily lives (Billard 2021; Galtung and Ruge 1965). These are the stories that will attract audiences and sell papers. Social movement actors have developed a variety of strategies to meet journalists’ criteria regarding relevance and emotional interest. These include staging visually and emotionally striking protests (Jiménez-Martínez 2021), connecting to local issues (Andrews and Caren 2010), recruiting celebrities who care about the issue (Kogen 2015), and relaying personal stories of victims (Briscoe and Gupta 2016), among others. These strategies help bring sympathy and passion to a movement.

But journalists also often require that information be legitimate and objective (Billard 2021). Some countries have developed different norms around the value of objectivity in news, but there is evidence that objectivity norms still prevail in democratic countries (Márquez-Ramirez et al. 2020). Journalists’ desire for objective, legitimate information, however, creates a Catch-22 when it comes to establishing relationships with social movement actors: Social movements need news media coverage to increase legitimacy, but news outlets require some level of legitimacy on the part of an organization in order to report on it or quote it. Organizations that are less well known or seen as heavily biased have difficulty convincing journalists to cover them. As Waisbord (2009: 377) argues, most social movement and civil society organizations “are not official sources” and lack both “newsmaking clout” and “definitional power.”

If social movement actors are not pre-established, they can easily come across as biased, as can journalists covering them. Media advocacy is almost never ‘objective,’ in that it is advocating for policy change based on a particular assessment of human morality. If reporters claim to be objective, they cannot be seen as endorsing a particular policy, party, or ideological group, and incorporating all sides of a complex debate is often difficult or impossible. This concern can lead journalists to avoid writing about potential policy solutions to chronic social problems (Kogen 2019; Lemert 1981).

With regard to establishing legitimacy around a cause, one strategy is for social movement actors to provide journalists with concrete evidence of wrongdoing, rather than simply assert that there is a problem. News organizations, in their ‘watchdog’ role, require evidence that a social problem exists, or of misbehavior on the part of powerful individuals. For news outlets, the investigative reporting required to uncover such evidence can often be costly. While current research suggests that the public still values investigative journalism, shrinking newsroom budgets mean the total number of issues investigated is on the decline (Carson and Farhall 2018).

Having social movement actors bring this evidence to journalists is thus a win-win for both sides. For information to be most effective, according to Keck and Sikkink (1998: 16), it must convincingly “show that a given state of affairs is neither natural nor accidental, identify the responsible party or parties, and propose credible solutions.” However, collecting “reliable and well documented” evidence of wrongdoing can be just as difficult for social movement actors as it is for journalists (Keck and Sikkink 1998: 19).

The Potential for Indices to Serve as Common Ground

Constraints placed on both news outlets and on social movements, such as those described above, complicate the ability of these groups to share resources and information. Resource mobilization theory (RMT), originally developed within organizational theory, has been used within social movement theory to highlight the kinds of resources required by social movements to be successful. RMT recognizes that social movements, like other kinds of organizations, need a variety of resources – such as money, skilled workers, and political power – to promote “collective action” (Jenkins 1983: 532).

RMT provides a useful lens through which to think strategically about the importance of different kinds of resources in promoting relationships between activists and journalists. Specifically, here, several of the ‘resources’ social movements require relate to entering the media sphere. These include legitimacy (both of the stakeholders involved in the movement and of their grievances), evidence of wrong-doing, and the presentation of policy solutions. These types of resources could, conceivably, be accessed by social movement actors through HR indices, which can provide journalists with legitimate information regarding evidence of ‘wrongdoing’ (powerful entities not adhering to ideal human rights principles) and can also provide information regarding policy solutions (aligning with internationally accepted human rights standards).

However, interpretation of raw data such as those associated with indices may require advanced statistical skills – a resource some social movements may lack (Baack, 2015). As boyd and Crawford (2012) argue, those who have the expertise to analyze data are few, are typically “the most privileged” (p. 675), and are the ones who usually get to decide how data are used. Crooks and Currie (2021: 202) also note that the use of data becomes especially problematic “when data work becomes the responsibility of the very communities who are experiencing harms stemming from state policy.” These are all potential barriers to the uptake of data and indicators by social movements.

Raw data are also, of course, analyzed by news outlets. Recent trends in data journalism suggest news outlets are relying more on ‘data journalists’ who focus on analyzing numbers as part of their investigative teams, now that the internet provides a plethora of data related to social issues (Carson and Farhall 2018).

Aim of the Present Study

The above discussion suggests that indices might be more likely to receive media attention (and thus to promote social change) if they present information in a way that is media-friendly, provide concrete evidence of social problems and potential solutions, and are perceived as legitimate and objective. Therefore, this analysis seeks to assess whether indices and indicators can function as communication tools for social movements and if so, which characteristics increase their utility.

These overarching questions are broken down into two specific research questions pertaining broadly to when and why indices might be influential communication tools for social movements.

RQ1: Can indices such as RDR's offer legitimacy to social movement actors?

RQ2: Can indices such as RDR's produce stories that align with what civil society organizations and journalists might consider newsworthy?

Method

The initial phase of this research was conducted through a midterm evaluation of RDR as required by the U.S. Department of State, one of its funders. The questions that guided that evaluation aligned with some, but not all, of the questions analyzed here. For this analysis, conducted in 2018, the interview transcripts were re-analyzed.

Data for both the midterm evaluation and this analysis were comprised of RDR internal documents (proposals, reports, monitoring and evaluation plans, records, etc.) and confidential in-depth interviews with civil society organizations (CSOs) regarding their use of the Index.¹

A purposive sampling method was used for the interviews. A list of CSOs engaged in issues of digital privacy and freedom of expression was compiled together with RDR. The list of stakeholders was global but was weighted toward the United States, where the project is based.² The interviews focused on understanding how the Index was used by CSOs, what made it useful, and what would increase usefulness. Ultimately, 14 CSO representatives (nine U.S.-based; five international) participated in interviews.³ Interviews lasted approximately 45 minutes. Below, all interview quotes are unidentified, per agreement with the interviewees.

Interviews were qualitatively analyzed for themes (Joffe, 2012) relevant to the two research questions, using Atlas.ti. The process was inductive, in that I did not enter the analysis with preconceived theories or hypotheses about what makes indices more or less useful to CSOs; the process was iterative, in that entries were re-read and recoded until a consistent set of codes was solidified (Bradley et al. 2007).

Findings and Analysis

While this analysis does not attempt to evaluate the impact of the RDR Index, it is worthwhile to briefly summarize the findings of the midterm evaluation with regard to Index impact. In other words, it is helpful to provide evidence that the RDR Index is influential before analyzing how to maximize the influence of this index and others. Evidence from the midterm evaluation suggests that the Index is indeed influencing corporate disclosures, policies, and discourse in the areas of privacy and freedom of expression, though the extent of that impact is unclear.

The general theory of change behind most corporate indices is that, at some point, companies seek a good score and care about how they look against their peers. This is clearly occurring, in that most of the ranked companies (though not all) actively seek to achieve positive scores and are disappointed if and when they do not.

Of the 22 companies ranked at the time of the analysis, 15 had direct verbal or written discussions with RDR about their preliminary results, meaning they were willing to devote significant time and resources responding to the Index results. Half of them (11) were willing to take the time to be interviewed by me. This suggests that they care about their scores, at least to some degree. In interviews, eight of the 12 corporate interviewees (one interviewee came from an unranked corporation) explicitly stated that they changed their disclosure statements to reflect disclosures advocated by RDR.

These findings suggest at least a limited impact of the Index on corporate policy disclosures to-date. The following sections examine the mechanisms and limits of influence by seeing how social movement actors are using the RDR data to promote change.

RQ1: Can Indices Such as RDR's Offer Legitimacy to Social Movement Actors?

Whether RDR's impact occurs because of actions by social movement actors, including media advocacy efforts, is outside the scope of this analysis, but the extant literature presented above suggests that media coverage of the rankings, or even the potential for coverage, would increase influence. This section therefore examines whether the Index provides the kinds of resources that we would expect to help CSOs garner media coverage, namely: legitimacy, evidence of wrongdoing, and presentation of policy solutions. This section at times refers to CSOs rather than ‘social movement actors’ because not all CSOs necessarily identify with one specific social movement.

One of the key resources that social movements need, and often lack, is legitimacy with respect to overall social concerns, to specific grievances, and to political solutions. Interviews with CSOs suggest that the Index does indeed provide usable, legitimate evidence, though not to the degree desired. Interviewees acknowledged that legitimate information regarding corporate misbehavior is sorely needed. One stated the need for “empirical evidence” that companies are “screwing up.” Another stated that advocacy groups need reliable data showing “very clearly, quantifiably, the way that… companies are… dropping the ball.” Five interviewees were already using the Index data to this end or said it was likely they would use it in the future. Others who were less familiar with the Index before the interview indicated they would review it to see if it was useful to them. Organizations that had used the Index in their communication with corporations said the data “serve as very strong backup for the anecdotal stuff we collect” and that it “kind of made our advice [to corporations] or our threat more credible by… forcing them to review their policies more objectively…”

The data are seen as legitimate primarily because of the methodological rigor that went into developing the Index (including the involvement of academics), the transparency regarding its methodology and scoring (which is based on internationally accepted standards), and the fact that scores are based on publicly available corporate disclosure statements.

Finally, a key piece of what makes the Index both legitimate and useful is the fact that it (like all indices) is composed of numbers. As discussed above, part of the allure of indices is that they can turn anecdotal evidence of a problem into quantitative and methodologically rigorous evidence of a problem. Numbers, of course, never tell the whole story when it comes to complex social phenomena, but they are perceived as persuasive in the public and policy spheres.

The allure of datafication came across in interviews with CSOs, such as the statements made above regarding the need for “quantifiable information” and the need to back up “anecdotal evidence.” Another interviewee stated that, in their discussions with policymakers, they find many reports on human rights to be written in a “discursive, narrative style” that is difficult to translate into a policy argument and that “objective scores” are important: “When I’m lobbying or writing an article, I want to be able to point to a sophisticated, thoroughly thought-out resource that can stand on its own two legs.” Another said that reliable statistics are “very useful” for them and they are always looking for ones they can provide to policymakers.

To summarize, the interviews suggest the Index does indeed provide legitimacy to CSO grievances, especially with respect to providing evidence of corporate wrongdoing. Factors that increase RDR's legitimacy include methodological transparency, alignment with internationally accepted standards, and assessments based on publicly accessible data. However, certain features of the Index limit its use by CSOs, to be addressed below.

RQ2: Can Indices Such as RDR's Produce Stories That Align With What Civil Society Organizations and Journalists Might Consider Newsworthy?

Can indices, based in statistics and numbers, produce the kinds of flashy, newsworthy headlines that are attractive to journalists? Gamson and Wolfsfeld (1993) hypothesized that two important characteristics of social movements that break into news media are that they have powerful framing and evoke sympathy. This begets the question of whether dry, numerical indices can provide enough of an interesting hook to receive media coverage. This analysis did not track media coverage or speak with journalists, so can only answer this question from the perspective of CSO actors who work with the media and based on previous research regarding what journalists value.

Overall, CSOs feel there is room for improvement regarding making the Index data ‘newsworthy.’ They want more attention-grabbing headlines and they want RDR to do a better job pulling out, as one interviewee put it, the “juiciest” findings. Suggestions for stories RDR could produce included “best versus worst” narratives, short headline pieces about companies that have substantially improved their score, and longer articles about general trends for each content category. CSOs working in different areas (e.g., children's rights, prevention of data breaches, democracy promotion) all want stories specific to their own concerns. CSOs said that if RDR could produce these pieces it would greatly help them bring the data into the media sphere.

In other words, CSOs clearly see the potential for the rankings to make good, eye-catching news stories that speak to a variety of publics – from the general public to politically engaged groups. However, the general consensus amongst interviewees was that RDR is not doing enough to produce these stories and that it must do more to help social movements shape the data into useful forms.

The fact that CSOs saw the role of shaping data and writing stories as belonging to RDR requires interrogation, however, as it presents a potential hurdle to maximizing the utility of indices more generally and includes some questionable assumptions about the ideal relationship between index developers, social movement actors, and the media.

On one hand, it makes sense that CSOs would want Index developers to produce stories about findings, given the Index developers’ obvious expertise in the methodology. However, RDR, like most CSOs, is small, with limited resources and staff. The scoring itself is resource-intensive and takes one year to complete. Producing an array of different news stories to cater to the various and changing needs of different social movement actors is, perhaps, an unrealistic ask.

It would seem more efficient, given the unpredictable, ever-changing needs of social movements addressed in the literature review, if social movement stakeholders themselves had the ability to use this information as they saw fit. Numbers and indicators do not offer one single narrative; they require interpretation. A set of indicators can be molded to a variety of stakeholder needs based on the intended audience. Such indicators can tell stories about improvements (or deteriorations) in policy over time, can assess human rights violations relevant to a particular demographic group, or can compare companies or regions, for example.

Indeed, the notion that CSOs see index developers as bearing the responsibility of writing stories seems to contradict extant theory presented above regarding the need for CSOs themselves to creatively use information as part of their “informational activities” (Jackson 1982), “information politics” (Keck and Sikkink 1998), or “communication jujitsu” (Waisbord 2009). Furthermore, CSOs’ assumption that RDR should produce the stories itself seemed to result in low use of the Index overall among interviewees. Why, then, do CSOs here assume that RDR should write stories, rather than seek to use the information themselves in ways that would benefit their organizations, movements, and narratives? If such data is in fact useful, why are CSOs not jumping upon it?

The interviews suggested, indirectly, that the usefulness of the Index would be greater if CSOs were better able to navigate the raw data. In fact, most CSOs seem to find the raw indicators impenetrable. They want RDR to distill findings and write headlines and articles because they do not feel capable of analyzing the data themselves. This is problematic, given that it is not possible for RDR, or any Index creator, to write all the pieces relevant to all stakeholders within a social movement.

The interviews suggest that the dataset is too big and too complex for organizations to handle. One interviewee stated the Index is a bit “too smart” in its hesitancy to “boil down [the results] into synthesizable messages.” Its methodological rigor is thus both a blessing and a curse: the thing that potentially makes indices a flexible information resource for civil society and social movements – the endless ways to combine and use the data – is also the thing that makes them difficult for many social movement actors to use. Put another way, part of the reason numbers are so alluring in terms of legitimacy is that they are perceived as scholarly and cerebral, but at the same time this perception poses a potential barrier to greater use by social movements that do not possess access to statisticians.

This was not stated explicitly in the interviews, but it was suggested. The reasoning for CSOs’ avoidance of the data was not obvious. Seven organizations interviewed were aware of the Index and saw it as highly relevant to their work but had not yet made the effort to utilize its data in more than superficial ways. When I pressed them on why they had not used the Index more, many gave responses akin to ‘I’m not sure. I should probably look at it again…’ They knew that there was potential for the Index to be useful to them, but something was keeping them from engaging fully with the data.

The purpose of indicators is to ‘indicate,’ or point to, more complex social phenomena in a relatively simple way, but indicators can still be quite complex. For example, within the broad category of ‘freedom of expression,’ RDR includes 11 ‘indicators’ that, when taken together, produce a ‘score’ (which is itself another indicator) regarding how well corporations adhere to human rights principles related to freedom of expression. What is more, each of these 11 indicators (e.g., ‘Sharing of User Information’) are calculated by aggregating between one and twelve ‘sub-indicators,’ which RDR refers to as “elements” (e.g., whether the company discloses sharing information with government). Altogether, the Index encompasses 11 indicators related to freedom of expression, 18 indicators related to privacy (each with multiple sub-indicators),⁴ across 24 companies, four continents, two broad industry categories,⁵ and nine specific services, resulting in more than 10,000 indicators in all for every iteration of the Index.

While it provides legitimacy to see how carefully the scores are calculated, it is also understandable why some organizations may be overwhelmed by the granularity and turned off by the level of attention needed to accurately interpret it. This led many of the interviewees here to assume that RDR ought to be the one to analyze the data. The following excerpts from five CSOs illustrate this sentiment:

“It’s a lot of different categories of information…”

“I know it’s great to have, like, a giant report… but if they found a way to drip out pieces of information separately I think it would just catch our attention more.”

“I do think it’s a lot of data and there might be a way it could be more… broken down.”

“It’s quite a lot of information… maybe if there was, like, splicing the information another way.”

“Device companies have different concerns from… telephone companies… and telcos have different concerns… from platform companies… So if it can kind of separate the ranking to like three different categories, maybe it will be more helpful on the ground.”

In other words, the CSOs suffered from information overload. As discussed in the literature review, everyday citizens often lack the expertise to effectively use data (Baack 2015; boyd and Crawford 2012). This certainly seems to be the case here. Access to data is less helpful if its audience is unable to use it effectively. Potential ways to shift this task to social movement actors is addressed in the next section.

Indices As a Flexible and Accessible Information Resource

If we turn again to the concepts of information politics and communication jujitsu it becomes clear that requiring index creators to interpret and analyze data is counterproductive. Social movement actors themselves are best placed to navigate the news and information space around a particular issue and quickly move data to advance a given narrative, supply journalists with needed information, or respond to a news event. Index creators cannot predict which stories will be most useful in any particular moment.

Instead, it would make more sense for RDR and other indices to find ways to make these data more accessible and approachable so that stakeholders can create the headlines and stories they need to facilitate their own causes. But for this to happen, the data must be user-friendly and easy to navigate.

The fact that indices are comprised of numbers should not make them impenetrable. Contrary to some data skeptics, using indicators does not always require advanced mathematical skills; it is often only a matter of understanding what indicators mean. In the case of RDR, for instance, ‘analysis’ might simply mean looking at the scores for a particular indicator (e.g., “Does the company notify users when terms of service change?”) and comparing scores across companies to see who performed the best or worst. One suggestion that came up in the interviews was for RDR to conduct virtual trainings with CSOs in order to teach them how to navigate the dataset. Training sessions need not teach advanced statistics, but rather could simply familiarize CSOs with the available data, show them how to access variables of interest to the organization, and provide examples of ways to use them.

In other words, the way for CSOs to effectively use indicators is to learn how to use them to suit their own needs. Put yet another way, rather than being data-driven, indicator usage should be objective driven and passion driven. Numbers cannot replace stories, passion, or emotions that breathe life into a movement, but they can buttress them. As Crooks and Currie (2021: 209) state, it is sometimes about using “feelings to make sense of data… not only… rational or logical faculties.” They use the slogan of the Occupy Wall Street Movement – “We are the 99 percent!” – as a prime example of using statistics to serve a movement and a passion rather than using data as a starting point and struggling to ‘interpret’ it.

Three Communication Resources

Waisbord (2015) argues that there is a need for better theory linking media advocacy and communication for social change (CSC). Likewise, Obregón and Tufte (2017: 641) argue that there is a need for a “more rigorous exploration of conceptual, theoretical, and applied perspectives” that connect social movement research and CSC research and call upon scholars to address “the increasingly influential role of communication dimensions in social movements.” While indicators are recognized as communication tools in the policy arena (e.g., European Commission, n.d.), they have not been sufficiently conceptualized as communication tools for social movements. This analysis suggests that human rights indices are potentially useful communication resources for social movements.

Social movement theory has utilized resource mobilization theory to consider how certain resources aid social movements. Following from the present analysis, we can start to think about communication resources and communication resource mobilization as important considerations when analyzing how social movements can most successfully enter the media sphere.

The analysis suggests three specific resources indices might provide related to communication and information: legitimate information, newsworthy information, and flexible information. All of these are in demand by social movement stakeholders.

With regard to legitimacy, the analysis supports the idea that indices can provide internationally accepted policy solutions and concrete evidence of wrongdoing. Indices that are perceived as ideologically independent, based on internationally accepted standards, and which feature a transparent methodology are more likely to provide legitimacy to social movements.

With regard to providing newsworthy information and flexible information, indices face the challenge of being simultaneously academically rigorous and user-friendly. Data, as raw indicators, are malleable, and can be arranged in a variety of permutations to suit the needs of various stakeholders. They are inherently flexible. Social movement stakeholders can selectively use data to communicate with journalists, policymakers, or the public. Journalists and policymakers can use the data to inform their own work. On the other hand, manipulating numbers may be outside the wheelhouse of many social movement actors, and index creators may therefore need to provide tools and training to help social movement actors learn how to use the raw data.

Based on the analysis here, the types of indices that are most likely to help social movement actors are ones that include indicators that can serve as evidence of social ills and would inform policy solutions, are easy to understand, are flexible, are perceived as ideologically independent, have a transparent methodology, and are regularly updated. Of course, these features guarantee neither that an index will be ideologically neutral nor that it will necessarily further social movement goals. Nevertheless, indices should be granted more attention as an additional tool in a movement's toolkit as social movements navigate an ever-changing and increasingly complex information sphere.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the U.S. Department of State,

ORCID iD

Lauren Kogen

Notes

Author Biography

Lauren Kogen is an Assistant Professor in the Department of Media Studies and Production at Temple University. Her research focuses on communication for social change and how social issues are framed in the news.

References

Andrews

K. T.

Caren

. 2010. “Making the News: Movement Organizations, media Attention, and the Public Agenda.” American Sociological Review 75 (6): 841–66. https://doi.org/10.1177/0003122410386689.

Baack

2015. “Datafication and Empowerment: How the Open Data Movement re-Articulates Notions of Democracy, Participation, and Journalism.” Big Data & Society 2 (2): 1–11. https://doi.org/10.1177%2F2053951715594634.

Billard

T. J.

2021. “Movement-media Relations in the Hybrid media System: A Case Study from the U.S. Transgender Rights Movement.” The International Journal of Press/Politics 26 (2): 341–61. https://doi.org/10.1177/1940161220968525.

boyd

Crawford

. 2012. “Critical Questions for big Data.” Information, Communication & Society 15 (5): 662–79. https://doi.org/10.1080/1369118X.2012.678878.

Bradley

E. H.

Curry

L. A.

Devers

K. J.

. 2007. “Qualitative Data Analysis for Health Services Research: Developing Taxonomy, Themes, and Theory.” Health Services Research 42 (4): 1758–72. https://doi.org/10.1111/j.1475-6773.2006.00684.x.

Briscoe

Gupta

. 2016. “Social Activism in and Around Organizations.” The Academy of Management Annals 10 (1): 671–727. https://doi.org/10.1080/19416520.2016.1153261.

Brunner

2017. “Wild Public Networks and Affective Movements in China: Environmental Activism, Social media, and Protest in Maoming.” Journal of Communication 67 (5): 665–77. https://doi.org/10.1111/jcom.12323.

Bruno

Didier

Vitale

. 2014. “Statactivism: Forms of Action Between Disclosure and Affirmation.” Partecipazione e Conflitto: The Open Journal of Sociopolitical Studies 7 (2): 198–220. https://doi.org/10.1285/i20356609v7i2p198.

Carroll

W. K.

Hackett

R. A.

. 2006. ““Democratic media Activism Through the lens of Social Movement Theory.” Media, Culture, & Society 28 (1): 83–104. https://doi.org/10.1177/0163443706059289.

10.

Carson

Farhall

. 2018. “Understanding Collaborative Investigative Journalism in a ‘Post-Truth’ age.” Journalism Studies 19 (13): 1899–911. https://doi.org/10.1080/1461670X.2018.1494515.

11.

Castells

2010. The Rise of the Network Society. 2nd edition. Malden, MA: Wiley-Blackwell.

12.

Crooks

Currie

. 2021. “Numbers Will not Save us: Agonistic Data Practices.” The Information Society 37 (4): 201–13. https://doi.org/10.1080/01972243.2021.1920081.

13.

Dayen

2017. “New think tank emails show ‘how Google wields its power’ in Washington.” Theintercept.com.

14.

European Commission. n.d. Ten step guide. https://knowledge4policy.ec.europa.eu/composite-indicators/10-step-guide/step-10-visualisation_en.

15.

European Commission. 2019. Composite indicators. https://ec.europa.eu/jrc/en/research-topic/composite-indicators.

16.

Fernback

Papacharissi

. 2007. “Online Privacy as Legal Safeguard: The Relationship among Consumer, Online Portal, and Privacy Policies.” New Media & Society 9 (5): 715–34. https://doi.org/10.1177/1461444807080336.

17.

Fioramonti

2013. Gross Domestic Problem: The Politics Behind the World’s Most Powerful Number. New York: Zed Books.

18.

Galtung

Ruge

M. H.

. 1965. “The Structure of Foreign News: The Presentation of the Congo, Cuba and Cyprus Crises in Four Norweigan Newspapers.” Journal of Peace Research 2: 64–90. https://doi.org/10.1177/002234336500200104.

19.

Gamson

W. A.

Wolfsfeld

. 1993. “Movements and media as Interacting Systems.” Annals of the American Academy of Political and Social Science 528: 114–25. https://doi.org/10.1177/0002716293528001009.

20.

Garrett

K. R.

2006. “Protest in an Information Society: A Review of Literature on Social Movements and new ICTs.” Information, Communication & Society 9 (2): 202–24. https://doi.org/10.1080/13691180600630773.

21.

George

2018. “Corporate Social Responsibility and Social media Corporations: Incorporating Human Rights Through Rankings, Self-Regulation and Shareholder Resolutions.” Duke Journal of Comparative & International Law 28 (3): 521–38. https://scholarship.law.duke.edu/djcil/vol28/iss3/9

22.

Global Network Initiative. 2017. GNI principles on freedom of expression and privacy. https://globalnetworkinitiative.org/wp-content/uploads/2018/04/GNI-Principles-on-Freedom-of-Expression-and-Privacy.pdf .

23.

Green

2001. “What we Talk About When we Talk About Indicators: Current Approaches to Human Rights Measurement.” Human Rights Quarterly 23 (4): 1062–97. https://doi.org/10.1353/hrq.2001.0054.

24.

Hawken

Munck

G. L.

. 2013. “Cross-national Indices with Gender-Differentiated Data: What do They Measure? How Valid are They?” Social Indicators Research 111: 801–38. https://doi.org/10.1007/s11205-012-0035-7.

25.

Jackson

1982. “Tactics of Confrontation.” In The Public Affairs Handbook, ed. Nagelschmidt

J. S.

, 211–20. New York: American Management Association.

26.

Jenkins

J. C.

1983. “Resource Mobilization Theory and the Study of Social Movements.” Annual Review of Sociology 9 (1): 527–53. https://doi.org/10.1146/annurev.so.09.080183.002523.

27.

Jiménez-Martínez

2021. “The Instrumental Mediated Visibility of Violence: The 2013 Protests in Brazil and the Limitations of the Protest Paradigm.” The International Journal of Press/Politics 26 (3): 525–46. https://doi.org/10.1177%2F1940161220923984.

28.

Joffe

2012. “Thematic Analysis.” In Qualitative Research Methods in Mental Health and Psychotherapy: A Guide for Students and Practitioners, eds. Harber

Thompson

A. R.

, 209–23. Chichester, West Sussex: John Wiley & Sons.

29.

Keck

M. E.

Sikkink

. 1998. Activists Beyond Borders: Advocacy Networks in International Politics. Ithaca, NY: Cornell University Press.

30.

Kelly

2019. “Evaluating the News: (Mis)Perceptions of Objectivity and Credibility.” Political Behavior 41 (2): 445–71. https://doi.org/10.1007/s11109-018-9458-4.

31.

Kogen

2015. “For the Public Good or Just Good Publicity? Celebrities and Humanitarianism.” Mass Communication and Society 18 (1): 37–57. http://doi.org/10.1080/15205436.2013.851699.

32.

Kogen

2019. “News you can use or News That Moves? Journalists’ Rationales for Coverage of Distant Suffering.” Journalism Practice 13 (1): 1–15. http://doi.org/10.1080/17512786.2017.1400395.

33.

Langer

A. I.

Gruber

J. B.

. 2021. “Political Agenda Setting in the Hybrid media System: Why Legacy media Still Matter a Great Deal.” The International Journal of Press/Politics 26 (2): 313–40. https://doi.org/10.1177%2F1940161220925023.

34.

Lemert

J. B.

1981. Does Mass Communication Change Public Opinion After all? Chicago: Nelson-Hall.

35.

Loukissas

Y. A.

2019. All Data are Local: Thinking Critically in a Data-Driven Society. Cambridge, Massachusetts: MIT Press.

36.

Maréchal

2015. “Ranking Digital Rights: Human Rights, the Internet and the Fifth Estate.” International Journal of Communication 9 (10): 3440–3449. https://ijoc.org/index.php/ijoc/article/view/3330

37.

Márquez-Ramirez

Mellado

Humanes

M. L.

Amado

Beck

Davydov

Mick

, et al. 2020. “Detached or Interventionist? Comparing the Performance of Watchdog Journalism in Transitional, Advanced and non-Democratic Countries.” The International Journal of Press/Politics 25 (1): 53–75. https://doi.org/10.1177/1940161219872155.

38.

Mattoni

Treré

. 2014. “Media Practices, Mediation Processes, and Mediatization in the Study of Social Movements.” Communication Theory 24 (3): 252–71. https://doi.org/10.1111/comt.12038.

39.

Merry

S. E.

2011. “Measuring the World: Indicators, Human Rights, and Global Governance.” Current Anthropology 52 (S3): S83–95. https://doi.org/10.1086/657241.

40.

Merry

S. E.

2016. The Seductions of Quantification: Measuring Human Rights, Gender Violence, and sex Trafficking. Chicago: University of Chicago Press.

41.

Obregón

Tufte

. 2017. “Communication, Social Movements, and Collective Action: Toward a new Research Agenda in Communication for Development and Social Change.” Journal of Communication 67 (5): 635–45. https://doi.org/10.1111/jcom.12332.

42.

Porter

T. M.

1995. Trust in Numbers: The Pursuit of Objectivity in Science and Public Life. Princeton, New Jersey: Princeton University Press.

43.

Stiglitz

J. E.

Sen

Fitoussi

. 2009. Report by the commission on the measurement of economic performance and social progress. https://rlms-hse.cpc.unc.edu/publications/bib/1921/.

44.

United Nations. 1948. Universal declaration of human rights. https://www.un.org/en/about-us/universal-declaration-of-human-rights .

45.

Waisbord

2009. “Advocacy Journalism in a Global Context.” In The Handbook of Journalism Studies, eds. Wahl-Jorgensen

Hanitzsch

, 371–85. New York, NY: Routledge.

46.

Waisbord

2015. “Three Challenges for Communication and Global Social Change.” Communication Theory 25 (2): 144–65. https://doi.org/10.1111/comt.12068.

47.

Whitten-Woodring

Kleinberg

M. S.

Thawnghmung

Thitsar

M. T.

. 2020. “Poison if you Don’t Know how to use it: Facebook, Democracy, and Human Rights in Myanmar.” The International Journal of Press/Politics 25 (3): 407–25. https://doi.org/10.1177%2F1940161220919666.

48.

Wichgers

Jacobs

Spanje

J. V.

. 2021. “The Battle of Frame Building: The Reciprocal Relationship Between Journalists and Frame Sponsors.” The International Journal of Press/Politics 26 (3): 674–98. https://doi.org/10.1177%2F1940161220942760.

49.

Wong

Brackley

Petroy

. 2019. Rate the raters 2019: Expert views on ESG ratings. SustainAbility. https://www.sustainability.com/globalassets/sustainability.com/thinking/pdfs/sa-ratetheraters-2019-1.pdf.