Bearing witness: Introducing the Perceived Mass Atrocities Dataset (PMAD)

Abstract

The risk factors and consequences of atrocities are deeply interconnected with questions of intra- and interstate stability and conflict, economic development, colonialism, and gender equality, as well as atrocity crime monitoring and prevention. However, there is no globally comparable measure of lethal and less-lethal atrocities. The Perceived Mass Atrocities Dataset (PMAD) is a country-year measure of atrocities with accompanying narratives. Built to support the US Congress’s Elie Wiesel Genocide and Atrocities Prevention Act of 2018, PMAD enables the systematic comparison of the occurrence and magnitude of seven atrocity types, in addition to group-perpetrated violence against women and LGBTQIA+ groups, with aggregate atrocities indices for 196 countries from 2018 to 2022. PMAD offers a foundation for quantitative studies of atrocities as well as more qualitative, process-focused research of lethal and less-lethal violence with its single, divisible framework. The PMAD data highlight several regions where analysis of atrocities using data on only lethal atrocities would be inadequate, especially Central and Eastern Asia. The data can also facilitate research into the relationships between mass atrocities and gender discrimination, neopatrimonialism, and political polarization.

Keywords

atrocities human rights quantitative international politics

Introduction

‘For the dead and the living, we must bear witness’, – Elie Wiesel

Researchers have long sought to gather data to better understand and anticipate atrocities (Butcher and Goldsmith, 2016; Butcher et al., 2020; Goldsmith et al., 2013; Harff and Gurr, 1988, 1998; Rost, 2013; Schrodt and Ulfelder, 2016; Williams, 2016). Yet, existing atrocities datasets have been criticized for their lack of consistency, incomplete information, and discordant or overly broad definitions. These limitations can bias our understanding of the causes and consequences of such events (Bagozzi and Koren, 2017; Demarest and Langler, 2022; Eck, 2012), raising questions about the validity of existing studies’ results. These possible mismeasurements may similarly skew the descriptive record of past atrocities.

To remedy this gap, we introduce the Perceived Mass Atrocities Dataset (PMAD), a resource built to support the US Congress’s Elie Wiesel Genocide and Atrocities Prevention Act of 2018. PMAD enables the systematic comparison of the scope and scale of lethal and less-lethal atrocities suspected to have been perpetrated by state and non-state actors. We use an index-based approach to measure that which cannot be measured directly (Ralph et al., 2015: 7–8). We emphasize ‘perceived’ atrocities given that some events may not be observed or reported (‘perceived’ using secondary sources) and because the definition of atrocity is highly subjective (‘perceived’ according to our coding criteria).

The PMAD data provide country-year, index-based measures of atrocities, including the occurrence and magnitude of seven atrocity types, the severity of group-perpetrated violence against women and LGBTQIA+ groups, and aggregate atrocities indices for 196 countries from 2018 to 2022. These data, described below, are recorded in country-year format and accompanied by country case narratives. The country-year format aggregates information from events into a panel dataset that can be used for statistical analysis. The variables coded for the country-year data include information on: country identification; perceived atrocities, including atrocity types; data reliability; and sourcing.

The PMAD data highlight several regions where analysis of atrocities using data on only lethal atrocities may be misleading given the relatively high frequency of less-lethal atrocities but relatively low frequency of lethal atrocities. The data also highlight avenues for further research into the relationships between mass atrocity perpetration and gender discrimination, neopatrimonialism, and political polarization.

Next, we briefly discuss previous research and remaining data limitations. We continue by describing the PMAD creation process. This is followed by a descriptive exploration of PMAD and validation against alternative series. We conclude with a cursory analysis and discussion of applications.

Previous research

The notion of what qualifies as an atrocity is elusive. Like terrorism, some may claim it is overly subjective to the point of prohibiting a commonly agreed upon definition (Ganor, 2002) – a position we acknowledge but reject given that decisionmaking too often requires concrete if imperfect definitions and quantifications (Keeney, 1996). Tools such as indices allow us to measure phenomena that are only loosely defined or may not be directly measured (Ralph et al., 2015: 7–8) while communicating inherent uncertainty (Gurr, 1972: 101–108).

For a common reference, many studies focus on atrocities in accordance with legal definitions of atrocity crimes in international human rights frameworks (Hirsch and Dixon, 2021; Raymond, 2013; Savelsberg, 2018; Scheffer, 2006). These frameworks include the Rome Statute of the International Criminal Court (United Nations, 1998), the 2005 World Summit Outcome of the United Nations (UN) General Assembly (2005), and the Elements of Crimes of the International Criminal Court (2013). Though they cannot eliminate subjectivity, definitions such as the following provide concrete elements around which data can be built:

A ‘crime against humanity’ means any of the following acts when committed as part of a widespread or systematic attack directed against any civilian population, with knowledge of the attack: (a) Murder; (b) Extermination; (c) Enslavement; (d) Deportation or forcible transfer of population; (e) Imprisonment or other severe deprivation of physical liberty in violation of fundamental rules of international law; (f) Torture; (g) Rape, sexual slavery, enforced prostitution, forced pregnancy, enforced sterilization, or any other form of sexual violence of comparable gravity; (h) Persecution against any identifiable group or collectivity on political, racial, national, ethnic, cultural, religious, gender [. . .] grounds […]; (i) Enforced disappearance of persons; (j) The crime of apartheid; (k) Other inhumane acts of a similar character intentionally causing great suffering, or serious injury to body or to mental or physical health. (United Nations, 1998: Article 7, paragraph 1)

One widely used data source is the Political Instability Task Force (PITF) Worldwide Atrocities Dataset (Goldsmith et al., 2013; Harff, 2003; Krain, 1997; Rost, 2013). PITF builds on Harff and Gurr (1998) and Harff (1992, 2003), defining an atrocity as ‘implicitly or explicitly political, direct, and deliberate violent action resulting in the death of noncombatant civilians’ (Schrodt and Ulfelder, 2016: 6). PITF uses a five-death minimum threshold for inclusion of observations.

Tracking ‘mass killings,’ which include genocide, politicide, and terrorism, the Targeted Mass Killing (TMK) dataset (Butcher et al., 2020) uses a minimum death threshold of 25 non-combatants. To establish intent, TMK includes clarification about whether the event was premeditated and designed to disproportionately target civilians, and if the group’s leadership approved this type of violence during any larger action.

Other datasets, such as Easterly et al. (2006) and Eck and Hultman (2007: 132), emphasize one-sided violence and the ‘essential defenselessness and helplessness of the victims.’ Ulfelder and Valentino (2008: 2) define ‘mass killing’ for their State-Sponsored Mass Killing dataset as ‘any event in which the deliberate actions of state agents or other organizations of the state kill at least 1,000 noncombatant civilians from a discrete group.’ Verdeja (2016) provides a useful summary of other recent efforts, including the Early Warning Project, which leverages several indicators to forecast annual country-level risks of mass killing.

Yet there remains tension about what scope is appropriate for atrocities analysis. Krain (1997: 331) contends a broad perspective is most useful, concluding that war, civil war, state-sponsored mass murder, and state terrorism ‘should be studied as related phenomena, using existing theories of conflict and political violence.’ However, narrower definitions may reveal important distinctions. Wayman and Tago (2010), comparing models produced by Harff (2003) and Rummel (1998), conclude that differences in these studies’ results are due to their respective foci on geno-politicide versus democide.

Recent studies of atrocities have expanded into less-lethal actions like torture (Beger and Hill, 2019) and mass expulsion (Garrity, 2022). Additionally, CIRIGHTS compiles indicators on human rights violations and protections, including a human rights index with a dichotomous measure of lethal atrocity presence (Mark et al., 2024). However, the relationship with lethal and other less-lethal atrocity types has been systematically studied in only a limited fashion.¹ With PMAD, we seek to overcome this limitation by incorporating lethal and less-lethal violence into a single, divisible framework. Our data-building process is described next.

Generating the Perceived Mass Atrocities Dataset

We use the following definition of ‘perceived mass atrocity’ for PMAD: ‘an act of violence against 25 or more civilians or otherwise defenseless individuals with the intent of destroying their social, cultural, ethnic, religious, or political group – or intimidating their group by creating a perception of imminent threat to its survival – through systematic or random, planned or unplanned acts by a group of official or unofficial state forces or non-state actors directly or indirectly resulting in death, injury, or widespread damage of property, excluding acts of terrorism that do not involve the pursuit of or threat of group elimination.’ This definition merits several points of clarification, which are included along with variable descriptions in the codebook, included in the Online appendix. Chief among clarifications is our assertion that intent to commit an atrocity may be inferred (Alonzo-Maizlish, 2002), a potentially controversial assumption (Gordon, 2002) that eases the burden of proof. Due to resource constraints, other clarifications – including a 25-victim threshold for an event or campaign,² the exclusion of acts of terrorism that are not in pursuit of or intended to threaten group elimination, and the exclusion of women as a distinct social group for atrocity types – are intended to limit the data to a tractable set of events to code.

We classify atrocities using seven broadly defined types and associated indices, where a single event may fit into one or more categories (e.g. long-term arbitrary detention of 25 or more individuals, coded as mass containment, may also involve torture of 25 or more individuals, coded as mass less-lethal violence). These atrocity types are: mass murder; mass less-lethal violence (including beatings, torture, and sexual violence); mass displacement; mass containment; mass conscription (including use of child soldiers); mass erasure (including both physical and cultural erasure, such as the widespread destruction of religious sites); and reduced citizenship (which involves unequal treatment codified in law, such as legalized racial segregation, but excludes non-codified structural racism, such as disproportionate incarceration rates for marginalized groups).

The relationship between PMAD atrocity types and atrocities indices – developed via a review of extant literature, an iterative process of exploratory coding to assess our ability to identify and quantify atrocities, and an assessment of face validity by atrocity risk assessment practitioners – is presented in Figure 1. See the PMAD codebook in the Online appendix for definitions. For an alternative typology, see Straus (2001).

Figure 1.

PMAD framework for atrocity types and associated indices.

In the PMAD framework, each atrocity type is associated with: an occurrence indicator, which specifies whether there was evidence of an atrocity type in a country-year; a magnitude indicator, which roughly follows a base-10 logarithmic scale, recommended by Gurr (1972: 110–111) to account for uncertainty;³ and a prevalence value, which is an approximate measure of the count of victims per million people residing in a country. Each offers a different perspective on the severity of atrocities in one country-year relative to others. Occurrence and magnitude indicators are multiplied, summed, and minimum-maximum normalized to a 0–100 scale to create country-year Atrocities Scope and Scale Heuristic (ASSH) scores.

Separate from the ASSH, we include two variables to measure the severity of violence against women and LGBTQIA+ groups using a 0–4 scale based on Allport’s (1954) scale for prejudice (see Figure 1), excluding anti-locution. These variables were separated given that coding of violence against these marginalized identities as ‘reduced citizenship’ would substantially wash out variation and merit the maximum magnitude value for most country-years.

When reviewing source materials – an overview of which is provided in Figure 2 and which were gathered via extensive searches of NexisUni, general search engines, and selected government and nongovernmental organization reports – coders first focused on identifying all potential atrocity events for a given country and year.⁴ In this process, they noted key information, such as suspected perpetrator and victim groups and group types, atrocity types, and estimated victim counts. Many atrocities involve multiple actions and can be described using multiple categories. For example, arbitrary detention may be accompanied by torture. Coders were trained to account for these interactions and record them independently if encountered.

Figure 2.

Prominent PMAD sources by organization, % of total.

Coders relied on the source materials’ context to make informed judgements to assign an approximate magnitude for each atrocity type in a country-year, recording substantial uncertainties in the ‘Notes’ column or under the ‘Uncertainty’ variables in the PMAD dataset.⁵ We tested the interrater reliability of our coding and found ‘moderate to substantial’ agreement among coders.⁶ Country-year data are accompanied by country case narratives, which summarize the recent history (typically 2018 onward) of perceived mass atrocities in a country. These narratives, along with source links in the dataset, allow for verification of the coded information and provide additional qualitative details to better understand coding decisions (Williams, 2016).

For example, Al Jazeera reported in 2021 that ‘60 adults and 44 children’ of the Ka’a Poty 1 Indigenous community were ‘forcibly evicted from their land’ in eastern Paraguay and that the community was ‘one of at least seven Indigenous communities to suffer violent evictions by state forces and armed civilians [. . .] in the last three months’ (Costa, 2021). The US State Department’s 2021 human rights report for Paraguay confirmed ‘reports of forced evictions,’ and The Guardian confirmed that ‘at least seven native communities across the country were forcibly removed from their homes,’ linking to a Spanish-language investigative report that over 400 families were victimized (Blair, 2021). These reports were used to code ‘mass displacement occurrence’ as 2 given the multiple occurrences and ‘mass displacement magnitude’ as 4 given our assumption that many of the 400 families comprised more than two people, likely amounting to more than 1,000 victims. Given information in these reports and others (recorded under ‘additional source links’ and the country case narrative) about physical violence, ‘mass less lethal violence’ was coded as well. Because information on this violence was sparser and we deemed it unlikely that all victims were physically attacked, ‘mass less lethal violence magnitude’ was coded as a 3 (500–1,000 victims) but noted as ‘uncertain.’

Descriptive analysis of PMAD

Quantitative comparisons of human suffering are fraught. But, as Meron (2018: 449) has articulated, ‘Authoritative indices based on [atrocities] data, like those used to track corruption perceptions, could [. . .] become a useful tool to track and publicize progress or the lack thereof.’ To this end, Figure 3 depicts the severity of atrocities worldwide in 2022, measured by ASSH scores.

Figure 3.

ASSH scores in 2022.

ASSH scores in several Latin American countries are relatively high – a sharp contrast to a recent Early Warning Project assessment of the risk of mass killings, which suggests such risk is largely concentrated in Africa and Asia.⁷ However, with a few exceptions, namely China, there is a notable visual overlap between ongoing conflict and atrocities, supporting the findings of Valentino et al. (2004). For comparison, Figure 4 provides a map of ongoing conflict in 2022 (Davies et al., 2023; Gleditsch et al., 2002).

Figure 4.

Ongoing conflicts in 2022.

Israel-Palestine (combined as a single territory for our analysis) ranks at the top for perceived atrocities per capita (in millions; Figure 5). The primary atrocity type in Israel-Palestine in 2022 is reduced citizenship experienced by Palestinians (Freedom House, 2022a). This raises important normative questions about the comparability of different atrocities. Does Israel-Palestine in 2022 merit a prevalence-based ranking worse than North Korea in 2022, with its roughly 120,000 political prisoners that are routinely subjected to torture, rape, and extrajudicial execution (Amnesty International, 2022; Freedom House, 2022b; US Department of State, 2022)? Perhaps not, but we presently lack an academic – and normative – basis to weight atrocity types differently. Meanwhile, PMAD’s transparent construction allows users to recalculate an aggregate atrocities index with alternative weights or by selecting specific atrocity types.⁸

Figure 5.

Ranking of 25 highest average ASSH (left) and prevalence (right) scores.

PMAD can also be used to track temporal patterns of atrocities. Figure 6 illustrates changes from 2018 to 2022 for countries with the highest average ASSH scores across that period. The data appear to track changes in atrocities with some accuracy. For instance, the greatly increased scope and scale of atrocities committed on Ukrainian territory, largely though not exclusively by Russian forces, stands out as the largest increase. Iraq stands out with the largest decrease, which was mostly a product of a reduction in (though not cessation of) state-sponsored violence against political protestors and ethnic Kurds (Human Rights Watch, 2023).

Figure 6.

Changes from 2018–22 for countries with the 25 highest average ASSH scores.

As Figure 7 depicts, mass murder, mass less-lethal violence, mass displacement, and mass containment appear to have become slightly more severe and then slightly less so in global average between 2018 and 2022. The rough overlap of these changes with the COVID-19 pandemic merits further exploration but conforms with Grasse et al.’s (2021) findings regarding pandemic-related ‘opportunistic repression.’ From 2018 to 2022, most atrocities were of a less-lethal nature – acts which would be ignored by data focusing only on lethal events and difficult to compare using data focusing only on specific less-lethal events.

Figure 7.

Global average PMAD atrocity type index values from 2018 to 2022.

Figure 8 depicts pooled subregional averages for PMAD’s seven atrocity types. North America stands out for mass displacement due to the refoulement of tens of thousands of migrants at the southern US border. The next highest subregional averages are for: mass displacement in Northern and Central Africa; reduced citizenship in Eastern, Southern, and Western Asia and Eastern Europe; and mass less-lethal violence in Central, Southern, and Southeastern Asia. Analysis of atrocities using data on only lethal atrocities would be inadequate for these regions, given their high average ASSH magnitudes but relatively much smaller scope and scale of mass murder atrocities.

Figure 8.

UN Statistical Division subregion averages (2018–22) by atrocity type index value.

Preliminary statistical analysis and applications

We analyze and validate PMAD by comparing its indices with several hypothesized correlates and related datasets (Butcher et al., 2020; Coppedge et al., 2023; Drazanova, 2020; Mark et al., 2024; US Department of State, 2022). As Table 1 reveals, PMAD’s indices exhibit moderate to strong inverse correlations with quality of life and democracy measures and strong positive correlations with measurements of injustice and violence. Ethnic fractionalization and infant mortality rates – risk factors identified by atrocity prevention practitioners (US Department of State, 2022) – weakly correlate with the total scope and scale of atrocities, as measured by ASSH scores.

Table 1.

Pairwise bivariate correlations between selected PMAD indices and hypothesized correlates.

DV = ASSH	r
GDP per capita^ψ	−0.283
GDP^ψ	0.215
Polyarchy^α	−0.497
Democracy^β	−0.375
Democracy^γ	−0.411
Neopatrimonialism^α	0.489
Clientelism^α	0.334
Government accountability^α	−0.492
Political corruption^α	0.413
Corruption perceptions^δ†	−0.472
Rule of law^ε	−0.575
Human rights index^η	−0.661
Political terror scale, mean of A, H, and S^θ	0.751
Prosperity^κ	−0.477
Ethnic fractionalization^π	0.067
Infant mortality rate^ψ	0.222
Mass containment index	r
Freedom from forced labor^α	−0.468
Mass conscription index	r
Freedom from slavery^α	−0.346
Mass less-lethal violence index	r
Physical violence^α‡	−0.522
Freedom from torture^α	−0.514
Mass murder index	r
Targeted mass killing ongoing^χ	0.218
Brutality-based mass atrocity^η	0.539
Freedom from political killings^α	−0.488
Reduced citizenship index	r
Equality before the law^α	−0.361
Equal protection^α	−0.317
Violence against women severity	r
Freedom from forced labor for women^α	−0.404
Access to justice for women^α	−0.443
Gender equality in respect for civil liberties^α	−0.505
Exclusion by gender^α	0.536
Women civil liberties^α	−0.476
Power distributed by gender^α	−0.459
Violence against LGBTQIA+ severity	r
Power distributed by sexual orientation^α	−0.428

DV = dependent variable; r = correlation coefficient.

V-Dem; ^βPolity; ^γVanhanen; ^δTransparency International; ^εFreedom House; ^ηCIRIGHTS; ^θPTS; ^κLegatum; ^πHIEF; ^χTMK; ^ψWorld Bank.

†

Higher index values = lower corruption perceptions.

‡

Higher index values = increased freedom from torture, political killings.

We also regress ASSH scores on select variables described by the US Department of State’s (2022) Atrocity Risk Assessment Framework. Given the small sample size, we select only three variables from the Framework to account for social fragmentation (proxied by political polarization and women’s political empowerment) and governance (proxied by neopatrimonialism), while also controlling for a country’s total population size.⁹ We also use subregion fixed effects to control for regional conflict dynamics.

A risk-based atrocities assessment using structural factors faces limitations given the omission of trigger events and process-related factors unique to a specific context (McLoughlin, 2016). Still, our preliminary analysis offers an indication of future research avenues. As Table 2 and Figure 9 illustrate, atrocities – assessed both with ASSH and a binary (yes/no) measure of whether atrocities were perceived to have occurred in a given country-year – can be modeled using a handful of independent variables that are of broad interest to political science scholars.

Table 2.

Multivariate regression of ASSH scores on selected variables.

Dependent variable = ASSH	OLS coef. (Std. error)
Population^ϕ	2.89E-08** (3.75E-09)
Neopatrimonialism^α	11.465** (2.923)
Women political empowerment^α	−24.962** (4.407)
Political polarization^α	0.928* (0.378)
Sample size	865
R-squared	0.439
Subregion fixed effects	Yes
Year fixed effects	Yes

p < 0.05; **p < 0.01.

V-Dem; ^ϕUNPD.

Figure 9.

Log-odds of atrocity occurrence regressed on selected variables.

Of the relationships studied here, women’s political empowerment correlated most strongly and negatively with atrocities, with greater political empowerment corresponding with substantially fewer observed atrocities (p < 0.01). This accords with Hudson et al.’s (2020) assertion that societies that treat women more equally generally experience less violence than less equal societies. Future analyses could assess whether improvements in gender equality across time correspond with reductions in atrocities.

Future research could also continue to explore the relationship between governance – a primary risk factor in the US Department of State (2022) Atrocity Risk Assessment Framework – and atrocities. The findings in our preliminary analysis, which illustrate a large, positive relationship between neopatrimonialism and atrocities, offer a potential extension to Erdmann and Engel’s (2007) connection between neopatrimonialism and insecurity. Further investigation would add to the growing literature on personalist dictatorship and the role of institutionalization (even in autocracies) in reducing the severity of state-led violence against civilians.

We see a much smaller, though statistically significant relationship between political polarization and atrocities. Could the type of polarization or speed with which a political system becomes polarized affect the propensity for groups to commit atrocities? Does political polarization lead to atrocities against ethnic or other identity groups that have become politicized by fear-based political narratives, as with terrorism (McAlexander, 2020)? What may this mean for increasingly polarized US politics (Lewis et al., 2023)?

The future of mass atrocities research

PMAD is, to our knowledge, the first and presently only global dataset on lethal and less-lethal atrocities contained within a single, consistent framework for quantitative analysis with supporting qualitative information in the form of country case narratives. The data highlight several regions where analysis of atrocities using data on only lethal atrocities may be inadequate, especially Central and Eastern Asia, given high magnitudes for less-lethal atrocities and relatively much lower magnitudes for lethal atrocities. All told, we expect PMAD will be of use for the fields of human rights and atrocity prevention as well as political scientists more broadly, including those studying gender inequality, personalist rule, and political polarization.

Footnotes

Acknowledgements

Special thanks to Mia Grant, Alexandra Brodsky, Juliet Wishner, Haylie Castor, Mary Cates, Henry Heilbroner, Pam Hoberman, Bill McDonald, Ryan Retzlaff, MacKenzie Roth, Henry Valuck, Bailey Yamshak, Camryn Hoelle, our three anonymous reviewers, and the JPR editors.

Replication data

The dataset, codebook with narratives, and do-file for our analysis are available at . Figures were created in Datawrapper and Stata/SE 15.1.

Funding

PMAD was created with funding from the United States Government (USG). Data and narratives for the US were separately generated with funding from a University of Denver faculty research grant. The results and views expressed are those of the authors alone and do not represent the views of the USG.

ORCID iDs

Collin J Meisel

Jonathan D Moyer

Austin S Matthews

Oliver Kaplan

Notes

COLLIN J MEISEL, MPP in Public Policy (Georgetown University, 2018); Associate Director of Geopolitical Analysis, Pardee Institute for International Futures (2018–present).

JONATHAN D MOYER, PhD in International Studies (University of Denver, 2012); Assistant Professor, University of Denver (2015–present); Director, Pardee Institute for International Futures (2015–present).

AUSTIN S MATTHEWS, PhD in Political Science (Louisiana State University, 2018); Assistant Professor, East Carolina University (2022–present).

OLIVER KAPLAN, PhD in Political Science (Stanford University, 2010); Associate Professor, University of Denver (2013–present).

RUTH BYRNES, MA in International Human Rights (University of Denver, 2022); Senior Evaluation and Research Associate, International Republican Institute (2023–present).

KERENT BENJUMEA, MA in International Studies (University of Denver, 2023); Research Aide Team Lead, Pardee Institute for International Futures (2022–23).

PHOEBE CRIBB, BA in Political Science (Bryn Mawr College, 2020); Research Aide Team Lead, Pardee Institute for International Futures (2022–present).

COLLIN VAN SON, BS in Physics (Pennsylvania State University, 2018); Research Aide, Pardee Institute for International Futures (2023–23).

References

Ackerman

Pinson

(2016) Speaking the truth to sources: Introducing a method for the quantitative evaluation of open sources in event data. Studies in Conflict & Terrorism 39(7-8): 617–640.

Allport

(1954) The Nature of Prejudice. Cambridge: Addison-Wesley.

Alonzo-Maizlish

(2002) In whole or in part: Group rights, the intent elements of genocide, and the ‘quantitative criterion’. New York University Law Review 77(5): 1369–1403.

Amnesty International (2022) North Korea. Amnesty International. Available at: https://www.amnesty.org/en/location/asia-and-the-pacific/east-asia/north-korea/report-korea-democratic-peoples-republic-of/ (accessed 18 September 2023).

Bagozzi

Koren

(2017) Using machine learning methods to identify atrocity perpetrators. In: IEEE International Conference, Boston, MA, 21–25 May.

Beger

Hill

Jr (2019) Examining repressive and oppressive state violence using the Ill-treatment and Torture data. Conflict Management and Peace Science 36(6): 626–644.

Blair

(2021) Indigenous community evicted as land clashes over agribusiness rock Paraguay. The Guardian, 21 November. Available at: https://www.theguardian.com/global-development/2021/nov/21/paraguay-evictions-land-indigenous-agribusiness (accessed 27 October 2023).

Brennan

Prediger

(1981) Coefficient kappa: Some uses, misuses, and alternatives. Educational and Psychological Measurement 41(3): 687–699.

Butcher

Goldsmith

Nanlohy

, et al. (2020) Introducing the targeted mass killing data set for the study and forecasting of mass atrocities. Journal of Conflict Resolution 64(7–8): 1524–1547.

10.

Butcher

Goldsmith

(2016) Economic risk factors and predictive modeling of genocides and other mass atrocities. In: Anderton

Brauer

(eds) Economic Aspects of Genocides, Other Mass Atrocities, and Their Preventions. New York, NY: Oxford Academic, 569–590.

11.

Coppedge

Gerring

Knutsen

, et al. (2023) V-Dem codebook v13. Varieties of Democracy (V-Dem) Project. Available at: https://v-dem.net/data/ (accessed 18 September 2023).

12.

Cordell

Clay

Fariss

, et al. (2022) Disaggregating repression: Identifying physical integrity rights allegations in human rights reports. International Studies Quarterly 66(2): sqac016.

13.

Costa

(2021) Paraguay indigenous community evicted in land dispute. Al Jazeera, 30 July. Available at: https://www.aljazeera.com/news/2021/7/30/paraguay-indigenous-community-evicted-in-land-dispute (accessed 27 October 2023).

14.

Davies

Pettersson

Oberg

(2023) Organized violence 1989–2022 and the return of conflicts between states? Journal of Peace Research 60(4): 691–708.

15.

Demarest

Langler

(2022) How events enter (or not) data sets: The pitfalls and guidelines of using newspapers in the study of conflict. Sociological Methods & Research 51(2): 632–666.

16.

Drazanova

(2020) Introducing the Historical Index of Ethnic Fractionalization (HIEF) dataset: Accounting for longitudinal changes in ethnic diversity. Journal of Open Humanities Data 6(1): 6.

17.

Dumaine

Gargiulo

Norda

, et al. (2022) Repertoires of conflict-related sexual violence: Introducing the RSVAC data package. Journal of Peace Research 59(4): 611–621.

18.

Easterly

Gatti

Kurlat

(2006) Development, democracy, and mass killing. Journal of Economic Growth 11(2): 129–156.

19.

Eck

(2012) In data we trust? A comparison of UCDP GED and ACLED conflict events datasets. Cooperation and Conflict 47(1): 124–141.

20.

Eck

Hultman

(2007) One-sided violence against civilians in war: Insights from new fatality data. Journal of Peace Research 44(2): 233–246.

21.

Erdmann

Engel

(2007) Neopatrimonialism reconsidered: Critical review and elaboration of an elusive concept. Commonwealth and Comparative Politics 45(1): 95–119.

22.

Freedom House (2022a) Israel: Freedom in the World 2022 Country Report. Freedom House. Available at: https://freedomhouse.org/country/israel/freedom-world/2022 (accessed 18 September 2023).

23.

Freedom House (2022b) North Korea: Freedom in the World 2022 Country Report. Freedom House. Available at: https://freedomhouse.org/country/north-korea/freedom-world/2022 (accessed 18 September 2023).

24.

Ganor

(2002) Defining terrorism: Is one man’s terrorist another man’s freedom fighter? Police Practice and Research 3(2): 287–304.

25.

Garrity

(2022) Introducing the government-sponsored mass expulsion dataset. Journal of Peace Research 59(5): 767–776.

26.

Gibney

Haschke

Arnon

, et al. (2023) The political terror scale 1976–2022. Available at: https://www.politicalterrorscale.org/Data/Download.html (accessed 26 October 2023).

27.

Gleditsch

Wallensteen

Eriksson

, et al. (2002) Armed conflict 1946–2001: A new dataset. Journal of Peace Research 39(5): 615–637.

28.

Goldsmith

Butcher

Semenovich

, et al. (2013) Forecasting the onset of genocide and politicide: Annual out-of-sample forecasts on a global dataset, 1988–2003. Journal of Peace Research 50(4): 437–452.

29.

Gordon

(2002) When intent makes all the difference in the world: Economic sanctions on Iraq and the accusation of genocide. Yale Human Rights and Development Law Journal 5: 57–84.

30.

Grasse

Pavlik

Matfess

, et al. (2021) Opportunistic repression: Civilian targeting by the state in response to COVID-19. International Security 46(2): 130–165.

31.

Gurr

(1972) Politimetrics: An Introduction to Quantitative Macropolitics. Englewood Cliffs, NJ: Prentice-Hall.

32.

Gwet

(2008) Computing inter-rater reliability and its variance in the presence of high agreement. British Journal of Mathematical and Statistical Psychology 61(May): 29–48.

33.

Harff

(1992) Recognizing genocides and politicides. In: Fein

(ed.) Genocide Watch. New Haven, CT: Yale University Press, 27–44.

34.

Harff

(2003) No lessons learned from the Holocaust? Assessing risks of genocide and political mass murder since 1955. American Political Science Review 97(1): 57–73.

35.

Harff

Gurr

(1988) Toward empirical theory of genocides and politicides: Identification and measurement of cases since 1945. International Studies Quarterly 32(3): 359–371.

36.

Harff

Gurr

(1998) Systemic early warning of humanitarian emergencies. Journal of Peace Research 35(5): 551–579.

37.

Harrell

Jr (2001) Regression Modeling Strategies. New York, NY: Springer.

38.

Hirsch

MBJ

Dixon

(2021) Conceptualizing and assessing norm strength in international relations. European Journal of International Relations 27(2): 521–547.

39.

Hudson

Bowen

Nielsen

(2020) The First Political Order: How Sex Shapes Governance and National Security Worldwide. New York, NY: Columbia University Press.

40.

Human Rights Watch (2023) World report 2023: Iraq, events of 2022. Available at: https://www.hrw.org/world-report/2023/country-chapters/iraq (accessed 18 September 2023).

41.

International Criminal Court (2013) Elements of Crimes. The Hague: International Criminal Court.

42.

Kaplan

(2022) War crimes, atrocities, and resistance in Colombia. In: Hola

Nzitatira

Weedesteign

(eds) The Oxford Handbook on Atrocity Crimes. New York, NY: Oxford University Press, 801–826.

43.

Keeney

(1996) Value-Focused Thinking: A Path to Creative Decisionmaking. Cambridge, MA: Harvard University Press.

44.

Krain

(1997) State-sponsored mass murder: The onset and severity of genocides and politicides. Journal of Conflict Resolution 41(3): 331–360.

45.

Landis

Koch

(1977) The measurement of observer agreement for categorical data. Biometrics 33(1): 159–174.

46.

Lewis

Poole

Rosenthal

, et al. (2023) Voteview: Congressional roll-call votes database. Available at: https://voteview.com/ (accessed 18 September 2023).

47.

McAlexander

(2020) How are immigration and terrorism related? An analysis of right- and left-wing terrorism in Western Europe, 1980–2004. Journal of Global Security Studies 5(1): 179–195.

48.

McLoughlin

(2016) From reaction to resilience in mass atrocity prevention: An analysis of the 2013 UN report ‘The responsibility to protect: State responsibility and prevention’. Global Governance 22(4): 473–489.

49.

Mark

Cingranelli

Filippov

(2024) CIRIGHTS: Quantifying respect for all human rights. Human Rights Quarterly. 46(2): 264–286.

50.

Meron

(2018) Closing the accountability gap: Concrete steps toward ending impunity for atrocity crimes. American Journal of International Law 112(3): 433–451.

51.

Ralph

O’Neill

Winton

(2015) A Practical Introduction to Index Numbers. Oxford, UK: Wiley.

52.

Raymond

(2013) Mass atrocity prevention and response options: Addressing the policy challenges. In: Franke

Dorff

(eds) Conflict Management and Peacebuilding: Pillars of a New American Grand Strategy. Carlisle, PA: Strategic Studies Institute, US Army War College, 295–316.

53.

Rost

(2013) Will it happen again? On the possibility of forecasting the risk of genocide. Journal of Genocide Research 15(1): 41–67.

54.

Rummel

(1998) Statistics of Democide: Genocide and Mass Murder Since 1900. Münster, Germany: LIT.

55.

Savelsberg

(2018) Genocide and other atrocity crimes: Toward remedies. In: Muschert

Budd

Christian

, et al (eds) Global Agenda for Social Justice: Volume One. Bristol: Bristol University Press, 111–120.

56.

Scheffer

(2006) Genocide and atrocities crimes. Genocide Studies 1(3): 229–250.

57.

Schrodt

Ulfelder

(2016) Political Instability Task Force Atrocities Event Data Version 1.1b1. The Computational Event Data System. Available at: http://eventdata.parusanalytics.com/data.dir/atrocities.html (accessed 18 September 2023).

58.

Straus

(2001) Contested meanings and conflicting imperatives: A conceptual analysis of genocide. Journal of Genocide Research 3(3): 349–375.

59.

Ulfelder

Valentino

(2008) Assessing risks of state-sponsored mass killing. SSRN. Available at: https://ssrn.com/abstract=1703426 (accessed 18 September 2023).

60.

United Nations (1998) Rome Statute of the International Criminal Court. Rome: United Nations.

61.

United Nations General Assembly (2005) 2005 World Summit Outcome, A/RES/60/1. New York, NY: United Nations.

62.

US Department of State (2022) US atrocity risk assessment framework. US Department of State, 21 December. Available at: https://www.state.gov/u-s-atrocity-risk-assessment-framework/#:~:text=The%20Atrocity%20Risk%20Assessment%20Framework,respond%20to%2C%20and%20mitigate%20atrocities (accessed 18 September 2023).

63.

Valentino

Huth

Balch-Lindsay

(2004) ‘Draining the sea’: Mass killing and guerilla warfare. International Organization 58(2): 375–407.

64.

Verdeja

(2016) Predicting genocide and mass atrocities. Genocide Studies and Prevention: An International Journal 9(3): 13–32.

65.

Wayman

Tago

(2010) Explaining the onset of mass killing, 1949–87. Journal of Peace Research 47(1): 3–13.

66.

Williams

(2016) More lessons learned from the Holocaust: Towards a complexity-embracing approach to why genocide occurs. Genocide Studies and Prevention: An International Journal 9(3): 137–153.