Measurement Invariance and Convergent Validity of Anger and Sadness Self-Regulation Among Youth From Six Cultural Groups

Abstract

The present study examined measurement invariance and convergent validity of a novel vignette-based measure of emotion-specific self-regulation that simultaneously assesses attributional bias, emotion-regulation, and self-efficacy beliefs about emotion regulation. Participants included 541 youth–mother dyads from three countries (Italy, the United States, and Colombia) and six ethnic/cultural groups. Participants were 12.62 years old (SD = 0.69). In response to vignettes involving ambiguous peer interactions, children reported their hostile/depressive attribution bias, self-efficacy beliefs about anger and sadness regulation, and anger/sadness regulation strategies (i.e., dysregulated expression and rumination). Across the six cultural groups, anger and sadness self-regulation subscales had full metric and partial scalar invariance for a one-factor model, with some exceptions. We found support for both a four- and three-factor oblique model (dysregulated expression and rumination loaded on a second-order factor) for both anger and sadness. Anger subscales were related to externalizing problems, while sadness subscales were related to internalizing symptoms.

Keywords

emotion regulation attribution bias self-efficacy measurement invariance cross-cultures

Self-regulation in childhood and adolescence has figured prominently into scientific inquiries about factors that contribute to psychological adjustment and well-being likely because both positive and negative adjustment in adulthood are predicted by consequences of self-regulation in childhood (Ayduk et al., 2000; Rutter, 2011). The ways children interpret others’ intentions in social exchanges, children’s beliefs about the capacity to deal with emotions, and the strategies children use to manage emotions, all have been significantly correlated with psychological maladjustment (e.g., Eisenberg, Spinrad, & Eggum, 2010; Orobio de Castro, Veerman, Koops, Bosch, & Monshouwer, 2002), and are often the focus of mental health interventions (Caprara et al., 2014; Conduct Problems Prevention Research Group, 1992). To expand the scientific evidence on self-regulation, more cross-cultural research is needed, as are evidence-based assessments that tap novel components of self-regulation. This study evaluates a measure of self-regulation in preadolescence that assesses self-regulation across multiple domains (i.e., cognitive and emotional) in reaction to social exchanges. The primary goals of this study were to investigate the structural properties of this measure across culturally diverse preadolescent samples and to examine its construct validity in relation to empirically validated measures and mental health symptoms. We focus our investigation on preadolescents because of the importance of identifying precursors of risk and resiliency at key developmental “switchpoints” such as adolescence (Dodge & Pettit, 2003). Evidence is clear that adolescence is associated with increases in the prevalence of behavioral and mental health issues (e.g., Cicchetti & Toth, 1998; Silk, Steinberg, & Morris, 2003). Better understanding self-regulation in preadolescence could advance our knowledge of key targets for prevention and early intervention strategies seeking to thwart the adolescent onset of behavioral and mental health issues.

Self-regulation encompasses a broad array of internal and external processes that are recruited in the service of goal-directed pursuits (Karoly, 1993). These processes include the management of one’s cognitions, attentional allocations, emotions, and behaviors (Karoly, 1993). Importantly, many self-regulation processes are believed to be under the voluntary control of the individual though they are not always controlled consciously (Eisenberg & Spinrad, 2004). Self-regulation processes are also psychosocial in nature, being flexibly initiated moment-to-moment within ever-changing social contexts (Morf & Mischel, 2002). As such, the interpersonal context is crucial for fully understanding the functions and outcomes of self-regulation. Culture plays a prominent role in defining guidelines and expectations for what is adaptive and maladaptive within daily interpersonal exchanges. Culture therefore likely provides the foundational framework for the development and implementation of self-regulation (Yasui & Dishion, 2007). It is thus crucial to understand whether measures of self-regulation are invariant across cultures. The current study assessed the cross-cultural invariance of three self-regulation processes: attribution biases, emotion regulation, and self-efficacy beliefs about emotion regulation.

Attribution Biases From Social Information Processing

Attributions of intent require the interpretation of social cues and identification of the reasons behind someone’s activities within a social exchange. In ambiguous social situations, attributional biases often arise. Some attributional biases may be self-blaming, depressive attribution biases, and others may be other-blaming, hostile attribution biases. Concurrent and longitudinal evidence suggests that negative self-blaming attribution biases are related to depressive symptoms (e.g., Luebbe, Bell, Allwood, Swenson, & Early, 2010; Prinstein, Cheah, & Guyer, 2005), whereas hostile other-blaming attribution biases are related to externalizing symptoms (e.g., Orobio de Castro et al., 2002).

Attribution biases are universally applicable characteristics (Dodge, 2006), although much of the evidence on attribution biases and their external correlates are based on samples from the United States. A few intriguing studies have been conducted in other countries. These studies suggest that hostile attribution biases are related to aggression among German adolescents (Möller & Krahé, 2009) and Dutch children (Reijntjes et al., 2011). Although these studies offer support for the universality of attribution biases, they do not make cross-cultural comparisons of the equivalence of attribution biases.

Few researchers have examined attribution biases using cross-cultural designs. Dodge et al. (2015) found that children’s mean levels of hostile attribution biases varied across 12 ethnic groups and that levels of hostile attribution biases mediated the relation between cultural group and aggressive behavior. This study highlights the importance of examining hostile attribution biases across cultures, yet to our knowledge, there is no existing cross-cultural study solely investigating the psychometric invariance of either hostile or depressive attribution biases. The present study fills this gap, providing crucial psychometric evidence necessary for future cross-cultural work on both hostile and depressive attribution biases.

Emotion Regulation

Emotion-related self-regulation is a burgeoning arena for understanding the multifaceted nature of self-regulation. Eisenberg and colleagues (e.g., Eisenberg & Spinrad, 2004) define emotion-related self-regulation as voluntary modulation of the presence, nature, strength, and extent of emotion-related experiences via activation, avoidance, suppression, and/or maintenance. Emotion-related experiences encompass emotion-focused internal and external processes, including internal sensations; physical, cognitive, and motivational states; and/or behavioral corollaries. Modulation of these emotion-related experiences occurs to achieve bio-psycho-social adjustment and personal objectives.

Emotion regulation is a broad construct composed of specific strategies, such as emotion awareness (Halberstadt, Denham, & Dunsmore, 2001), emotional clarity (Gratz & Roemer, 2004), controlling impulsive emotional responses (Cole, Michel, & O’Donnell, 1994), controlling rumination (Caprara, 1986; Nolen-Hoeksema, 1991), and many other strategies. The current article focuses on two types of emotion regulation: controlling impulsive emotional responses and controlling rumination. Failures in these emotion regulation strategies lead to dysregulated expressions of emotions and rumination, respectively. Dysregulated expression includes the over- or undercontrol of one’s emotions (Eisenberg et al., 2010). Rumination involves passively dwelling on one’s feelings and thoughts associated with negative events (Nolen-Hoeksema, Wisco, & Lyubomirsky, 2008). As we review empirical evidence on emotion regulation, we report findings on the broad construct of emotion regulation, highlighting these strategies where relevant.

There is clear empirical support for relations between emotion regulation and psychological adjustment in childhood and adolescence (see Eisenberg et al., 2010, for a review). The successful regulation of emotions has been concurrently associated with lower levels of internalizing symptoms (e.g., Silk, Shaw, Forbes, Lane, & Kovacs, 2006; Yap, Allen, & Sheeber, 2007) and both concurrently and longitudinally associated with lower levels of externalizing symptoms (e.g., Chaplin, Cole, & Zahn-Waxler, 2005; Eisenberg et al., 2009). Relations between emotion regulation and internalizing symptoms are less robust and potentially less stable across age than relations between emotion regulation and externalizing symptoms (Eisenberg et al., 2010).

Recent research differentiates specific emotions that fall under the broader construct of negative emotionality (e.g., Caprara, Di Giunta, Pastorelli, & Eisenberg, 2012; Zimmerman & Iwanski, 2014). Anger and sadness have received the greater part of this empirical attention in child and adolescent samples. There is strong evidence that failure to regulate anger is related to externalizing symptoms (e.g., Morris et al., 2011; Zeman, Shipman, & Suveg, 2002) and possibly internalizing symptoms (Eisenberg et al., 2009; Zeman et al., 2002). Failure to regulate sadness has been prospectively linked to depression (Feng et al., 2009) as well as to externalizing and internalizing symptoms (e.g., Eisenberg et al., 2009; Folk, Zeman, Poon, & Dallaire, 2014). Some research even suggests that specific self-regulation processes are more or less effective depending on the specific emotion being examined (Rivers, Brackett, Katulak, & Salovey, 2007; Waters & Thompson, 2014). For example, hostile rumination has been associated with feelings of anger and violent and/or aggressive behaviors (Bushman, 2002; Caprara, Paciello, Gerbino, & Cugini, 2007). Likewise, research has found that depressive rumination increases feelings of sadness and consequently depression (Nolen-Hoeksema et al., 2008). Overall, research evidence highlights the importance of examining specific emotions when investigating emotion regulation because each might have a unique amalgamation of psychological outcomes.

The vast majority of research on emotion regulation has been conducted within the United States, with few notable exceptions. Rydell, Berlin, and Bohlin (2003) found significant negative relations between emotion regulation and internalizing and externalizing symptoms measured approximately 1½ years later in European samples. Moreover, a construct involved in emotion-related regulation (i.e., effortful control—the effortful, self-regulatory aspect of temperament) has been associated with externalizing symptoms among European adolescents (e.g., Hofer, Eisenberg, & Reiser, 2010; Oldehinkel, Hartman, Ferdinand, Verhulst, & Ormel, 2007). There is also some support for measurement invariance of emotion-regulation-related measures (i.e., effortful control) across three ethnic groups of children in the United States (i.e., European Americans, African Americans, and Hispanics; Sulik et al., 2010). There is at least preliminary support for the notion that emotion regulation processes are applicable and have similar outcomes across cultures. However, further research is necessary to support the use of emotion-regulation-related measures in preadolescent samples across multiple cultural contexts.

Self-Efficacy Beliefs About Emotion Regulation

Much of the emotion regulation research focuses on what individuals actually do or report that they would do to modulate their affective experiences, yet it is equally important to understand what individuals believe themselves to be capable of doing. Self-efficacy beliefs tap this related yet distinct aspect of self-regulation (Bandura, 2006). Self-efficacy beliefs about emotion regulation measure how well people believe they can control all aspects of an emotional experience, including exerting control over the origins and intensity of, reactions to, and consequences of one’s own emotions (Bandura, Caprara, Barbaranelli, Gerbino, & Pastorelli, 2003).

Poor self-efficacy beliefs about the regulation of negative emotions have been related to depressive feelings and delinquency (Bandura et al., 2003). Evidence suggests that self-efficacy beliefs about emotion regulation might be emotion specific. Among Italian young adults, Caprara et al. (2008) found that self-efficacy beliefs about regulating anger were better predictors of irritability and aggression, whereas self-efficacy beliefs about regulating sadness were better predictors of shyness and depression.

There is some empirical support for measurement invariance of self-efficacy beliefs in dealing with anger and sadness among young adults from three different countries (i.e., Italy, the United States, and Bolivia; Caprara et al., 2008). Because much of the work on self-efficacy beliefs about emotion regulation has been conducted with young adults, more research with adolescent samples from multiple countries is needed to advance our cross-cultural understanding of self-efficacy beliefs about emotion regulation.

Self-Regulation as Composed of Attributions, Emotion Regulation, and Self-Efficacy Beliefs

The current study tests a unique model of the association between three self-regulation-related mechanisms: attributions, emotion regulation, and self-efficacy beliefs. Our rationale for modeling self-regulation as composed of these three processes is twofold. First, each process maps onto the defining components of self-regulation. Karoly (1993) posited that self-regulation includes the management of one’s cognitions, attentional allocation, emotions, and behaviors. Attribution biases pertain to one’s cognitions; emotion regulation pertains to one’s emotions and their expression (behavior); and self-efficacy beliefs pertain to one’s cognitions and behaviors. Second, empirical evidence indicates that these processes are interrelated. Emotions are related to children’s attributions of intent (Nelson & Coyne, 2009) and individuals are more prone to attribution biases when emotionally aroused (Dodge & Somberg, 1987). There is also evidence that self-efficacy beliefs about emotion regulation are related to emotion regulation. Self-efficacy beliefs about dealing with anger and sadness are positively related to choosing adaptive emotion regulation coping strategies (Gunzenhauser et al., 2013). These interrelations suggest these three processes might be tapping a similar construct (i.e., self-regulation). We therefore believe there is theoretical and empirical support for investigating the notion that self-regulation includes, among other processes, attributions, emotion regulation, and self-efficacy beliefs.

To our knowledge, current measures pertaining to self-regulation-related processes are compartmentalized, measuring only one facet of relevant processes for one emotion and without a specific reference to the social context surrounding the self-regulation (e.g., peer interactions). There is a need for a valid measure of multiple aspects of self-regulation that pertains to the regulation of more than just one emotion and is assessed within the context of social situations. The need for standardizing the social context when assessing self-regulation-related processes is especially important. People vary in terms of the kinds of situations they think about when reporting on their self-regulation. The use of social vignettes standardizes the context of their reporting. The self-regulation assessment literature would be enhanced by data on the measurement invariance of a tool that simultaneously taps attribution biases, emotion regulation, and self-efficacy beliefs about emotion regulation in multiple ethnic and national groups.

Current Study

Using a sample of preadolescents, this study examined the validity of a newly developed vignette-based measure of self-regulation. This measure is composed of subscales tapping four constructs—attribution bias, dysregulated expression of emotion, rumination, and self-efficacy beliefs about emotion regulation—measured across both anger and sadness, resulting in eight subscales total. First, we examined measurement invariance of a monofactorial model of each subscale separately for anger self-regulation (four subscales) and sadness self-regulation (four subscales) across three countries and six cultural groups: Italy (two sites: Rome and Naples), the United States (three ethnic groups: African American, European American, and Hispanic American), and Colombia. Evidence reviewed above suggests that the constructs tapped by this measure are likely applicable across cultures. We therefore predicted similarities in the structure (if not the intercepts) across the six groups. We further examined the latent structure of this measure using an aggregated total sample (i.e., collapsing across the six cultural groups). We tested two competing models separately for anger and sadness (four models total). In Model 1, items designed to assess attribution bias, rumination, dysregulated expression of emotion, and self-efficacy beliefs in dealing with emotions loaded on separate and related first-order factors. In Model 2, the first-order factors for dysregulated expression and rumination loaded on a second-order factor for emotion dysregulation. Model 2 accounted for the fact that dysregulated expression of emotions and rumination are both forms of the broader construct of emotion dysregulation.

Finally, to assess convergent validity, we examined the extent to which the eight subscales were related to empirically established measures of similar constructs. We expected positive correlations between our subscales and existing measures of similar constructs. To further evaluate convergent validity, we examined correlations between our eight subscales and mental health symptoms. We expected hostile attribution bias, dysregulated expression of anger, and hostile rumination to be positively related, and self-efficacy beliefs about anger regulation to be negatively related, to externalizing problems. We also expected depressive attribution biases, dysregulated expression of sadness, and depressive rumination to be positively related, and self-efficacy beliefs about sadness regulation to be negatively related, to internalizing problems.

Method

Participants

Participants were recruited from the longitudinal Parenting Across Cultures Study (e.g., Lansford et al., 2014). An overarching goal was to include cultural groups that were diverse on several sociodemographic dimensions, including predominant ethnicity, predominant religion, economic indicators, and indices of child well-being. For example, on the Human Development Index, a composite indicator of a country’s status with respect to health, education, and income, participating countries had a rank of 5 for the United States, 28 for Italy, and 98 for Colombia, out of 187 countries with available data (United Nations Development Programme, 2012). However, it is important to note that our samples were convenience samples and were therefore not nationally representative.

Data for the present study included measures administered 4 years after initial recruitment into the study, at which time 87% of the original sample provided data. Participants included 541 children (age range = 10-14 years, M = 12.57, SD = 0.69; 50% female) and their mothers (n = 541). Families were drawn from Medellín, Colombia (n = 88); Naples, Italy (n = 90); Rome, Italy (n = 100);¹ and Durham, North Carolina, the United States (n = 97 European Americans, n = 90 African Americans, n = 76 Hispanic Americans). Participants were recruited through letters sent from schools. To ensure economic diversity, we included students from private and public schools and from high- to low-income families, sampled in proportions representative of each site. Nearly all adult respondents were biological mothers, with 3% being grandmothers, stepmothers, or other female adults. Overall, maternal reporters had 12 years of education (top section of Table 1).

Table 1.

Sample Descriptives and Descriptive Statistics by Site for Anger and Sadness Self-Regulation Subscales.

	Italy—Naples (n = 90)	Italy–Rome (n = 100)	USA—European American (n = 98)	USA–African American (n = 90)	USA—Hispanic American (n = 76)	Colombia (n = 88)
Maternal years of education	10.60 (4.51)	13.50 (3.93)	16.71 (2.99)	13.68 (2.26)	10.38 (3.85)	10.29 (5.12)
Females	56%	47%	41%	53%	50%	53%
Youth age	12.36 (0.49)	12.34 (0.77)	12.88 (0.57)	12.82 (0.63)	12.73 (0.75)	12.30 (0.63)
Hostile attribution bias	3.34 (0.76) [.73]	3.17 (0.57) [.48]	3.08 (0.58) [.69]	3.36 (0.69) [.60]	3.32 (0.60) [.56]	3.49 (0.66) [.57]
Self-efficacy beliefs about anger	3.35 (0.69) [.73]	3.14 (0.67) [.74]	3.57 (0.70) [.79]	3.44 (0.75) [.69]	3.40 (0.74) [.80]	3.55 (0.66) [.75]
Dysregulated anger expression	1.65 (0.83) [.89]	1.65 (0.77) [.85]	1.85 (0.77) [.83]	2.36 (0.96) [.81]	2.15 (0.94) [.88]	1.35 (0.51) [.69]
Hostile rumination	3.07 (0.88) [.76]	2.83 (0.98) [.86]	2.46 (0.88) [.86]	2.85 (0.87) [.76]	2.82 (0.90) [.85]	3.04 (1.03) [.86]
Depressive attribution bias	2.84 (0.72) [.73]	2.81 (0.65) [.69]	2.70 (0.61) [.75]	2.94 (0.72) [.69]	2.89 (0.62) [.66]	3.01 (0.76) [.75]
Self-efficacy beliefs about sadness	3.57 (0.66) [.73]	3.57 (0.68) [.77]	3.85 (0.66) [.79]	3.79 (0.75) [.71]	3.72 (0.66) [.75]	3.70 (0.61) [.74]
Dysregulated sadness expression	1.72 (0.61) [.70]	1.66 (0.61) [.79]	1.68 (0.58) [.77]	1.72 (0.74) [.80]	1.76 (0.65) [.79]	1.91 (0.85) [.80]
Depressive rumination	2.95 (0.93) [.81]	2.80 (0.98) [.85]	2.35 (0.88) [.86]	2.51 (0.83) [.70]	2.61 (0.87) [.81]	2.95 (1.04) [.84]

Note. Standard deviations are in parentheses and αs are in brackets.

Measures

Primary Measure Being Validated: Anger and Sadness Self-Regulation Scale

The current study sought to validate a measure we created to simultaneously assess attribution biases, emotion dysregulation, and self-efficacy beliefs about emotion regulation. We created stories that present youth with ambiguous social situations involving peer interactions because of the salience of peers at this age (Prinstein et al., 2005) and the importance of situational specificity in assessing self-regulation-related processes (Crick & Dodge, 1994).

Measure development and pilot testing

We used a three-stage process (Gould, 1996) to create vignettes. First, we derived our scenarios from existing social-information-processing vignettes that have extensive empirical support (Crick & Dodge, 1996). Second, a panel of six experts on attributions, emotion regulation, and self-efficacy beliefs reviewed the stories, offering feedback and revisions. Finally, stories were pilot tested with five children from the United States, seven children from Rome, and three children from Naples.² Pilot participants offered feedback on (a) feasibility and believability of scenarios, (b) clarity of questions asked, and (c) factors that may have affected their responses.

Final measure

The final measure included six vignettes. For each vignette, youths answered questions about (a) why the peers acted the way they did (i.e., attribution biases; “How likely is it that the kids acted that way because they wanted to be mean to you?”), (b) how likely it was that they would engage in certain behaviors pertaining to dysregulated emotion expression (e.g., “You would angrily yell, say mean things, or hit, kick, or slam something”) and rumination (e.g., “You would keep thinking and thinking what went wrong and how angry you feel”), and (c) how well they could deal with their emotion (e.g., “If you were in this situation, how well could you deal with your anger?”). Youths answered the same set of questions for both sadness and anger across all vignettes (counterbalanced across stories). Response options ranged from 1 (not at all likely; not at all sad/angry; not at all well) to 5 (very likely; very sad/angry; very well). This measure has eight subscales: (a) hostile attribution bias, (b) dysregulated expression of anger, (c) hostile rumination, (d) self-efficacy beliefs about anger regulation, (e) depressive attribution bias, (f) dysregulated expression of sadness, (g) depressive rumination, and (h) self-efficacy beliefs about sadness regulation. Total scores for each subscale were created by averaging item responses across the six vignettes. With few exceptions, reliability was adequate to very good across all samples (see bottom section of Table 1).

Convergent Constructs and Measures

Hostile attribution bias

Youth were read 10 vignette-based stories portraying hypothetical ambiguous social situations in which a peer provokes him or her (see Dodge et al., 2015, for more details). Youth indicated whether the peer’s behavior was an accident (coded as 0) or done on purpose (coded as 1). This measure was administered 1 year prior to the current study and includes several of the same stories as those in the measure we are validating (M α across the six sites = .63).

Dysregulated expression of anger and sadness

Youths rated (1 = hardly ever to 3 = often) six items from the Children’s Anger Management Scale (Zeman et al., 2002; dysregulated expression of anger and anger coping subscales; e.g., “I attack whatever it is that makes me very angry” and “I stay calm and keep my cool when I am feeling mad”). Children also rated six items from the Children’s Sadness Management Scale (Zeman et al., 2002; dysregulated expression of sadness and sadness coping subscales; e.g., “I whine/fuss about what is making me sad” and “I can stop myself from losing control of my sad feelings”). One Children’s Sadness Management Scale item (i.e., “I do things like mope around when I am sad”) was dropped only in Colombia due to a very low item-scale correlation (i.e., r = −.04). Total scores comprise the mean of the dysregulated expression and coping (reverse-scored) subscales separately for anger and sadness. Mean reliability across the six sites was α = .66, for anger and α = .55, for sadness. Given the low reliability for sadness, we conducted a factor analysis and found that each of the indicators loaded on the factor of dysregulated expression of sadness at greater than the recommended .30 cutoff (values ranged from .48 to .56; Tabachnick & Fidell, 2001), suggesting these items are interrelated. We suspect this low reliability is the result of limited sampling of the content domain which introduces random measurement error. Random measurement error does not lead to spuriously significant correlational results but rather attenuates them (Cohen, Cohen, West, & Aiken, 2003).

Youths also rated (1 = almost always untrue of you to 5 = almost always true of you) subscales from the Early Adolescent Temperament Questionnaire–Revised (Capaldi & Rothbart, 1992) about their irritability (11 items; e.g., “I get irritated when I have to stop doing something that I am enjoying”; M α across the six sites was .79) and sadness (9 items; e.g., “I feel depressed when unable to accomplish some task”; M α across the six sites was .78).

Depressive rumination (Children’s Response Styles Questionnaire; Abela, Brozina, & Haigh, 2002)

Youths rated (1 = almost never to 4 = almost always) 13 items assessing the extent to which they respond to sad feelings with rumination (M α across the six sites = .86).

Hostile rumination (Caprara, 1986)

Youths rated (1 = completely true for me to 6 = completely false for me) 10 items tapping the extent to which they respond to angry feelings with rumination (M α across the six sites = .78).

Self-efficacy in dealing with anger and sadness (Regulatory Emotional Self-Efficacy Scale; Caprara et al., 2008; Caprara & Gerbino, 2001)

Youths rated their ability to deal with anger and sadness across four and five items, respectively. Response options ranged from 1 = not well at all to 5 = very well. Mean reliability across the six sites was α = .75, for anger and α = .78, for sadness.

Externalizing and internalizing symptoms

Mothers completed 33 items from the externalizing scale and 31 items from the internalizing scale of the Child Behavior Checklist (Achenbach, 1991). Youths completed 30 items from the externalizing scale and 29 items from the internalizing scale of the Youth Self-Report (Achenbach, 1991). Items were rated on a 3-point scale of 0 (not true) to 2 (very true or often true). Mean reliability across the six sites was α = .87 and .84, for youth-reported internalizing and externalizing symptoms, respectively, and α = .87 and .89, for mother-reported internalizing and externalizing symptoms, respectively.

Procedure

Measures were administered in the predominant language at each site, following forward- and back-translation and meetings to resolve any item-by-item ambiguities in linguistic or semantic content (Erkut, 2010). Forward- and back-translation was used to ensure the linguistic and conceptual equivalence of measures across languages (see Maxwell, 1996). Translators were fluent in English and the target language. Translators also noted items that did not translate well, were inappropriate for participants, were culturally insensitive, or elicited multiple meanings and suggested improvements. Site coordinators and translators reviewed discrepant items and made appropriate modifications. Measures were administered in Spanish (Colombia and U.S. Hispanics), Italian (Italy), and American English (the United States).

Interviews (1.5 to 2 hours) were conducted in participants’ homes, schools, or other private locations. Procedures were approved by local institutional review boards at universities in each country. Mothers and children provided consent and assent, respectively, and were interviewed separately to ensure privacy. Mothers were given the option to have questionnaires administered orally (with rating scales provided as visual aids) or complete them on their own. All children completed the questionnaires orally, with questions read and responses recorded by trained interviewers. Families were paid a modest amount for their participation.

Analytic Approach

Multigroup confirmatory factor analyses (CFAs) were run in Mplus 3.0 (Muthén & Muthén, 1998-2007) to test our first hypothesis of measurement invariance across cultures for the monofactorial solution of each of the eight subscales in our measure. We adopted a model-fitting process based on Vandenberg and Lance (2000) as well as others (e.g., Chan, 2000). We tested three models: configural invariance (constraining the pattern of fixed and free factor loadings across all groups), metric invariance (constraining equality of all factor loadings across all groups), and scalar invariance (constraining equality of all intercepts of like items’ regressions on the latent variables across all groups). The most frequent additional tests were of partial invariance at each step, and modification indices (MIs) from each step were used to refine the structural models (Steenkamp & Baumgartner, 1998). Noninvariant items were identified by inspecting MIs on a one-by-one basis (Muthén & Muthén, 1998-2007). Each form of invariance was nested in the previous model and involved added constraints at each step that built on previous constraints. The logic was that invariance restrictions may hold for some but not all manifest measures across populations, and relaxing invariance constraints where they do not hold controls for partial measurement inequivalence (Vandenberg & Lance, 2000). We performed chi-square difference tests to compare nested models (Kline, 1998). We focused on model fit indices that are less sensitive to sample size given that obtaining a nonsignificant χ² becomes increasingly unlikely with large samples (Kline, 1998). We accepted comparative fit index (CFI) values >.90 (Kelloway, 1998; Kline, 1998), root mean square error of approximation (RMSEA) values <.07 (Browne & Cudeck, 1993), and standardized root mean square residual (SRMR) values <.08 (Kelloway, 1998). For the RMSEA, a nonsignificant p value means the hypothesized model is a good approximation of the population.

Based on a common rule of thumb for determining the minimum sample size needed per group for CFA (i.e., ratio of N to the number of variables in a model (p), N/p ≥ 10; Mundfrom, Shaw, & Ke, 2005), we could not examine measurement invariance of multifactorial models. Our smallest sample size was 76, which resulted in a value of 3.17 (i.e., 76/[6 vignettes * 4 scales]) which was substantially less than the necessary value of 10 or greater.

We used CFA to test our second hypothesis on the latent structure of multiple distinct self-regulation-related processes tapped by our measure (separately for anger and sadness) using the aggregated total sample. In the first model, attribution bias, rumination, dysregulated expression, and self-efficacy beliefs were considered separate and correlated constructs at the first level. For example, regarding anger, six hostile attribution bias questions from the six stories loaded on the first-order factor of hostile attribution bias; six hostile rumination questions from the six stories loaded on the first-order factor of hostile rumination; and so on for all hypothesized first-order anger-related factors. In this model, it was also hypothesized that these first-order factors were correlated with each other. In the second model, rumination and dysregulated expression were modeled as expressions of a second-order latent factor reflecting emotion dysregulation, whereas attribution bias and self-efficacy beliefs were modeled as expressions of two first-order factors correlated with the second-order factor (i.e., emotion dysregulation). Self-efficacy belief scores were reverse-coded (i.e., the latent factor refers to lack of self-efficacy beliefs) so that all factors in both models were positively associated. To test our two competing models, we examined model fit indices reported above as well as Akaike information criterion (AIC; a helpful index when comparing nonnested models; Tabachnick & Fidell, 2001) and Bayesian information criterion (BIC). Finally, we used correlations to test our third hypothesis regarding convergent validity.

Results

Except for Dysregulated Expression of Anger (DEA), all skewness and kurtosis values across the six stories for each subscale for each cultural group were within normal ranges (skewness < 2 and kurtosis < 7; Curran, West, & Finch, 1996). We used maximum likelihood as the estimator for all variables except DEA, for which we used robust maximum likelihood (maximum likelihood ratio).

Measurement Invariance of Monofactorial Models Across Six Cultural Groups

Independent multigroup CFAs were run to test measurement invariance across cultures for each of the subscales (i.e., as monofactorial models), separately for anger and sadness (eight models total). We discuss results separately by subscale (see Table 2, for complete details).

Table 2.

Fit Indices for Multigroup CFAs Testing Measurement Invariance of the Monofactorial Structure of Anger and Sadness Self-Regulation Subscales.

		χ²	df	p	CFI	RMSEA [CI]	Δχ^2g	Δdf	p
Hostile attribution bias	CI^a	64.25	52	.12	.96	.05 [.00, .08]
	MI^a	94.05	77	.09	.94	.04 [.00, .08]	29.80	25	.23
	SI^a	176.57	102	.00	.75	.09 [.06, .11]	82.52	25	.00
	PSI^a	120.18	98	.06	.93	.05 [.00, .07]	26.13	21	.20
Self-efficacy about anger regulation	CI^b	73.61	52	.03	.97	.06 [.02, .10]
	MI^b	94.53	77	.09	.97	.05 [.00, .08]	20.92	25	.69
	SI^b	153.02	102	.00	.92	.07 [.04, .09]	58.49	25	.00
	PSI^b	125.77	99	.04	.96	.05 [.01, .08]	31.24	22	.09
Dysregulated expression of anger	CI^c	70.71^h	53	.05	.97	.06 [.00, .09]
	MI^c	98.49ⁱ	81	.09	.98	.04 [.00, .08]	48.50	31	.02
	PMI^c	84.74^j	79	.31	.99	.03 [.00, .06]	36.39	29	.16
	PSI^c	111.01^k	102	.25	.99	.03 [.00, .06]	28.09	21	.14
Hostile rumination	CI	87.06	54	.01	.97	.08 [.05, .11]
	MI	115.61	79	.01	.97	.07 [.04, .10]	28.54	25	.28
	SI	180.95	104	.00	.93	.09 [.07, .11]	65.34	25	.00
	PSI	149.81	101	.00	.95	.07 [.05, .10]	34.21	22	.06
Depressive attribution bias	CI^d	61.14	50	.13	.98	.05 [.00, .09]
	MI^d	100.48	75	.03	.95	.06 [.02, .09]	39.34	25	.03
	PMI^d	89.30	74	.11	.97	.05 [.00, .08]	28.16	24	.25
	PSI^d	117.97	94	.05	.96	.05 [.01, .08]	28.67	20	.09
Self-efficacy about sadness regulation	CI	74.98	54	.03	.97	.07 [.02, .10]
	MI	91.221	74	.09	.97	.05 [.00, .08]	16.24	20	.70
	SI	136.29	99	.01	.94	.07 [.03, .09]	45.07	25	.01
	PSI	125.06	97	.03	.95	.06 [.02, .08]	33.84	23	.07
Dysregulated expression of sadness	CI^e	71.68	52	.04	.98	.07 [.02, .10]
	MI^e	137.27	77	.00	.93	.09 [.07, .12]	65.59	25	.00
	PMI^e	102.45	74	.02	.97	.06 [.02, .09]	30.77	22	.10
	PSI^e	121.67	96	.04	.97	.05 [.01, .08]	19.22	22	.63
Depressive rumination	CI^f	75.63	53	.02	.98	.07 [.03, .10]
	MI^f	120.14	78	.00	.96	.08 [.05, .10]	44.51	25	.00
	PMI^f	109.40	77	.01	.97	.07 [.03, .09]	33.77	24	.08
	PSI^f	138.70	100	.011	.96	.06 [.03, .09]	18.56	22	.23

Note. CFA = confirmatory factor analysis; CFI = comparative fit index; df = degrees of freedom; RMSEA = root mean square error of approximation; CI = configural invariance; MI = metric invariance; PMI = partial metric invariance; SI = scalar invariance; PSI = partial scalar invariance. Superscripts a to f indicate which covariations between errors were estimated: ^aStories 2 and 3 in Rome and Stories 1 and 3 in U.S. African Americans; ^bStories 5 and 6 in Rome and Stories 2 and 3 in Colombia; ^cStories 1 and 4 in U.S. European American and Stories 2 and 5 in U.S. African American; ^dStories 1 and 3 in Rome and U.S. Hispanic and Stories 5 and 6 in U.S. European American and U.S. African American; ^eStories 2 and 5 in U.S. European American and U.S. African American; ^fStories 1 and 3 in U.S. European American; ^gOnly for dysregulated expression of anger, when conducting χ² difference tests using MLR (maximum likelihood ratio) we adjusted the χ² using the Satorra–Bentler scaling correction; ^hScaling correction factor for MLR = 1.43; ⁱScaling correction factor for MLR = 1.65; ^jScaling correction factor for MLR = 1.65; ^kScaling correction factor for MLR = 1.52.

Hostile Attribution Bias

We found full metric invariance across the six groups. We also found partial scalar invariance: Intercepts for Story 2 in Rome and U.S. African Americans and Stories 3 and 5 in U.S. European Americans were relaxed to differ from the other groups.

Self-Efficacy Beliefs About Anger Regulation

We found full metric invariance across the six groups. We also found partial scalar invariance: Intercepts for Stories 3 and 4 in U.S. European Americans and Story 6 in Colombia were relaxed to differ from the other groups.

Dysregulated Expression of Anger

We found partial metric invariance across the six groups: Factor loadings for Story 4 in Rome and Story 6 in U.S. Hispanic Americans were relaxed to differ from the other groups. We also found partial scalar invariance: Intercepts were relaxed for stories for which factor loadings were already relaxed.

Hostile Rumination

We found full metric invariance across the six groups. We also found partial scalar invariance: Intercepts for Story 4 in Rome, Story 1 in U.S. European Americans, and Story 6 in Colombia were relaxed to differ from the other groups.

Depressive Attribution Bias

We found partial metric invariance across the six groups: The factor loading for Story 5 in U.S. European Americans was relaxed to differ from other groups. We also found partial scalar invariance: Intercepts for Story 5 in U.S. European Americans; Story 3 in Naples, U.S. European Americans, and Colombia; and Story 2 in U.S. African Americans were relaxed to differ from other groups.

Self-Efficacy Beliefs About Sadness Regulation

We found full metric invariance across the six groups. We also found partial scalar invariance: Intercepts for Story 2 in Rome and Naples were relaxed to differ from other groups.

Dysregulated Expression of Sadness

We found partial metric invariance across the six groups: Factor loadings for Story 4 in U.S. European Americans, Story 6 in U.S. African Americans, and Story 2 in Colombia were relaxed to differ from other groups. We also found partial scalar invariance: Intercepts were relaxed for stories for which factor loadings were already relaxed.

Depressive Rumination

We found partial metric invariance across the six groups: The factor loading for Story 4 in Rome was relaxed to differ from other groups. We also found partial scalar invariance: Intercepts for Story 4 in Rome and Story 6 in U.S. European Americans were relaxed to differ from other groups.

Independent multigroup CFAs indicated that the monofactorial structure of our anger and sadness self-regulation subscales generally replicated across the six cultural groups. It is important to note that for all examined models MIs suggested estimation of covariations between story errors for some of the groups. We estimated five covariations between errors when we had substantive reasons in support of such covariation based on similarities in vignette content.³

Latent Structure of Multiple Distinct Self-Regulation-Related Processes (Total Sample)

We tested two alternative models using CFA to examine the latent structure of multiple distinct self-regulation-related processes (i.e., subscales) separately for anger and sadness (four models total). For these analyses, we used the aggregate sample (N = 541). In Model 1, the four subscales were considered as separate and correlated constructs at the first level. Results for anger self-regulation were: χ²(226; N = 541) = 405.29; p < .001; CFI = .95; RMSEA = .04 [.03, .04]; AIC = 36329.94; BIC = 36751.05, and for sadness self-regulation: χ²(226; N = 541) = 406.68; p < .001; CFI = .95; RMSEA = .04 [.03, .04]; AIC = 35469.13; BIC = 35890.25. In Model 2, rumination and dysregulated expression were modeled as expressions of a second-order latent factor reflecting emotion dysregulation, whereas attribution bias and self-efficacy beliefs were modeled as expressions of two first-order factors correlated with the second-order factor.⁴ Results for anger self-regulation were: χ²(228; N = 541) = 414.18; p < .001; CFI = .94; RMSEA = .04 [.03, .04]; AIC = 36336.77; BIC = 36749.30, and for sadness self-regulation: χ²(228, N = 541) = 415.82; p < .001; CFI = .95; RMSEA = .04 [.03, .04]; AIC = 35474.27; BIC = 35886.79. For both anger and sadness, both models provided good fit to the data. Given these findings, rumination and dysregulated expression of both anger and sadness may be viewed as separate and correlated factors (i.e., Model 1) as well as expressions of a second-order factor reflecting emotion dysregulation (i.e., Model 2).

Given the small differences between model fit indices, we provide some guidance regarding when one model might be preferred over the other based on model selection goals associated with AIC and BIC. If our scale is used to predict mental and behavioral health outcomes, the model selection goals associated with AIC (minimizing Type II errors; Dziak, Coffman, Lanza, & Li, 2012) would support Model 1 as representative of the structure of our measure (Figures 1 and 2). If the focus of model selection is on choosing the most parsimonious model and both Type I and Type II errors are considered equally adverse, the model selection goals associated with BIC (Dziak et al., 2012) would support Model 2 as representative of the structure of our measure (Figures 3 and 4).

Convergent Validity of Anger and Sadness Self-Regulation Subscales

Using a Bonferroni adjustment to control for Type I errors, we consider only correlations significant at the .001 level (bolded values in Table 3). Regarding anger, significant correlations held across the six groups for self-efficacy beliefs about anger regulation (SEAR) and dysregulated expression of anger (DEA) and across four groups for hostile rumination (HR). Among the entire sample, hostile attribution bias (HAB) was significantly related to mother-reported externalizing symptoms; HR and SEAR were significantly related to child-reported externalizing symptoms; and DEA was significantly related to both. Regarding sadness, significant correlations held across five groups for depressive rumination (DR), across four groups for self-efficacy beliefs about sadness regulation (SESR), and across three groups for dysregulated expression of sadness (DES). Among the entire sample, DAB, DR, and SESR were significantly related to only child-reported internalizing symptoms, whereas DES was significantly related to both mother- and child-reported internalizing symptoms.

Table 3.

Convergent Validity: Correlations Between Anger and Sadness Self-Regulation Subscales and Related Constructs.

Correlated variables	Italy—Naples (n = 90)	Italy—Rome (n = 100)	USA—European American (n = 98)	USA—African American (n = 90)	USA—Hispanic American (n = 76)	Colombia (n = 88)	Externalizing problems (n = 541), child report (mother report)
HAB with HAB 1 year prior	.12	.14	.31**	.20	.38**	−.04	.11 (.18)
SEAR with anger on RESE	.58**	.69**	.66**	.37**	.52**	.54**	−.38 (−.12)
DEA with CAMS	.54**	.44**	.52**	.51**	.62**	.47**	.60 (.27)
DEA with irritability on EATQ-R	.38**	.34**	.44**	.40**	.47**	.21*	.48 (.23)
HR with HRS	.41**	.19*	.51**	.37**	.38**	.08	.15** (.08)
							Internalizing problems (n = 541), child report (mother report)
DAB with internalizing on YSR and CBCL	.30** (.05)	.26** (.17)	.19 (−.01)	.14 (−.03)	.10 (.01)	.09 (−.01)	.17** (.03)
SESR with sadness on RESE	.34**	.43**	.60**	.29**	.40**	.27**	−.21** (−.08)
DES with CSMS	.38**	.39**	.43**	.27**	.30**	.20	.39 (.22)
DES with sadness on EATQ-R	.37**	.28**	.38**	.23*	.33**	.27**	.64 (.25)
DR with DRS	.38**	.32**	.41**	.35**	.44**	.33**	.31** (.11*)

Note. HAB = Hostile Attribution Bias; DEA = Dysregulated Expression of Anger; SEAR = Self-Efficacy beliefs about Anger Regulation; RESE = Regulatory Emotional Self-Efficacy Scale; CAMS = Children’s Anger Management Scale; EATQ-R = Early Adolescent Temperament Questionnaire–Revised; HR = Hostile Rumination; HRS = Hostile Rumination Scale; DAB = Depressive Attribution Bias; YSR = Youth Self-Report; CBCL = Child Behavior Checklist; SESR = Self-Efficacy beliefs about Sadness Regulation; DES = Dysregulated Expression of Sadness; CSMS = Children’s Sadness Management Scale; DR = Depressive Rumination; DRS = Depressive Rumination Scale. The first variable in the correlated variable column is from the Anger and Sadness Self-Regulation measure being validated in the current study. The second variable in the correlated variable column is from convergent measures. Bolded values indicate significant correlations after Bonferroni correction.

p ≤ .05. **p ≤ .01.

Discussion

The current study evaluated the internal and convergent validity of a vignette-based scale measuring four self-regulation-related processes: attribution biases, dysregulated expression of emotions, rumination, and self-efficacy beliefs about emotion regulation. We examined these processes separately for anger and sadness among preadolescents from three countries and six cultural groups. Participants came from cultural groups and countries that varied on sociodemographic indicators, which provided a good initial test of the external validity of the scale’s psychometric properties.

Overall, results were generally consistent with our first hypothesis that the internal structure of our measure would be similar across cultures. Full metric invariance held for all but one of our anger self-regulation subscales—dysregulated expression. Dysregulated expression of anger demonstrated partial metric invariance with half of the factor loadings invariant across the six cultural groups. We found partial scalar invariance for all anger self-regulation scales; however, the majority of story intercepts were invariant across the six groups. Regarding our sadness self-regulation subscales, we found partial support for measurement invariance. Full metric invariance held for one of the four subscales (i.e., self-efficacy beliefs about sadness regulation). Partial metric invariance held for the remaining three subscales although factor loadings were invariant for the majority of the six stories. Partial scalar invariance was evident for all sadness self-regulation subscales; however, the majority of story intercepts were invariant across the six groups.

Overall, story factor loadings and intercepts that needed to be freely estimated across groups were not consistent. For both anger and sadness self-regulation subscales, factor mean comparisons across the six groups may still be regarded as meaningful given that at least one story intercept, in addition to anchor one, was invariant (Steenkamp & Baumgartner, 1998). Nonetheless, two caveats are warranted. There is no consensus on a minimum level of partial invariance (i.e., the amount of freely estimated parameters) that is required for meaningful factor mean comparisons (Millsap & Kwok, 2004; Widaman & Reise, 1997). Furthermore, measurement invariance results may change depending on which item (or story in our case) is chosen as the anchor (Vandenberg, 2002). The data on scalar invariance suggest that the constructs related to anger and sadness self-regulation do not have exactly the same meaning across our six cultural groups. This is not entirely surprising given that the cultures examined in this study likely differ on important characteristics, such as social norms, that influence the way one perceives, interprets, manages, and reacts to social exchanges. Caution must, therefore, be exercised when comparing mean values of the subscales (or stories) across groups.

Prior research on hostile attribution bias (Dodge et al., 2015) indicates that mean levels vary across cultures. Our study adds to this growing literature base by providing some evidence of measurement invariance for hostile and depressive attribution biases across cultures, with stronger support for hostile attribution biases. To our knowledge, this is the first study to include evidence on the metric invariance of attribution biases. Moreover, our results offer preliminary support for making mean comparisons of both hostile and depressive attribution biases in preadolescence across multiple culturally different contexts. Similar to the attribution biases literature, evidence on the measurement invariance of emotion regulation is lacking. Prior work on effortful control (which is highly related to emotion regulation) indicates that its internal structure is consistent across three ethnic groups of low-income preschoolers in the United States (Sulik et al., 2010). Our findings, which spanned more diverse and older samples of youth, were generally consistent with this prior work. This study offers preliminary support for making mean comparisons of rumination and dysregulated expression of both anger and sadness in preadolescence across multiple culturally diverse contexts. Self-efficacy beliefs about emotion regulation have a much larger evidence base in support of measurement invariance (Caprara et al., 2008; Caprara et al., 2012), and our results are consistent with this prior work. Our findings suggest that measurement invariance of self-efficacy beliefs about emotion regulation holds across diverse ages and cultures.

Using the total sample, we found support for the latent structure of multiple distinct self-regulation-related processes tapped by our self-regulation subscales. Both competing models fit the data well. Model choice will therefore depend on model selection goals and how the measure will be used. A four-factor oblique model, where subscales were represented on separate factors, appears appropriate (based on AIC) when the measure will be used to predict mental health and behavioral outcomes and goals are to minimize Type II errors (Dziak et al., 2012). A three-factor oblique model, where dysregulated expression and rumination loaded on a second-order factor (i.e., emotion dysregulation) and attribution bias and self-efficacy beliefs were modeled as expressions of two first-order factors, appears appropriate (based on BIC) when the goal is to select the most parsimonious model and minimize both Type I and Type II errors (Dziak et al., 2012).

Factors within both the four- and three-factor oblique models were positively and moderately associated with each other, except for attribution bias and lack of self-efficacy beliefs about emotion regulation. Correlational patterns between factors were generally similar for anger and sadness regulation-related constructs. One notable difference occurred in the four-factor model: Whereas the correlation between latent factors for hostile rumination and dysregulated expression of anger was weak (r = .19), the correlation between latent factors for depressive rumination and dysregulated expression of sadness was strong (r = .52). This suggests that when youth are in a peer context that makes them feel angry, if they repetitively think about what went wrong and how angry they feel, they do not necessarily act on those feelings (e.g., yell or slam something). However, when youth are in a peer context that makes them feel sad, if they repetitively think about what went wrong and how sad they feel, they are much more likely to act on those feelings (e.g., make a sad face or cry). It is important to note that we cannot rule out the possibility that such correlations may exist in specific cultural groups but not in others. More studies are needed to determine whether these models replicate and to fully interpret correlations between these constructs. Studies using samples from more cultural groups, samples that are more demographically diverse (e.g., socioeconomic status), and clinical samples would improve our understanding of the psychometric properties and appropriate applications of this measure.

Our vignette-based measure demonstrated good preliminary evidence of convergent validity, generally correlating with correspondent measures assessed via questionnaire. Dysregulated expression of anger and self-efficacy beliefs about anger regulation were strongly and positively associated with correspondent measures in all six cultural groups, whereas hostile rumination was moderately and positively associated with correspondent measures in four of the six cultural groups. We found strongest evidence of validity for dysregulated expression of anger, which was strongly related to both mother- and child-reported externalizing behaviors. Hostile attribution bias was strongly related only to mother-reported externalizing behaviors. Hostile rumination and self-efficacy beliefs about anger regulation were strongly related to only child-reported externalizing behaviors. This pattern of findings may have emerged because both children and parents are equally accurate reporters of overt failures in anger self-regulation (e.g., dysregulated expression), whereas children are more accurate reporters of internalized self-regulation processes (e.g., rumination; e.g., De Los Reyes & Kazdin, 2005).

Depressive rumination was most consistently correlated with theoretically related questionnaires across groups. Dysregulated expression of sadness and self-efficacy beliefs about sadness regulation were significantly correlated with theoretically related questionnaires across at least half of the groups. Some of the findings for dysregulated expression of sadness may have been attenuated due to low reliability. We did not find evidence of construct validity for depressive attribution bias for any of the six groups. Construct validity for our sadness self-regulation subscales was strongest among participants of European descent (i.e., both Italian samples and U.S. European Americans). In the entire sample, all four sadness self-regulation subscales were strongly related to child- but not mother-reported internalizing symptoms. The lack of association with mother-reported internalizing problems was not surprising considering that previous studies suggest that youth are more accurate reporters of internalizing problems (De Los Reyes & Kazdin, 2005).

Hostile and depressive attribution biases were the only subscales with very few significant correlations with convergent measures across the six groups. The lack of significant correlations is likely due to limitations in our convergent measures; for hostile attribution bias, the convergent measure was assessed 1 year prior and for depressive attribution bias, the measure covered a broader range of characteristics beyond attribution biases. Nonetheless, among the entire sample, attribution biases were significantly related to concurrent levels of externalizing and internalizing symptoms, offering general evidence of their construct validity.

Limitations and Future Directions

The relatively small sample sizes within cultural groups are an important limitation to our findings. Moreover, our findings are based on youth from only three countries. Empirical inferences and conclusions about the cross-cultural validation of subscales within our sadness and anger self-regulation measures may vary when larger and more diverse cross-cultural samples are used. Further research is needed to collect data from larger and more culturally diverse samples to more completely establish the validity of our measure.

Latent structural models were tested within our total sample that aggregated across cultural groups because sample sizes within each group were too small relative to the number of parameters estimated. We found partial support for measurement invariance (i.e., equal structure, partially equal loadings, partially equal intercepts, but unequal error variances and covariances) of each factor related to each emotion across the six cultural groups. That may lead one to expect that the hypothesized latent structural models might be invariant within each cultural group. However, we cannot infer from measurement invariance tests whether correlations between multiple factors are invariant across groups. Future studies with larger sample sizes within cultural groups could examine the measurement invariance of our multifactorial models separately by group status.

Conclusions

This study provides crucial psychometric evidence to support cross-cultural examinations of self-regulation mechanisms associated with discrete negative emotions. We examined multiple mechanisms reflecting independent and related aspects of anger and sadness regulation among participants from a variety of cultural contexts. Overall, our findings provide preliminary support for partial measurement invariance and construct validity of a novel tool measuring anger and sadness self-regulation mechanisms in preadolescence in six diverse cultural contexts. This assessment instrument is a tool that can be used in future studies examining the complex relations among emotion-specific self-regulation-related processes and psychological adjustment among youth in cross-cultural contexts.

Footnotes

Acknowledgements

We thank the families who participated in this research and the many research assistants who helped gather data. We thank Craig Enders for his assistance with the interpretation of results for this article.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by the Jacobs Foundation, the Josiah Charles Trent Memorial Foundation, the Transdisciplinary Prevention Research Center at Duke University, the Eunice Kennedy Shriver National Institute of Child Health and Human Development [grant RO1-HD054805], and the Fogarty International Center [grant RO3-TW008141].

Notes

References

Abela

J. Z.

Brozina

Haigh

E. P.

(2002). An examination of the response styles theory of depression in third- and seventh-grade children: A short-term longitudinal study. Journal of Abnormal Child Psychology, 30, 515-527. doi:10.1023/A:1019873015594

Achenbach

T. M.

(1991). Integrative guide for the 1991 CBCL/4-18, YSR, and TRF profiles. Burlington: University of Vermont.

Ayduk

Mendoza-Denton

Mischel

Downey

Peake

P. K.

Rodriguez

(2000). Regulating the interpersonal self: Strategic self-regulation for coping with rejection sensitivity. Journal of Personality and Social Psychology, 79, 776-792. doi:10.1037//0022-3514.79.5.776

Bandura

(2006). Guide to the construction of self-efficacy scales. In Pajares

Urdan

(Eds.), Self-efficacy beliefs of adolescents (pp. 1-43). Greenwich, CT: Information Age.

Bandura

Caprara

G. V.

Barbaranelli

Gerbino

Pastorelli

(2003). Role of affective self-regulatory efficacy on diverse spheres of psychosocial functioning. Child Development, 74, 769-782. doi:10.1111/1467-8624.00567

Bombi

A. S.

Pastorelli

Bacchini

Di Giunta

Miranda

M. C.

Zelli

(2011). Attributions and attitudes of mothers and fathers in Italy. Parenting: Science and Practice, 11, 129-141. doi:10.1080/15295192.2011.585557

Browne

M. W.

Cudeck

(1993). Alternative ways of assessing model fit. In Bollen

K. A.

Long

S. J.

(Eds.), Testing structural equation models (pp. 136-162). Newbury Park, CA: Sage.

Bushman

B. J.

(2002). Does venting anger feed or extinguish the flame? Catharsis, rumination, distraction, anger and aggressive responding. Personality and Social Psychology Bulletin, 28, 724-731. doi:10.1177/0146167202289002

Capaldi

D. M.

Rothbart

M. K.

(1992). Development and validation of an early adolescent temperament measure. Journal of Early Adolescence, 12, 153-173. doi:10.1177/0272431692012002002

10.

Caprara

G. V.

(1986). Indicators of aggression: The Dissipation-Rumination scale. Personality and Individual Differences, 7, 763-769. doi:10.1016/0191-8869(86)90074-7

11.

Caprara

G. V.

Di Giunta

Eisenberg

Gerbino

Pastorelli

Tramontano

(2008). Assessing regulatory emotional self-efficacy. Psychological Assessment, 20, 227-237. doi:10.1037/1040-3590.20.3.227

12.

Caprara

G. V.

Di Giunta

Pastorelli

Eisenberg

(2012). Mastery of negative affect: A hierarchical model of emotional self-efficacy beliefs. Psychological Assessment, 25, 105-116. doi:10.1037/a0029136

13.

Caprara

G. V.

Gerbino

(2001). Affective perceived self-efficacy: The capacity to regulate negative affect and to express positive affect. In Caprara

G. V.

(Ed.), Self-efficacy assessment (pp. 35-50). Trento, Italy: Edizioni Erickson.

14.

Caprara

G. V.

Luengo Kanacri

B. P.

Gerbino

Zuffianò

Alessandri

Vecchio

G. M.

. . . Bridglall

(2014). Positive effects of promoting prosocial behavior in early adolescence evidence from a school-based intervention. International Journal of Behavioral Development, 38, 386-396. doi:10.1177/0165025414531464.

15.

Caprara

G. V.

Paciello

Gerbino

Cugini

(2007). Individual differences conducive to aggression and violence: Trajectories and correlates of irritability and hostile rumination through adolescence. Aggressive Behavior, 33, 1-16. doi:10.1002/ab.20192

16.

Chan

(2000). Detection of differential item functioning on the Kirton Adaption-Innovation Inventory using multiple-group mean and covariance structure analyses. Multivariate Behavioral Research, 35, 169-199. doi:10.1207/S15327906MBR3502_2

17.

Chaplin

T. M.

Cole

P. M.

Zahn-Waxler

(2005). Parental socialization of emotion expression: Gender differences and relations to child adjustment. Emotion, 5, 80-88. doi:10.1037/1528-3542.5.1.80

18.

Cohen

West

S. G.

Aiken

L. S.

(2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Mahwah, NJ: Erlbaum.

19.

Cicchetti

Toth

S. L.

(1998). The development of depression in children and adolescents. American Psychologist, 53, 221-241.

20.

Cole

P. M.

Michel

M. K.

O’Donnell

T. L.

(1994). The development of emotion regulation and dysregulation: A clinical perspective. Monographs of the Society for Research in Child Development, 59(2-3), 73-100.

21.

Conduct Problems Prevention Research Group. (1992). A developmental and clinical model for the prevention of conduct disorders: The FAST Track Program. Development and Psychopathology, 4, 509-527. doi:10.1017/S0954579400004855

22.

Crick

N. R.

Dodge

K. A.

(1994). A review and reformulation of social information-processing mechanisms in children’s social adjustment. Psychological Bulletin, 115, 74-101. doi:10.1037/0033-2909.115.1.74

23.

Crick

N. R.

Dodge

K. A.

(1996). Social information-processing mechanisms on reactive and proactive aggression. Personality and Social Psychology Bulletin, 67, 993-1002. doi:10.1111/j.1467-8624.1996.tb01778.x

24.

Curran

P. J.

West

S. G.

Finch

J. F.

(1996). The robustness of test statistics to non-normality and specification error in confirmatory factor analysis. Psychological Methods, 1, 16-29. doi:10.1037/1082-989X.1.1.16

25.

De Los Reyes

Kazdin

A. E.

(2005). Informant discrepancies in the assessment of childhood psychopathology: A critical review, theoretical framework, and recommendations for further study. Psychological Bulletin, 131, 483-509. doi:10.1037/0033-2909.131.4.483

26.

Dodge

K. A.

(2006). Translational science in action: Hostile attribution style and the development of aggressive behavior problems. Development and Psychopathology, 18, 791-814. doi:10.1017/S0954579406060391

27.

Dodge

K. A.

Malone

P. S.

Lansford

J. E.

Sorbring

Skinner

A. T.

Tapanya

. . . Pastorelli

(2015). Hostile attributional bias and aggressive behavior in global context. Proceedings of the National Academy of Sciences, 112, 9310-9315. doi:10.1073/pnas.1418572112

28.

Dodge

K. A.

Pettit

G. S.

(2003). A biopsychosocial model of the development of chronic conduct problems in adolescence. Developmental Psychology, 39(2), 349-371. doi:10.1037/0012-1649.39.2.349

29.

Dodge

K. A.

Somberg

D. R.

(1987). Hostile attributional bias among aggressive boys are exacerbated under conditions of threat to the self. Child Development, 58, 213-224.

30.

Dziak

J. J.

Coffman

D. L.

Lanza

S. T.

(2012). Sensitivity and specificity of information criteria (Tech rep no 12-119). University Park: The Pennsylvania State University.

31.

Eisenberg

Spinrad

T. L.

(2004). Emotion-related regulation: Sharpening the definition. Child Development, 75, 334-339.

32.

Eisenberg

Spinrad

T. L.

Eggum

N. D.

(2010). Emotion-related self-regulation and its relation to children’s maladjustment. Annual Review of Clinical Psychology, 6, 495-525. doi:10.1146/annurev.clinpsy.121208.131208

33.

Eisenberg

Valiente

Spinrad

T. L.

Cumberland

Liew

Reiser

. . . Losoya

S. H.

(2009). Longitudinal relations of children’s effortful control, impulsivity, and negative emotionality to their externalizing, internalizing, and co-occurring behavior problems. Developmental Psychology, 45, 988-1008. doi:10.1037/a0016213

34.

Erkut

(2010). Developing multiple language versions of instruments for intercultural research. Child Development Perspectives, 4, 19-24. doi:10.1111/j.1750-8606.2009.00111.x

35.

Feng

Keenan

Hipwell

A. E.

Henneberger

A. K.

Rischall

M. S.

Butch

. . . Babinski

D. E.

(2009). Longitudinal associations between emotion regulation and depression in preadolescent girls: Moderation by the caregiving environment. Developmental Psychology, 45, 798-808. doi:10.1037/a0014617

36.

Folk

J. B.

Zeman

J. L.

Poon

J. A.

Dallaire

D. H.

(2014). A longitudinal examination of emotion regulation: Pathways to anxiety and depressive symptoms in urban minority youth. Child and Adolescent Mental Health, 19, 243-250. doi:10.1111/camh.12058

37.

Gould

(1996). Using vignettes to collect data for nursing research studies: How valid are the findings? Journal of Clinical Nursing, 5, 207-212. doi:10.1111/j.1365-2702.1996.tb00253.x

38.

Gratz

K. L.

Roemer

(2004). Multidimensional assessment of emotion regulation and dysregulation: Development, factor structure, and initial validation of the Difficulties in Emotion Regulation Scale. Journal of Psychopathology and Behavioral Assessment, 36, 41-54. doi: 10.1023/B:JOBA.0000007455.08539.94

39.

Gunzenhauser

Heikamp

Gerbino

Alessandri

von Suchodoletz

Di Giunta

. . . Trommsdorff

(2013). Self-efficacy in regulating positive and negative emotions: A validation study in Germany. European Journal of Psychological Assessment, 29, 197-204. doi:10.1027/1015-5759/a000151

40.

Halberstadt

A. G.

Denham

S. A.

Dunsmore

J. C.

(2001). Affective social competence. Social Development, 10, 79-119. doi:10.1111/1467-9507.00150

41.

Hofer

Eisenberg

Reiser

(2010). The role of socialization, effortful control, and ego resiliency in French adolescents’ social functioning. Journal of Research on Adolescence, 20, 555-582. doi:10.1111/j.1532-7795.2010.00650.x

42.

Karoly

(1993). Mechanisms of self-regulation: A systems view. Annual Review of Psychology, 44, 23-52. doi:0066-4308/93/0201-0023

43.

Kelloway

E. K.

(1998). Using LISREL for structural equation modeling: A researcher’s guide. Thousand Oaks, CA: Sage.

44.

Kline

R. B.

(1998). Principles and practices of structural equation modeling. New York, NY: Guilford.

45.

Lansford

J. E.

Sharma

Malone

P. S.

Woodlief

Dodge

K. A.

Oburu

. . . Di Giunta

(2014). Corporal punishment, maternal warmth, and child adjustment: A longitudinal study in eight countries. Journal of Clinical Child and Adolescent Psychology, 43(4), 670-685. doi:10.1080/15374416.2014.893518.

46.

Luebbe

A. M.

Bell

D. J.

Allwood

M. A.

Swenson

L. P.

Early

M. C.

(2010). Social information processing in children: Specific relations to anxiety, depression, and affect. Journal of Clinical Child & Adolescent Psychology, 39, 386-399. doi:10.1080/15374411003691685

47.

Lynn

(2009). In Italy, north-south differences in IQ predict differences in income, education, infant mortality, stature, and literacy. Intelligence, 38, 93-100.

48.

Maxwell

(1996). Translation and cultural adaptation of the survey instruments. In Martin

M. O.

Kelly

D. L.

(Eds.), Third International Mathematics and Science Study (TIMSS) technical report, Volume I: Design and development. Chestnut Hill, MA: Boston College. Retrieved from http://timss.bc.edu/timss1995i/TIMSSPDF/TRall.pdf

49.

Millsap

R. E.

Kwok

O. M.

(2004). Evaluating the impact of partial factorial invariance on selection in two populations. Psychological Methods, 9, 93-115. doi:10.1037/1082-989X.9.1.93

50.

Möller

Krahé

(2009). Exposure to violent video games and aggression in German adolescents: A longitudinal analysis. Aggressive Behavior, 35, 75-89. doi:10.1002/ab.20290

51.

Morf

C. C.

Mischel

(2002). Introduction to special issue: Self-concept, self-regulation, and psychological vulnerability. Self and Identity, 1, 103-104.

52.

Morris

A. S.

Silk

J. S.

Morris

M. D.

Steinberg

Aucoin

K. J.

Keyes

A. W.

(2011). The influence of mother–child emotion regulation strategies on children’s expression of anger and sadness. Developmental Psychology, 47(1), 213-225. doi:10.1037/a0021021

53.

Mundfrom

D. J.

Shaw

D. G.

T. L.

(2005). Minimum sample size recommendations for conducting factor analyses. International Journal of Testing, 5, 159-168. doi:10.1207/s15327574ijt0502_4

54.

Muthén

L. K.

Muthén

B. O.

(1998-2007). Mplus user’s guide. Los Angeles, CA: Author.

55.

Nelson

D. A.

Coyne

S. M.

(2009). Children’s intent attributions and feelings of distress: Associations with maternal and paternal parenting practices. Journal of Abnormal Child Psychology, 37, 223-237. doi:10.1007/s10802-008-9271-3

56.

Nolen-Hoeksema

(1991). Response to depression and their effects on the duration of depressive episodes. Journal of Abnormal Psychology, 100, 569-582.

57.

Nolen-Hoeksema

Wisco

Lyubomirsky

(2008). Rethinking rumination. Perspectives on Psychological Science, 3, 400-424. doi:10.1111/j.1745-6924.2008.00088.x

58.

Oldehinkel

A. J.

Hartman

C. A.

Ferdinand

R. F.

Verhulst

F. C.

Ormel

(2007). Effortful control as modifier of the association between negative emotionality and adolescents’ mental health problems. Development and Psychopathology, 19, 523-539. doi:10.1017/S0954579407070253

59.

Orobio

Castro

Veerman

J. W.

Koops

Bosch

J. D.

Monshouwer

H. J.

(2002). Hostile attribution of intent and aggressive behavior: A meta-analysis. Child Development, 73, 916-934. doi:10.1111/1467-8624.00447

60.

Prinstein

M. J.

Cheah

C. L.

Guyer

A. E.

(2005). Peer victimization, cue interpretation, and internalizing symptoms: Preliminary concurrent and longitudinal findings for children and adolescents. Journal of Clinical Child & Adolescent Psychology, 34, 11-24. doi:10.1207/s15374424jccp3401_2

61.

Reijntjes

Thomaes

Kamphuis

J. H.

Bushman

B. J.

de Castro

Telch

M. J.

(2011). Explaining the paradoxical rejection-aggression link: The mediating effects of hostile intent attributions, anger, and decreases in state self-esteem on peer rejection-induced aggression in youth. Personality and Social Psychology Bulletin, 37, 955-963. doi:10.1177/0146167211410247

62.

Rivers

S. E.

Brackett

M. A.

Katulak

N. A.

Salovey

(2007). Regulating anger and sadness: An exploration of discrete emotions in emotion regulation. Journal of Happiness Studies, 8, 393-427. doi:10.1007/s10902-006-9017-2

63.

Rutter

(2011). Child psychiatric diagnosis and classification: Concepts, findings, challenges and potential. Journal of Child Psychology and Psychiatry, 52, 647-660. doi:10.1111/j.1469-7610.2011.02367.x

64.

Rydell

Berlin

Bohlin

(2003). Emotionality, emotion regulation, and adaptation among 5- to 8-year-old children. Emotion, 3, 30-47. doi:10.1037/1528-3542.3.1.30

65.

Silk

J. S.

Shaw

D. S.

Forbes

E. E.

Lane

T. L.

Kovacs

(2006). Maternal depression and child internalizing: The moderating role of child emotion regulation. Journal of Clinical Child & Adolescent Psychology, 35, 116-126. doi:10.1207/s15374424jccp3501_10

66.

Silk

J. S.

Steinberg

Morris

(2003). Adolescents’ emotion regulation in daily life: Links to depressive symptoms and problem behavior. Child Development, 74, 1869-1880. doi:10.1046/j.1467-8624.2003.00643

67.

Steenkamp

J. E. M.

Baumgartner

(1998). Assessing measurement invariance in cross-national consumer research. Journal of Consumer Research, 25, 78-90. doi:10.1086/209528

68.

Sulik

M. J.

Huerta

Zerr

Eisenberg

Spinrad

T. L.

Valiente

. . . Taylor

(2010). The factor structure of effortful control and measurement invariance across ethnicity and sex in a high-risk sample. Journal of Psychopathology and Behavioral Assessment, 32, 8-22. doi:10.1007/s10862-009-9164-y

69.

Tabachnick

B. C.

Fidell

L. S.

(2001). Using multivariate statistics (4h ed.). Needham Heights, MA: Allyn & Bacon.

70.

United Nations Development Programme. (2012). Human Development Reports. Retrieved from http://hdr.undp.org/en/content/human-development-index-hdi

71.

Vandenberg

R. J.

(2002). Toward a further understanding of and improvement in measurement invariance methods and procedures. Organizational Research Methods, 5, 139-158. doi:10.1177/1094428102005002001

72.

Vandenberg

R. J.

Lance

C. E.

(2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4-69. doi:10.1177/109442810031002

73.

Waters

S. F.

Thompson

R. A.

(2014). Children’s perceptions of the effectiveness of strategies for regulating anger and sadness. International Journal of Behavioral Development, 38, 174-181. doi:10.1177/0165025413515410

74.

Widaman

K. F.

Reise

S. P.

(1997). Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. In Bryant

K. J.

Windle

West

S. G.

(Eds.), The science of prevention: Methodological advances from alcohol and substance abuse research (pp. 281-324). Washington, DC: American Psychological Association.

75.

Yap

M. H.

Allen

N. B.

Sheeber

(2007). Using an emotion regulation framework to understand the role of temperament and family processes in risk for adolescent depressive disorders. Clinical Child and Family Psychology Review, 10, 180-196. doi:10.1007/s10567-006-0014-0

76.

Yasui

Dishion

T. J.

(2007). The ethnic context of child and adolescent problem behavior: Implications for child and family interventions. Clinical Child and Family Psychology Review, 10, 137-179. doi:10.1007/s10567-007-0021-9

77.

Zeman

Shipman

Suveg

(2002). Anger and sadness regulation: Predictions to internalizing and externalizing symptoms in children. Journal of Clinical Child & Adolescent Psychology, 31, 393-398. doi:10.1207/153744202760082658

78.

Zimmermann

Iwanski

(2014). Emotion regulation from early adolescence to emerging adulthood and middle adulthood: Age differences, gender differences, and emotion-specific developmental variations. International Journal of Behavioral Development, 38(2), 182-194. doi:10.1177/0165025413515405