Psychometric Evaluation of the Chinese Version of the Emotion Regulation Questionnaire in Taiwanese College Students

Abstract

Emotion regulation has been considered a vital mechanism for psychological and physical health. The Emotion Regulation Questionnaire (ERQ) was originally developed by Gross and John to measure the emotion regulation strategies of cognitive reappraisal and expressive suppression. Given the practical importance of functional emotion regulation for psychological assessment in the Taiwanese cultural context, this study examined the psychometric properties of the Chinese version of the ERQ in a sample of 909 Taiwanese college students. Results showed that the independent two-factor measurement model was an adequate representation of the structure of the Chinese-ERQ. Reliability and criterion validity of the Chinese-ERQ were also demonstrated. The scalar invariant model across genders was supported and male students reported using suppression to a greater degree than female students. Overall, the Chinese-ERQ is a valuable tool for assessing emotion regulation strategies in research and practice in clinical psychology. Practical implications and future research directions are discussed.

Keywords

emotion regulation reappraisal suppression validation measurement invariance

Emotion regulation has long been considered a vital mechanism for psychological and physical health. Research on emotion regulation has been connected to psychological well-being, social relations, and affective functioning (see, e.g., Balzarotti, John, & Gross, 2010; Gross & John, 2003; Gullone & Taffe, 2011). Because emotions unfold over the course of a regulatory process by which individuals modulate or control their positive and negative emotions in various ways, emotion regulation strategies can be differentiated as antecedent-focused or response-focused (Gross & John, 2003). Two of the most well-researched and operationalized strategies that people use consciously or unconsciously in everyday life are cognitive reappraisal (antecedent-focused) and expressive suppression (response-focused).

Cognitive reappraisal is defined as a cognitive change that takes place prior to the emotional response being fully activated, and efficiently reshapes the trajectory of emotional development over time. Cognitive reappraisal could occur if, for example, during a new product launch, a presenter fails to clearly introduce the newly developed features. She or he might take this failure as a personal growth opportunity, rather than as a personal liability. Generally speaking, habitual reappraisers can down-regulate the experiential and behavioral components of negative emotions, and thereby modify the impact of negative emotion response tendencies. On the other hand, expressive suppression is the inhibition of emotion-expressive behavior when an emotion is under way and the emotional response is already being elicited. For example, if during a job interview, a candidate keeps a neutral facial expression to avoid appearing anxious, even though she or he is experiencing excessive hand sweating, expressive suppression is occurring. That is, habitual suppressors exert behavioral control over negative emotions, as well as positive emotions, which create the feeling of inner conflict, leading to conscious efforts to manage the emotions.

Over the past few decades, Taiwanese college students have been reported with problems of emotion regulation or management mainly because they experience more frequent and intense emotions than younger or older individuals in their social interdependence, romantic relationships, and academic achievement (Chang, Lin, & Yeh, 2010; Hsu, 2015; Y. Lin, 2014). Dysfunctional emotion regulation can be destructive to social interaction with peers and to intimate relationship with a significant other. In addition, the paramount importance of academic excellence has profound roots within Taiwanese culture. Many college students, especially from public universities in Taiwan, have continually experienced immense pressure and negative emotions generated from high parental expectations, academic burnout and/or job uncertainty on graduation (K. Lin, 2015; Lu, 2016).

As an important part of educational philosophy and cultural heritage, Confucianism plays a pivotal role in understanding how the Taiwanese manage or regulate emotions. Some valuable concepts from the Chinese cultural context, such as harmony, the golden mean, and interdependence, have profoundly affected the ways of handling emotions among the Taiwanese (Chiang, 2012; Ford & Mauss, 2015; Kwon, Yoon, Joormann, & Kwon, 2013; Sundararajan, 2015). To achieve or maintain social harmony, the principle of the gold mean in Confucianism inhibits emotional expression but promotes the prevalence of emotional restraint. Additionally, a strong emphasis placed on interdependence in Taiwan prioritizes social responsibilities and in-group harmonious relationships rather than self-expression and affective autonomy. Learning how to manage or regulate one’s emotions and emotional dynamics toward goal attainment in an adaptive and socially appropriate manner is unfortunately underscored in college settings.

Research on emotional development in a wide array of research contexts has brought emotion regulation into the research spotlight. We used the database, National Digital Library of Theses and Dissertations in Taiwan, to locate master’s theses and doctoral dissertations related to emotion regulation from 2003 to 2017. The average numbers of theses and dissertations per year for each 5-year period are 6 (in the 2003-2007 period), 12 (in the 2008-2012 period), and 22.8 (in the 2013-2017 period), about a fourfold increase over the past 15 years. Although an increasing amount of research has been conducted to study emotion regulation and its antecedents or outcomes among the Taiwanese, never before have psychometrically validated emotion regulation measures been made available to applied researchers in Taiwan and/or Chinese-speaking societies, as far as we are aware. Given this context, a robust and appropriate assessment tool for evaluating emotion regulation in Chinese is needed as a foundation for research practice and clinical interventions in Chinese research communities all over the world. Specifically, from the perspective of clinical psychology, it is believed that a reliable and valid assessment tool can provide valuable insight into clinical or practical problems, and help facilitate the subsequent development of effective cognitive–behavioral interventions (Goldin et al., 2014; Rottenberg & Gross, 2007).

Psychometric Properties of ERQ Across Cultures

The Emotion Regulation Questionnaire (ERQ) was originally developed by Gross and John (2003) to measure the emotion regulation strategies of cognitive reappraisal and expressive suppression. The coefficients of internal consistency of the scores ranged from .75 to 82 (averaging .79) for cognitive reappraisal and from .68 to .76 (averaging .73) for expressive suppression across four college student samples. The coefficient of test–retest reliability over a 3-month period was reported to have been .69 for both strategies. Results from exploratory factor analysis and confirmatory factor analysis (CFA) revealed that the ERQ was better conceptualized as an independent two-factor measurement model than a correlated two-factor model. That is, students’ scores on use of cognitive reappraisal were unrelated to their degree of using expressive suppression.

Given the practical importance of functional emotion regulation for psychological assessment, several culture-specific versions of the ERQ have been developed and documented, such as the Italian version (Balzarotti et al., 2010); the French version (Christophe, Antoine, Leroy, & Delelis, 2009); the Spanish version (Cabello, Salguero, Fernández-Berrocal, & Gross, 2013); and the German version (Abler & Kessler, 2009; Wiltink et al., 2011). In addition, some researchers have also established age-specific versions of the ERQ, such as the Spanish version of the ERQ for children and adolescents (Gómez-Ortiz, Romera, Ortega-Ruiz, Cabello, & Fernández-Berrocal, 2016); the Chinese version of the ERQ for children and adolescents (W. Liu, Chen, & Tu, 2015); the Australian version of the ERQ for children and adolescents (Gullone & Taffe, 2011); and the Portuguese version of the ERQ for children and adolescents (Teixeira, Silva, Tavares, & Freire, 2014). Across different translations and versions, score reliability, represented by Cronbach’s α values, is generally adequate, ranging from .70 to 84 (averaging .79) for cognitive reappraisal scores and from .65 to .76 (averaging .72) for expressive suppression scores.

Besides evidence from exploratory and confirmatory factor analyses, validity of the meaning of cognitive reappraisal and expressive suppression scores from these instruments has also been established through criterion evidence. Many research attempts have been made to broadly link cognitive reappraisal to constructs such as positive well-being, better social relations, better quality of decision making, lower anxiety and depression, greater experience and expression of positive affect, as well as lesser experience and expression of negative affect (Balzarotti et al., 2010; Fernández-Berrocal & Extremera, 2008; Gross & John, 2003; Gullone & Taffe, 2011; Heilman, Crisan, & Houser, 2010; John & Gross, 2004; Spaapen, Waters, Brummer, Stopa, & Bucks, 2014; Wiltink et al., 2011). Expressive suppression, on the other hand, is predictive of higher levels of psychological distress and depressive symptoms, worse interpersonal functioning, memory impairment, lesser experience and expression of positive affect, and greater experience of negative affect (Balzarotti et al., 2010; Gross & John, 2003; Gullone & Taffe, 2011; Richards & Gross, 2000; Spaapen et al., 2014; Srivastava, Tamir, McGonigal, John, & Gross, 2009; Werner & Gross, 2009; Wiltink et al., 2011). It is clear that the use of cognitive reappraisal brings about more beneficial consequences than expressive suppression.

Although the existing Chinese version of the ERQ for Children and Adolescents has gone through psychometric examination (ERQ-CCA: W. Liu et al., 2015), it was administered in simplified Chinese characters commonly used in Mainland China, rather than in traditional Chinese characters used in Taiwan. In addition, this Chinese version was translated and adapted from the ERQ for Children and Adolescents (ERQ-CA; Gullone & Taffe, 2011) that is not suitable for adult samples. The participants were all elementary school students (ranged from 7 to 12 years) in the study of W. Liu et al. (2015). Results showed that a two-factor measurement model of the ERQ-CCA was confirmed and weak interfactor correlation coefficients of .08 and .10 were observed without reporting associated levels of significance (p values). Thus, to better increase cross-cultural generalizability of the constructs measured in ERQ, there is a strong need to thoroughly examine the psychometric properties of the Chinese version of the ERQ in traditional Chinese characters to obtain the evidence of reliability and validity based on a sample of Taiwanese adults.

Gender Differences on ERQ

Detecting the presence of gender differences in using cognitive reappraisal and expressive suppression strategies appears the most prevalent additional research objective in the existing literature. Prior studies have conducted univariate or multivariate analyses of “observed” scores on two emotion regulation strategies for male and female gender groups (see, e.g., Balzarotti et al., 2010; Gómez-Ortiz et al., 2016; Gross & John, 2003; Gullone & Taffe, 2011; Melka, Lancaster, Bryant, & Rodriguez, 2011; Spaapen et al., 2014; Wiltink et al., 2011). The findings of gender differences in using expressive suppression are consistent, suggesting that males scored significantly higher than did females. Findings about gender differences in using cognitive reappraisal are mixed, but show that female respondents tend to report using cognitive reappraisal no less frequently than (i.e., more than or equal to) their male peers.

One major limitation of previous studies of gender differences in emotion regulation is the common failure to establish measurement invariance before carrying out the gender comparisons, which may raise questions about the meaningfulness of such comparisons and potentially muddle the literature with misleading conclusions. The fulfilment of measurement invariance is widely recognized as a prerequisite for substantive cross-group mean comparisons. Although a few previous studies tested for measurement invariance across genders (see, e.g., W. Liu et al., 2015; Melka et al., 2011; Spaapen et al., 2014), these studies did not proceed to make direct gender comparisons based on the established groundwork of measurement invariance. Therefore, gender differences in using cognitive reappraisal and expressive suppression strategies are not well-founded empirically. Furthermore, gender differences in latent (not only observed) mean scores on these strategies remain entirely unexplored.

Present Study

Because of a positive link between functional emotion regulation and a healthy psychological and physical profile, and the substantive importance of emotion regulation in the Taiwanese cultural context, a thorough psychometric evaluation of the Chinese version of the ERQ is warranted. The goal of this study can be broken down into four specific objectives: (a) to evaluate the psychometric properties of the Chinese version of the ERQ in Taiwanese college student samples, (b) to establish measurement invariance across genders, (c) to identify any gender differences in latent means for use of cognitive reappraisal and expressive suppression strategies, and (d) to gather criterion validity evidence by examining the relationships between scores on the Chinese-ERQ and on the Chinese version of the Life Orientation Test (Chinese-LOT: Li, 2012; LOT: Scheier & Carver, 1985), and on the Chinese version of the Regulatory Focus Questionnaire (Chinese-RFQ: Li & Wu, 2011; RFQ: Lockwood, Jordan, & Kunda, 2002). It was postulated that cognitive reappraisal would be positively associated with optimism and promotion focus, while expressive suppression would be positively associated with pessimism and prevention focus. More specifically, we hypothesized that cognitive reappraisers would reinterpret negative situations in a positive manner and motivate themselves toward the goal with a strategic approach orientation (e.g., advancement or personal growth), while suppressors inhibit expression of feelings from stressful events along with pessimistic attitudes and orient themselves toward the goal with a strategic avoidance orientation (e.g., nonlosses or security).

Method

Participants and Procedure

A total of 909 college students were recruited from 17 universities spread across four geographical areas of Taiwan. Indeed, 735 students (80.9%) were from 13 public universities and 174 students (19.1%) were from 4 private universities. The geographical distribution of participants was 46.1% Northern Taiwan, 18.8% Central, 32.8% Southern, and 2.3% Eastern, which was quite similar to the population distribution of college students in Taiwan (Ministry of Education, Taiwan, 2018): 51.2% (Northern Taiwan), 19.8% (Central), 26.5% (Southern), and 2.5% (Eastern). After selecting these universities, we contacted one to three course instructors and explained the procedure and the goal of the study to them. After the instructors agreed to help data collection in the classroom setting, students were all invited to participate and were informed of the anonymous and confidential nature of participation. Students were also informed that they would receive no credit for their participation in the study. Next, they completed three scales: the Chinese-ERQ, Chinese-LOT, and Chinese-RFQ, along with a brief questionnaire inquiring about their demographic background, during regular university classes.

Of the 909 participants, 481 (52.9%) were female undergraduates and 428 were male undergraduates, with a mean age of 20.31 years (SD = 1.17). The sample was randomly split into two subsamples. One of the resulting two subsamples was one third of the overall sample and used as a calibration sample to confirm whether the factor structure of the Chinese-ERQ would agree with previous cross-culture studies on ERQ. The calibration sample comprised 303 participants (M_age = 20.25 years, SD = 1.15), of whom 155 were female (51.2%) and 148 were male. The second subsample was two third of the overall sample and served as a validation sample for subsequent cross-validation of the CFA model and establishment of measurement invariance. The validation sample (n = 606, M_age = 20.34 years, SD = 1.18) were composed of 326 females (53.8%) and 280 males. This random splitting not only can avoid capitalizing on chance by using the same sample for data analysis but also can help increase generalizability of the results.

Measures

The ERQ was originally developed by Gross and John (2003) to measure two emotion regulation strategies: cognitive reappraisal and expressive suppression. The Chinese translation of the ERQ was first created and then back-translated into English by an independent translator to verify accurate translation of the text. Discrepancies between the original and translated texts were thoroughly discussed until agreement on the best translation into Chinese was reached. Participants were asked to indicate the extent to which they agreed on a 4-point scale (from 1 = strongly disagree to 4 = strongly agree) with six statements related to cognitive reappraisal, such as “When I want to feel less negative emotion, I change the way I’m thinking about the situation” and four statements in relation to expressive suppression, such as “I control my emotions by not expressing them.” No negatively phased items appeared in the questionnaire. Higher scores on each strategy represented higher tendency to use the corresponding emotion regulation strategy.

Among instruments used to assess optimism in social and behavioral research, the LOT (Scheier & Carver, 1985) is the most widely used. The Chinese version of the LOT has undergone rigorous psychometric examination (for further details, see Li, 2012). Student respondents were asked to indicate their agreement with each statement on a 4-point scale (from 1 = strongly disagree to 4 = strongly agree). The scale consists of four items worded in an optimistic manner (e.g., In uncertain times, I usually expect the best) and four items worded in a pessimistic manner (e.g., I rarely count on good things happening to me). Higher scores represented higher levels of generalized optimism and pessimism.

The RFQ (Lockwood et al., 2002) has been used to assess promotion and prevention in many applications. Promotion-focus people actively attain their goals to affirm advancement, growth, and accomplishment, whereas prevention-focus people passively purse their goals to sustain a concern with protection, safety, and responsibility (Crowe & Higgins, 1997; Higgins, 1997). Of the total 12 items in the Chinese version of the RFQ (for details, see Li & Wu, 2011), 6 items were used in the promotion subscale (Cronbach α = .72) and 6 items in the prevention subscale (Cronbach α = .70), respectively, on a 4-point scale (from 1 = strongly disagree to 4 = strongly agree). Example items are “In general, I am focused on achieving positive outcomes in my life” for promotion and “I often imagine myself experiencing bad things that I fear might happen to me” for prevention. Higher scores indicated greater in promotion and prevention self-regulation.

Data Analysis

CFA was conducted on the Chinese-ERQ using Mplus 7.4. Because the observed variables were ordinally scaled and had only four response categories, diagonally weighted least squares estimation with robust corrections to the standard errors and chi-square statistics (ESTIMATOR = WLSMV in Mplus) was used to obtain parameter estimates with minimum bias and maximum precision (Li, 2016a, 2016b). In addition to chi-square statistics, the overall model fit for the hypothesized CFA model was evaluated using common fit indices, including the root mean square error of approximation (RMSEA), comparative fit index (CFI), and Tucker–Lewis index (TLI). As recommended by Hu and Bentler (1999), cutoff values of .06 or below for the RMSEA and values of .95 or above for the CFI and TLI were taken to indicate adequate model fit. Finally, factor congruence coefficients were calculated to evaluate factor similarity across samples (Lorenzo-Seva & ten Berge, 2006; Tucker, 1951), comparing the two subsamples (n = 303 & n = 606) in the present study with the two U.S. samples (n = 336 & n = 791) in the study of Gross and John (2003). A value in the range (0.85, 0.94) corresponds to a fair similarity and a value higher than 0.95 indicates a good similarity (Lorenzo-Seva & ten Berge, 2006).

Next, measurement invariance across genders was tested. There is a general consensus in the methodology literature that testing the equality of the factor structure (i.e., configural invariance, Horn & McArdle, 1992) should come first, followed by testing the equality of the factor loadings (i.e., metric/weak invariance, Horn & McArdle, 1992; Meredith, 1993), and the equality of the intercepts (i.e., scalar/strong invariance, Meredith, 1993) to allow meaningful comparisons of latent means across groups (see, e.g., Bauer, 2017; Millsap, 2011; Vandenberg & Lance, 2000; Yuan & Chan, 2016). Because the observed variables in this study were ordinal, rather than continuous, we tested the equality of the factor loadings and thresholds in tandem, rather than sequentially (Muthén & Muthén, 1998-2015; Schmitt & Kuljanin, 2008). Note that threshold estimates are analogous to intercept estimates when ordered categorical variables are analyzed.

Although multiple-group CFA with nested model chi-square difference tests is a useful tool of assessing measurement invariance, it has been hampered by two potential limitations associated with model identification, as discussed at length in the literature (e.g., Millsap, 2011; Raykov, Marcoulides, & Li, 2012; Rensvold & Cheung, 2001). A two-stage approach for testing measurement invariance proposed by Raykov et al. (2012) was adopted in this study. In the first step, we examined whether the same factor structure was measured across groups. If configural invariance was found to be plausible, we then added constraints of equal factor loadings and equal thresholds simultaneously. Only if this highly constrained model was associated with tenable model fit indices did we intend to then compare gender differences in latent means on cognitive reappraisal and expressive suppression.

To provide reliability evidence, the coefficients of internal consistency (i.e., Cronbach’s α values) were calculated for scores from the entire sample. Given that Cronbach’s α is informative for reliability only in restrictive settings (i.e., true-score equivalent and uncorrelated error, see Raykov, 2004), an index of composite reliability was also computed to avoid potential underestimation of reliability by using Cronbach’s α values for ordinal variables with less than five response categories (Zumbo, Gadermann, & Zeisser, 2007). In addition, ordinal reliability coefficients proposed by Gadermann, Guhn, and Zumbo (2012) using polychoric correlation estimates were also computed.

Results

Factor Structure

In the calibration sample, the measurement model for the Chinese-ERQ item responses was first specified as a correlated two-factor model. The correlated two-factor model demonstrated an adequate fit to the data, χ²(34, n = 303) = 81.09, p < .01; RMSEA = .068 with a 90% confidence interval [CI: .049, .087]; CFI = .98; TLI = .97. The interfactor correlation was quite small and not statistically significant (r = .07, p = .29), indicating that the two latent constructs of reappraisal and suppression are distinct. Next, an independent two-factor model showed a moderate improvement in the goodness of fit and the standardized factor loadings were all significant (see Table 1). A chi-square difference test was performed to evaluate the adequacy of the independent two-factor model using the DIFFTEST option of SAVEDATA and ANALYSIS commands in Mplus, as suggested by Muthén and Muthén (1998-2015). A nonsignificant change in the value of the chi-square statistic (Δχ² = 1.09, Δdf = 1, p = .30) was an indication that the restriction of zero interfactor correlation imposed here did not worsen the overall model fit. These results suggested that the independent two-factor measurement model was a better fit to the data than the correlated two-factor model.

Table 1.

Standardized Factor Loadings, Overall Model Fit Indices, and Congruence Coefficients in the Independent Two-Factor Model for the Calibration and Validation Samples.

Factor loadings (SE)
	Calibration sample (n = 303)	Validation sample (n = 606)
Factor 1: Cognitive reappraisal
CR #1	.65 (.038)	.72 (.023)
CR #3	.79 (.025)	.79 (.021)
CR #5	.53 (.043)	.43 (.035)
CR #7	.73 (.030)	.80 (.021)
CR #8	.83 (.029)	.59 (.031)
CR #10	.84 (.023)	.78 (.024)
Factor 2: Expressive suppression
ES #2	.60 (.049)	.51 (.038)
ES #4	.57 (.055)	.40 (.043)
ES #6	.88 (.041)	.83 (.038)
ES #9	.65 (.043)	.70 (.033)
Overall model fit indices
χ²	60.569	83.698
df	35	35
p	.004	<.001
RMSEA	.049	.048
90% CI for RMSEA	[.027, .069]	[.035, .061]
CFI	.988	.985
TLI	.984	.980
Congruence coefficients
	Calibration sample (n = 303)	Validation sample (n = 606)
Factor 1: Cognitive reappraisal
n = 791 (G & J, 2003)	.969	.961
n = 336 (G & J, 2003)	.945	.938
Factor 2: Expressive suppression
n = 791 (G & J, 2003)	.977	.945
n = 336 (G & J, 2003)	.981	.951

Note. SE = standard error; CFI = comparative fit index; TLI = Tucker–Lewis index; df = degrees of freedom; RMSEA = root mean square error of approximation; CI = confidence interval; G & J, 2003 = Gross and John (2003).

In the validation sample, two competing measurement models for the Chinese-ERQ item responses were estimated. As shown in Table 1, the independent two-factor model yielded a slightly better fit to the data than did the correlated two-factor model (r = .07, p = .14), χ² (34, n = 606) = 98.59, p < .01; RMSEA = .056 with a 90% CI [.043, .069]; CFI = .98; and TLI = .97. Similarly, the p value of the chi-square difference test was .14 (Δχ² = 2.17, Δdf = 1), indicating that the restriction of zero correlation between the cognitive reappraisal and expressive suppression factors was considered tenable. The standardized factor loadings were all significant in Table 1. These results cross-validated that the independent two-factor measurement model was an adequate representation of the structure of the Chinese-ERQ. Finally, the results of the factor similarity analysis are presented in Table 1. Factor congruence coefficients, ranging from .94 to .98, suggested that the independent two-factor structure in the Taiwanese samples is fairly or very similar to that of the U.S. samples (i.e., a fairly high degree of congruence).

Reliability

Coefficients of the internal consistency and composite reliability of the reappraisal and suppression subscales for the entire sample were examined. Note that the composite reliability coefficients were estimated using maximum likelihood estimation. However, these reliability estimates can serve as the lower bound of composite reliability coefficients since maximum likelihood may lead to moderate underestimation of factor loadings when ordinal observed variables are employed. The Cronbach’s alpha and the composite reliability estimates were .78 and .79 (95% CI [.76, .81]) for the reappraisal subscale, respectively, suggesting that reliability of the reappraisal subscale was satisfactory. However, the Cronbach’s alpha and composite reliability estimates for the suppression subscale were .66 and .67 (95% CI [.64, .71]), slightly below the widely accepted cutoff value of .7. This could be possibly due to a negative bias in estimating the reliability for ordinal variables with fewer than five categories (Zumbo et al., 2007). The ordinal reliability coefficients using polychoric correlation estimates were then computed to improve the weakness of these reliability estimates. Finally, the ordinal reliability coefficient was .85 for reappraisal and .72 for suppression, indicating that both subscales were adequate. Taken together, this reliability evidence suggests that the observed scores obtained from the Chinese-ERQ were consistent and reliable.

Criterion Validity

Bivariate Pearson correlation coefficients were calculated between the Chinese-ERQ, the Chinese-LOT, and the Chinese-RFQ scores for the entire sample. As hypothesized, students’ cognitive reappraisal scores were significantly positively associated with their optimism scores (r = .39, p < .01) and promotion scores (r = .31, p < .01); on the other hand, students’ expressive suppression scores were significantly positively associated with their pessimism scores (r = .29, p < .01) and prevention scores (r = .26, p < .01).

In the framework of latent variable modeling, a moderately positive interfactor correlation between cognitive reappraisal and optimism (ϕ = .59, SE = .03, p < .01) and a moderately positive interfactor correlation between cognitive reappraisal and promotion (ϕ = .44, SE = .03, p < .01) were observed; a moderately positive interfactor correlation between expressive suppression and pessimism (ϕ = .43, SE = .04, p < .01) and a significantly positive interfactor correlation between expressive suppression and prevention (ϕ = .36, SE = .04, p < .01) were verified. Note that two correlated three-factor measurement models were used to estimate the above interfactor correlations and each provided an adequate fit to the data: one measurement model of reappraisal, optimism, and promotion constructs (χ² = 678.54, df = 101, p < .01; RMSEA = .079 with a 90% CI [.074, .085]; CFI = .94; TLI = .93) and the other measurement model of suppression, pessimism, and prevention constructs (χ² = 329.26, df = 74, p < .01; RMSEA = .062 with a 90% CI [.055, .068]; CFI = .94; TLI = .92).

Measurement Invariance Across Genders

The results of testing measurement invariance across genders in the validation sample are presented in Table 2. First, the overall model fit of the CFA model was confirmed separately for each gender group. Next, between-gender configural invariance was examined. The baseline model, without equality constraints imposed, fitted the data reasonably well. Also, all significant loadings of observed variables were on their prespecified construct for both gender groups. Since configural invariance was statistically supported, a highly constrained model restricting both the factor loadings and thresholds to equality across genders was tested. Note that this highly constrained model is identified, as suggested by Raykov et al. (2012), by fixing all factor loadings to be equal and setting the variances of the two factors at 1 only in the first group (the female group in this case) and freely estimating factor variances in the second group, rather than fixing the factor loading of each construct to 1 and freely estimating all variances of the two factors in both groups. This scalar invariant model yielded an adequate fit to the data, indicating that the scalar invariant model across genders was plausible.

Table 2.

Tests of Measurement Invariance Across Genders in the Validation Sample.

Models	χ²	df	CFI	TLI	RMSEA [90% CI]
Students
Female only (n = 326)	73.337	35	.980	.975	.058 [.039, .077]
Male only (n = 280)	63.453	35	.977	.971	.054 [.032, .075]
Configural invariance (n = 606)	136.638	70	.979	.973	.056 [.042, .070]
Scalar invariance (n = 606)	190.693	96	.970	.972	.057 [.045, .069]

Note. CFI = comparative fit index; TLI = Tucker–Lewis index; RMSEA = root mean square error of approximation; CI = confidence interval; df = degrees of freedom.

Finally, any latent gender differences in reappraisal and suppression were tested by setting the latent means of the female group to 0 (serving as the reference group), and testing whether the latent means of male students differed from 0 in the population. This testing procedure was internally implemented by Mplus while testing the scalar invariant model. The effect size of gender differences (latent effect size d) was calculated using latent mean differences and pooled latent variances (Hancock, 2001). A value of d = .2 was interpreted as a small effect, d = .5 as a medium effect, and d = .8 as a large effect. Results showed that there was a small latent mean difference in cognitive reappraisal (M_diff = −.20, SE = .10, t = −2.05, p = .04, effect size d = −.19), suggesting that female students scored significantly higher on reappraisal than did male students. A large latent mean difference in expressive suppression was observed (M_diff = .77, SE = .11, t = 6.86, p < .01, effect size d = .75). This implies that male students reported using suppression to a greater degree than female students.

Discussion

The present study sought to develop and validate a Chinese version of the ERQ and examine it in Taiwanese college students to ensure that the psychometric properties of the Chinese-ERQ were as sound as those demonstrated for previous culture-specific translations on the instrument. Overall, this study makes three contributions. First, through rigorous psychometric evaluation, the factor structure of the Chinese-ERQ is empirically validated as a two-factor orthogonal measurement model underlying the original ERQ and the Chinese-ERQ is a valuable assessment tool for emotion regulation in terms of reliability and validity evidence. Second, we establish measurement invariance across genders prior to embarking on gender latent mean comparisons and find a significant gender difference in the use of cognitive reappraisal, with a higher level of cognitive change evident for female than for male college students in the Taiwanese cultural context. Finally, we make one methodological contribution by introducing the two-stage approach for testing measurement invariance with ordinal data to circumvent the two potential limitations associated with model identification in the multiple-group CFA.

Although a few studies reported mixed findings of weak correlation between reappraisal and suppression in either a positive (r = .20, Wiltink et al., 2011; r = .34, Gómez-Ortiz et al., 2016) or negative direction (r = −.13, Gullone & Taffe, 2011), the relationship between reappraisal and suppression was found to be orthogonal in the present study, consistent with a majority of empirical studies in the literature (Balzarotti et al., 2010; Cabello et al., 2013; Christophe et al., 2009; Gross & John, 2003; Melka et al., 2011; Teixeira et al., 2014). This finding suggests that there is no linear relationship between reappraisal and suppression in most populations. One practical implication of this result for educators and school counselors is the implied need to devise separate cognitive–behavioral programs (a) promoting students’ ability to manage their emotions strategically and (b) discouraging students from hiding their emotions skillfully, even if unintentionally. Particularly, screening for habitual suppression may help better predict, and through intervention reduce, maladaptive functioning among Taiwanese college students. For example, normative data for the Chinese version of the ERQ could be obtained from a large, randomly selected representative sample from the general college student population in Taiwan. Furthermore, population-based norms calculated using percentiles (cumulative percentages) and predetermined criterion-referenced values for each gender group would assist educators and school counselors in better identifying potential habitual suppressors.

To the best of our knowledge, this study represents the first attempt to compare gender latent mean differences in reappraisal and suppression after establishment of measurement invariance (i.e., scalar equivalence) across genders. As expected on the basis of previous studies (Balzarotti et al., 2010; Gómez-Ortiz et al., 2016; Gross & John, 2003; Gullone & Taffe, 2011; Melka et al., 2011; Spaapen et al., 2014; Wiltink et al., 2011), the present investigation showed that male college students engaged in significantly more expressive suppression than female college students did. This observation joins the growing literature and highlights the social norm that men, both in Western and Eastern societies, are often educated to inhibit negative emotions related to sadness and fear to a greater extent than women are, for example, crying, by the process of socialization and cultural values transmitted from their childhood. That is, expressing too often sadness and fear gives the impression that one is too vulnerable, which is normatively perceived as inappropriate for men. Another educational implication of this finding is that educators and school counselors should aim to actively identify emotional suppressors among male students, and assist them in developing more adaptive emotion regulatory styles.

It bears noting that cognitive reappraisal showed greater prevalence in female college students, which is in accord with results of one previous study utilizing samples from Australia and the United Kingdom (Spaapen et al., 2014). However, this finding stands in contrast to that of most previous studies, in which no gender difference in the use of cognitive reappraisal was observed (see, e.g., Gross & John, 2003; Gullone & Taffe, 2011). One possible explanation would be that parents in Taiwan tend to encourage their daughters to (a) express emotions to receive social support, (b) make sacrifices for their family members because of social obligations, and (c) yield to others for social mindfulness. Moreover, female college students in Taiwan like to share their emotions with close others and seek advice and assistance from peer support groups. Altogether, female Taiwanese college students have been taught to be considerate and attentive, which can help them produce more refined emotions and behavioral responses, compared with their male counterparts. However, future research is needed to determine whether this finding can be replicated, particularly in the Taiwanese cultural context.

Criterion validity evidence of score correlations consistently in the predicted direction provided additional support of the meaning of scores from the Chinese-ERQ. Optimism was significantly positively correlated with cognitive reappraisal and pessimism was significantly positively correlated with expressive suppression. This finding underscores that of Gross and John (2003), who demonstrated that habitually cognitive reappraisal users had a higher level of optimism, whereas frequent suppressors tended to have increased pessimistic attitudes. That is, reappraisers are more likely than suppressors to be able to skillfully negotiate setbacks or frustrations by reinterpreting what they find stressful in a more optimistic light. In addition, promotion inclination was significantly positively associated with cognitive reappraisal and prevention inclination was significantly positively associated with expressive suppression, which echoed the findings of Yan’s (2012) study. Namely, reappraisal-oriented individuals are more likely motivated toward achieving favorable outcomes, while individuals who use suppression are more likely oriented toward avoiding undesirable events.

The design of the present study was not without limitations. Although participants represented geographically diverse areas of the country and had a balanced gender composition, the sample was limited solely to Taiwanese college students within the same age-based cohort. A natural extension of this study would consist of bolstering the external validity surrounding the measurement of emotion regulation strategies by using a much wider population, such as children and adolescents as well as older adults. Also, the number of children of foreign-born mothers is rapidly increasing in Taiwan. More than 90% of immigrant mothers are from Southeast Asia (Ministry of the Interior, Taiwan, 2018), such as Vietnam (62.9%), Indonesia (18.3%), Philippines (5.4%), Thailand (3.7%), and Cambodia (2.7%). Children born to these immigrant mothers are more likely to encounter significant social marginalization in educational settings due to family risk factors, such as socioeconomic disadvantage, language and communication barriers, and cross-cultural maladaptation (X. Liu, 2003; Ministry of the Interior, Taiwan, 2014, 2016; Wang, 2013; Wu, 2015). Besides measurement invariance across genders, an interesting avenue of further investigation may consider whether there are differences in use of these two emotion regulation strategies between children of native-born mothers and those of foreign-born mothers. Continued efforts to collect validity evidence using samples of students with varied socioeconomic status in Taiwan are also suggested. Self-report measurement bias of the Chinese-ERQ is another possible threat to the meaning of the scores, so convergent evidence gleaned from additional methods of data collection (e.g., peer reports or observer ratings) might provide further assurance that the scores have the meanings intended.

In summary, the ERQ has been widely used to measure two emotion regulation strategies across different cultures, yet psychometric examination of a Chinese version of the ERQ is limited. The provision of a valid and reliable instrument to measure emotion regulation strategies in educational settings can help psychologists and practitioners better monitor students’ emotion regulation, and link it to their affective and social functioning. Given that college students experience frequent and intense emotions in the Taiwanese cultural context, the importance of honing students’ emotional skills cannot be overemphasized. The current study provides solid evidence that the Chinese-ERQ is a valuable tool for assessing emotion regulation strategies in research and practice in clinical psychology, and a broad range of other psychological research areas as well.

Footnotes

Acknowledgements

We thank Anne Traynor for her valuable comments on an earlier version of this article.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Cheng-Hsien Li

References

Abler

Kessler

(2009). Emotion Regulation Questionnaire—Eine deutschsprachige Fassung des ERQ von Gross und John [Emotion Regulation Questionnaire—A German version of the ERQ by Gross and John]. Diagnostica, 55, 144-152.

Balzarotti

John

O. P.

Gross

J. J.

(2010). An Italian adaptation of the Emotion Regulation Questionnaire. European Journal of Psychological Assessment, 26, 61-67.

Bauer

D. J.

(2017). A more general model for testing measurement invariance and differential item functioning. Psychological Methods, 22, 507-526.

Cabello

Salguero

J. M.

Fernández-Berrocal

Gross

J. J.

(2013). A Spanish adaptation of the Emotion Regulation Questionnaire. European Journal of Psychological Assessment, 29, 234-240.

Chang

Lin

Yeh

(2010). The study on the relationships among emotional intelligence, interpersonal relationships and self-concept of college students. Archive of Guidance & Counseling, 32, 1-25.

Chiang

(2012). The suppression of emotional expression in interpersonal context. Bulletin of Educational Psychology, 43, 657-680.

Christophe

Antoine

Leroy

Delelis

(2009). Assessment of two emotional regulation processes: Expressive suppression and cognitive reevaluation. European Review of Applied Psychology, 59, 59-67.

Crowe

Higgins

E. T.

(1997). Regulatory focus and strategic inclinations: Promotion and prevention in decision-making. Organizational Behavior and Human Decision Processes, 69, 117-132.

Fernández-Berrocal

Extremera

(2008). A review of trait meta-mood research. International Journal of Psychology Research, 2, 39-67.

10.

Ford

B. Q.

Mauss

I. B.

(2015). Culture and emotion regulation. Current Opinion in Psychology, 3, 1-5.

11.

Gadermann

A. M.

Guhn

Zumbo

B. D.

(2012). Estimating ordinal reliability for Likert-type and ordinal item response data: A conceptual, empirical, and practical guide. Practical Assessment, Research & Evaluation, 17, 1-13.

12.

Goldin

P. R.

Lee

Ziv

Jzsaieri

Heimberg

R. G.

Gross

J. J.

(2014). Trajectories of change in emotion regulation and social anxiety during cognitive-behavioral therapy for social anxiety disorder. Behaviour Research and Therapy, 56, 7-15.

13.

Gómez-Ortiz

Romera

E. M.

Ortega-Ruiz

Cabello

Fernández-Berrocal

(2016). Analysis of emotion regulation in Spanish adolescents: Validation of the Emotion Regulation Questionnaire. Frontiers in Psychology, 6, 1959. doi:10.3389/fpsyg.2015.01959

14.

Gross

J. J.

John

O. P.

(2003). Individual differences in two emotion regulation processes: Implications for affect, relationships and well-being. Journal of Personality and Social Psychology, 85, 348-362.

15.

Gullone

Taffe

(2011). The Emotion Regulation Questionnaire for Children and Adolescents (ERQ-CA): A psychometric evaluation. Psychological Assessment, 24, 409-417.

16.

Hancock

G. R.

(2001). Effect size, power, and sample size determination for structured means modeling and MIMIC approaches to between-groups hypothesis testing of means on a single latent construct. Psychometrika, 66, 373-388.

17.

Heilman

R. M.

Crisan

L. G.

Houser

(2010). Emotion regulation and decision making under risk uncertainty. Emotion, 10, 257-265.

18.

Higgins

E. T.

(1997). Beyond pleasure and pain. American Psychologist, 52, 1280-1300.

19.

Horn

J. L.

McArdle

J. J.

(1992). A practical and theoretical guide to measurement invariance in aging research. Experimental Aging Research, 18, 117-144.

20.

Hsu

(2015). The relations between partner dependency, boundary and ambivalence over emotional expression on college students (Unpublished master’s thesis). Ming Chuan University, Taipei, Taiwan.

21.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.

22.

John

O. P.

Gross

J. J.

(2004). Healthy and unhealthy emotion regulation: Personality processes, individual differences, and life span development. Journal of Personality, 72, 1301-1334.

23.

Kwon

Yoon

K. L.

Joormann

Kwon

(2013). Cultural and gender differences in emotion regulation: Relation to depression. Cognition and Emotion, 27, 769-782.

24.

(2012). Validation of the Chinese version of the Life Orientation Test with a robust weighted least squares approach. Psychological Assessment, 24, 770-776.

25.

(2016a). Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares. Behavior Research Methods, 48, 936-949.

26.

(2016b). The performance of ML, DWLS, and ULS estimation with robust corrections in structural equation models with ordinal variables. Psychological Methods, 21, 369-387.

27.

(2011). The structural relationships between optimism and innovative behavior: Understanding potential antecedents and mediating effects. Creativity Research Journal, 23, 119-128.

28.

Lin

(2015). The relationship among academic emotions, academic burnout, and learning engagement and the moderating effect of classroom goal structures (Unpublished master’s thesis). National Cheng Kung University, Tainan, Taiwan.

29.

Lin

(2014). A study of the relationship between academic stress and emotional management: An example of college students in National Taichung University of Education (Unpublished master’s thesis). National Taichung University of Education, Taichung, Taiwan.

30.

Liu

(2003). A study on the family environment of foreign brides and their children’s behavior under cross-cultural adjustment shock (Unpublished master’s thesis). National Chung Cheng University, Chiayi, Taiwan.

31.

Liu

Chen

(2015). Chinese adaptation of Emotion Regulation Questionnaire for Children and Adolescents (ERQ-CCA): A psychometric evaluation in Chinese children. International Journal of Psychology, 52, 398-405.

32.

Lockwood

Jordan

C. H.

Kunda

(2002). Motivation by positive or negative role models: Regulatory focus determines who will best inspire us. Journal of Personality and Social Psychology, 83, 854-864.

33.

Lorenzo-Seva

ten Berge

J. M. F.

(2006). Tucker’s congruence coefficients as a meaningful index of factor similarity. Methodology, 2, 57-64.

34.

(2016). The relationships among college students’ work value, career anxiety, and career adaptability (Unpublished master’s thesis). National Taichung University of Education, Taichung, Taiwan.

35.

Melka

S. E.

Lancaster

S. L.

Bryant

A. R.

Rodriguez

B. F.

(2011). Confirmatory factor and measurement invariance analyses of the Emotion Regulation Questionnaire. Journal of Clinical Psychology, 67, 1283-1293.

36.

Meredith

(1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58, 525-543.

37.

Millsap

R. E.

(2011). Statistical approaches to measurement invariance. New York, NY: Routledge.

38.

Ministry of Education, Taiwan. (2018). Students in universities, colleges, and junior colleges. Retrieved from http://depart.moe.edu.tw/ED4500/News.aspx?n=5A930C32CC6C3818&sms=91B3AAE8C6388B96

39.

Ministry of the Interior, Taiwan. (2014). A study on disadvantaged foreign spouses in Taiwan. Retrieved from https://www.immigration.gov.tw/public/Data/51269314471.pdf

40.

Ministry of the Interior, Taiwan. (2016). The impact of biculturalism on identity among Taiwanese immigrants and their children. Retrieved from https://www.immigration.gov.tw/public/attachment/731311353918.pdf

41.

Ministry of the Interior, Taiwan. (2018). Foreign spouses by nationality from National Immigration Agency. Retrieved from https://www.immigration.gov.tw/ct.asp?xItem=1343512&ctNode=29699&mp=1

42.

Muthén

L. K.

Muthén

B. O.

(1998-2015). Mplus user’s guide (7th ed.). Los Angeles, CA: Muthén & Muthén.

43.

Raykov

(2004). Behavioral scale reliability and measurement invariance evaluation using latent variable modeling. Behavior Therapy, 35, 299-331.

44.

Raykov

Marcoulides

G. A.

(2012). Measurement invariance for latent constructs in multiple populations: A critical view and refocus. Educational and Psychological Measurement, 72, 954-974.

45.

Rensvold

R. B.

Cheung

G. W.

(2001). Testing for metric invariance using structural equation models: Solving the standardization problem. In Schresheim

C. A.

Neider

L. L.

(Eds.), Equivalence in measurement: Research in management (pp. 25-50). Greenwich, CT: Information Age.

46.

Richards

J. M.

Gross

J. J.

(2000). Emotion regulation and memory: The cognitive costs of keeping one’s cool. Journal of Personality and Social Psychology, 79, 410-424.

47.

Rottenberg

Gross

J. J.

(2007). Emotion and emotion regulation. A map for psychotherapy researchers. Clinical Psychology: Science and Practice, 14, 323-328.

48.

Scheier

M. F.

Carver

(1985). Optimism, coping, and health: Assessment and implications of generalized outcome expectancies. Health Psychology, 4, 219-247.

49.

Schmitt

Kuljanin

(2008). Measurement invariance: Review of practice and implications. Human Resource Management Review, 18, 210-222.

50.

Spaapen

D. L.

Waters

Brummer

Stopa

Bucks

S. R.

(2014). The Emotion Regulation Questionnaire: Validation of the ERQ-9 in two community samples. Psychological Assessment, 26, 46-54.

51.

Srivastava

Tamir

McGonigal

K. M.

John

O. P.

Gross

J. J.

(2009). The social costs of emotional suppression: A prospective study of the transition to college. Journal of Personality and Social Psychology, 96, 883-897.

52.

Sundararajan

(2015). Understanding emotion in Chinese culture: Thinking through psychology. New York, NY: Springer SBM.

53.

Teixeira

Silva

Tavares

Freire

(2014). Portuguese validation of the Emotion Regulation Questionnaire for children and adolescents (ERQ-CA): Relations with self-esteem and life satisfaction. Child Indicator Research, 8, 605-621.

54.

Tucker

L. R.

(1951). A method for synthesis of factor analysis studies (Personnel Research Section Report No. 984). Washington, DC: Department of the Army.

55.

Vandenberg

R. J.

Lance

C. E.

(2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4-70.

56.

Wang

(2013). The relationships of cultural-capital, school adjustment, and academic achievement in elementary school children of new inhabitants in Taiwan (Unpublished master’s thesis). Taiwan Shoufu University, Tainan, Taiwan.

57.

Werner

Gross

J. J.

(2009). Emotion regulation and psychopathology. In Kring

A. M.

Sloan

D. M.

(Eds.), Emotion regulation and psychopathology: A transdiagnostic approach to etiology and treatment (pp. 3-37). New York, NY: Guilford.

58.

Wiltink

Glaesmer

Canterino

Wölfling

Knebel

Kessler

. . . Buetel

M. E

. (2011). Regulation of emotions in the community: Suppression and reappraisal strategies and its psychometric properties. GMS Psycho-Social-Medicine, 8, 1-12.

59.

(2015). A comparative study of positive psychology, life adjustment, and hope of native and immigrant adolescents (Unpublished master’s thesis). National Changhua University of Education, Changhua, Taiwan.

60.

Yan

E. M.

(2012). Children’s emotion regulation in unfair situations: Using regulatory focus theory (Unpublished doctoral dissertation). Massey University, Manawatu, New Zealand.

61.

Yuan

K. H.

Chan

(2016). Measurement invariance via multigroup SEM: Issues and solutions with chi-square-difference test. Psychological Methods, 21, 405-426.

62.

Zumbo

B. D.

Gadermann

A. M.

Zeisser

(2007). Ordinal versions of coefficient alpha and theta for Likert rating scales. Journal of Modern Applied Statistical Methods, 6, 21-29.