Abstract
This study examined statistical bias in the measurement of personality psychopathology in the Latinx population using the Minnesota Multiphasic Personality Inventory-3 (MMPI-3). Data were extracted from two studies that yielded a composite data set of 103 White individuals and 250 Latinx individuals. All participants were administered the MMPI-2-Restructured Form-Extended Battery (MMPI-2-RF-EX) or MMPI-3 and the Personality Inventory for the DSM-5 Short Form (PID-5-SF). First, we conducted correlation analyses between theoretically overlapping scales of the PID-5-SF and the MMPI-3 among White and Latinx individuals. The majority of theoretically associated scales were found to be at least moderately associated in the total sample. In addition, Steiger’s z-tests indicated that correlations were similar in magnitude across the White and Latinx ethnic groups. Hierarchical regression subsequently determined the presence of slope and/or intercept bias. Only one analysis (the MMPI-3 Anger Proneness prediction of PID-5-SF Negative Affectivity) indicated statistically significant intercept bias. No evidence of slope bias was found. In other words, these analyses indicated that the vast majority of the relationships between MMPI-3 scales and associated personality psychopathology constructs (as measured by the PID-5-SF) remained consistent across both ethnic groups. Overall, the results supported the appropriate cross-cultural use of the MMPI-3 to assess personality psychopathology.
In recent decades, the demographic landscape of the United States has become more diverse than ever before. This is especially evidenced by the rise in the Latinx population, which has increased from comprising 7% of the total U.S. population in 1980 to 19% of the population in 2021 (Moslimani & Noe-Bustamante, 2023). The U.S. Census Bureau (2017) predicts the Latinx community will only continue to grow and is estimated to make up over one quarter of the national population by 2060 (Colby & Ortman, 2015). Such exponential growth of the Latinx community holds important implications for the accuracy of psychological assessment, which is essential for establishing equitable mental health care within this newly emerging landscape.
As part of this effort, it is important to consider that the expression of psychopathology is known to differ across cultural groups (Chapman et al., 2014). For example, those who experience mental illness within the Latinx community seem to present with symptomatology that is more chronic and pervasive as compared to their non-Hispanic White counterparts (Breslau et al., 2005). This study also found that Latinx individuals present with more distress at intake to mental health services than both African American and White individuals. This suggests delays from symptom onset to treatment, which may be resulting in higher levels of need and missed opportunities for prevention and early intervention for Latinx communities (Lipson et al., 2018). Indeed, discrimination against the Latinx population and other cultural-specific stressors (e.g., immigration) are potentially causing more life stressors that could both increase symptom burden and adversely impact mental health across the ethnic group (Gudiño et al., 2011; Philbin et al., 2018). Of relevance to the current study, existing research has underscored the importance of culturally informed methods of personality assessment in the Latinx population due to possible cultural differences in response patterns (Hopwood et al., 2009) and the expression of symptoms of personality disorders (Alamilla & Wojcik, 2013).
However, difficulties exist that have thus far imposed significant barriers for access to equitable mental health care in the Latinx community (e.g., Chapman et al., 2014; Turner et al., 2016). Research has highlighted a variety of biases—including cultural, language, and socioeconomic biases—that could affect psychological assessment in this population (Cervantes & Acosta, 1992; Malgady et al., 1987). Latinx individuals may be misdiagnosed due to language barriers, lack of cultural awareness by clinicians, or the decreased likelihood of client disclosure to dyadically mismatched clinicians (Hamilton et al., 2018; Minsky et al., 2003).
Yet, Latinx mental health service utilization remains 50% lower than that of non-Latinx Whites even when accounting for ethnic differences in external barriers (Benuto et al., 2019). This occurrence could present a mental health crisis in the United States for Latinx individuals. Therefore, it is imperative that the psychology field comprehensively addresses differences in mental health care for Latinx communities, including cross-cultural accuracy in clinical practice across the realms of assessment, diagnosis, and treatment.
Assessment and Diagnosis of Personality Psychopathology
In addition to issues related to cross-cultural assessment, there are also important broader considerations in assessment and diagnosis. The conceptualization of personality disorder (PD) models has increasingly been a topic of contention among clinicians and researchers alike (e.g., Krueger et al., 2007). Specifically, categorical methods for PD diagnosis have been largely questioned as the field has moved toward the use of dimensional models and measures. In divergence from categorical models, dimensional models allow for symptom-level analysis, rather than grouping individuals into a category based on common criteria (Hopwood et al., 2018).
The Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM-5) took a hybrid approach by introducing the Alternative Model for Personality Disorders (AMPD) in Section III (Emerging Models and Measures) of the manual. The AMPD attempts to strike a balance between dimensional and categorical conceptualizations of PDs and introduces dimensional elements with substantial empirical support while simultaneously preserving the continuity of DSM categories that have demonstrated clinical traction (American Psychiatric Association [APA], 2013). The AMPD defines personality disorders as a combination of deficits in personality functioning and pathological personality traits. These traits are defined as 25 facets organized into five higher-order trait dimensions: Negative Affectivity, Detachment, Antagonism, Disinhibition, and Psychoticism. This model has been empirically supported, both in its own five-factor structure (e.g., Wright et al., 2012) and in its general alignment with other models of normal and pathological personality (e.g., Anderson et al., 2013; Gore & Widiger, 2013; McCabe & Widiger, 2020; Thomas et al., 2013). In order to create a bridge between this model and existing clinical practice, these domains and facets are also organized into proposed diagnostic criteria for six categorical diagnoses. The clinical utility and construct validity of this model have strongly supported the continued use of dimensional models in practice (Milinkovic & Tiliopoulos, 2020; Watters et al., 2019).
Relevant to the current study, a measure was created to assess this personality trait model called the Personality Inventory for the DSM-5 (PID-5; Krueger et al., 2012). Shorter versions of this 220-item measure have also been developed, including the 100-item PID-5-Short Form (PID-5-SF; Maples et al., 2015) and the 25-item PID-5-Brief Form (PID-5-BF; Krueger et al., 2013). Previous research has supported the PID-5 in terms of factor structure, reliability, and validity (see Al-Dajani et al., 2016; Krueger & Markon, 2014, for reviews). Although minimal work has focused on potential bias, research on the PID-5 has supported the use of the instrument in multiple populations (e.g., Bach et al., 2018; Dunne et al., 2018) and various ethnic groups (e.g., Freilich et al., 2023; Zhang et al., 2022). Likewise, research has supported the psychometric properties of both the PID-5-SF and the PID-5-BF (Gomez et al., 2020; Thimm et al., 2016). Importantly, previous research has also supported the validity and reliability of both the full-length PID-5 and the PID-5-SF in a Spanish national sample (Aluja et al., 2019; García et al., 2022; Gutiérrez et al., 2017). Even so, many clinicians may be hesitant to incorporate another measure into their assessment batteries, necessitating the use of broadband measures in assessing personality psychopathology.
Personality Disorder Assessment With the MMPI
One of the most important developments in the area of dimensional assessment has been the Minnesota Multiphasic Inventory-2-Restructured Form (MMPI-2-RF; Ben-Porath & Tellegen, 2008/2011). One advantage of the MMPI-2-RF is its extensive history of utility in assessing personality psychopathology using both categorical and dimensional conceptualizations (e.g., Anderson & Sellbom, 2021). Indeed, a large body of previous research has demonstrated the ability of the MMPI-2-RF to accurately assess categorical PD symptoms (e.g., Anderson et al., 2015, 2018; De Saeger et al., 2020; Kamphuis et al., 2008; Sellbom et al., 2014; Sellbom & Smith, 2017; Van der Heijden et al., 2012; Zahn et al., 2017) as well as dimensional pathological traits (e.g., Anderson et al., 2015; Sellbom et al., 2013).
Despite the use of the MMPI-2-RF in multiple settings and across multiple populations (Sellbom, 2019), the study of psychopathology in the Latinx population has been scarce in the MMPI literature. Notably, although the normative sample represented projected census data at the time of publication of the MMPI-2 (Butcher et al., 1989), the use these norms in developing the MMPI-2-RF led to poor ethnic representation of the normative sample for the modern U.S. demographic. Having appropriate reference groups is imperative in producing accurate personality measure reports and interpretations (Gaddis et al., 2015). Thus, the lack of a representative norming sample may negatively affect the results of the MMPI-2-RF. Even so, studies of the MMPI-2-RF in diverse populations have indicated that the measure is unbiased across White, African American, and Hispanic/Latinx 1 ethnic groups (Marek et al., 2015; Whitman et al., 2019). However, the lack of representation in the normative sample remains problematic. For instance, it has been found that a substantially higher proportion of Latinx individuals produce invalid MMPI-2-RF profiles than do Non-Latinx White individuals at both a statistical and clinically significant level (Benuto et al., 2020).
The more recently released MMPI-3 intended to improve upon this problem. Though there remain many similarities carried over from the MMPI-2-RF in terms of its scale make-up and structure, the update provided an opportunity to enhance the item pool, optimize existing scales, and introduce new scales where warranted (Ben-Porath, & Tellegen, 2020). Test norms were also updated for the first time since the publication of the MMPI-2 in 1989 in order to match the projected ethnicity distributions of the 2020 United States census (Sellbom, 2019). Irrespective of recent emphasis on its importance, there remains a dearth of literature on potential measurement bias, and research using ethnic minorities has been historically limited (American Psychological Association [APA], 2002; Leong & Park, 2016). Moreover, the recency of the MMPI-3’s development has meant limited opportunity for such work to be conducted thus far. Overall, with the national population demographics shifting alongside the field’s movement toward the use of dimensional measures, understanding how well the new MMPI-3 instrument operationalizes accurate personality psychopathology in the Latinx subgroup is an important cross-section that needs to be addressed by assessment research.
Current Study
This study aimed to assess potential bias in the measurement of personality psychopathology within the Latinx population using the MMPI-3. This was investigated by first studying the associations between theoretically overlapping scales of the PID-5-SF and the MMPI-3 among non-Hispanic White and Latinx individuals. Hypothesized associations were drawn from past research literature (e.g., Anderson et al., 2013, 2015; Anderson & Sellbom, 2021) and applied knowledge of scale constructs. Further, we took a conservative approach in deriving hypotheses to increase the likelihood of capturing bias if it exists. As such, the hypotheses include not only theoretically matched concepts but also constructs that we would expect to show moderate to strong correlations with one another. A full list of conceptually expected associations is shown in Table 1.
Hypothesized Associations.
Note. EID = Emotional/Internalizing Dysfunction; BXD = Behavioral/Externalizing Dysfunction; THD = Thought Dysfunction; RCd = Demoralization; RC1 = Somatic Complaints; RC2 = Low Positive Emotions; RC4 = Antisocial Behaviors; RC6 = Ideas of Persecution; RC7 = Dysfunctional Negative Emotions; RC8 = Aberrant Experiences; RC9 = Hypomanic Activation; MLS = Malaise; NUC = Neurological Complaints; EAT = Eating Concerns; COG = Cognitive Complaints; SUI = Suicidal/Death Ideation; HLP = Helplessness/Hopelessness; SFD = Self-Doubt; NFC = Inefficacy; STR = Stress; WRY = Worry; CMP = Compulsivity; ARX = Anxiety-Related Experiences; ANP = Anger Proneness; BRF = Behavior-Restricting fears; FML = Family Problems; JCP = Juvenile Conduct Problems; SUB = Substance Abuse; IMP = Impulsivity; ACT = Activation; AGG = Aggression; CYN = Cynicism; SFI = Self-Importance; DOM = Dominance; DSF = Disaffiliativeness; SAV = Social Avoidance; SHY = Shyness; AGGR = Aggressiveness; PSYC = Psychoticism; DISC = Disconstraint; NEGE = Negative Emotionality/Neuroticism; INTR = Introversion/Low Positive Emotionality.
In addition, data analyses determined whether the relationships between MMPI-3 scales and associated personality psychopathology constructs (as measured by the PID-5-SF) remained consistent across ethnic groups. Using correlation analyses in Latinx and non-Latinx White groups, we first examined differences across groups in association magnitudes. Next, we used slope and intercept bias analyses to assess for potential differences. The presence of intercept bias would imply that the two regression lines for each ethnicity run parallel. In other words, intercept bias implies that one ethnic group is predicted to have higher scores on the PID-5 than the other ethnic group across the range of MMPI-3 scale scores. On the other hand, the presence of slope bias implies a difference in the rate the dependent variable increases by ethnicity. In this case, slope bias would imply that the difference in the predicted PID-5 scores by ethnicity would become more exaggerated as the MMPI-3 scale score increases.
Based on previous research supporting racial/ethnic equivalence when using the MMPI-2-RF (Marek et al., 2015; Whitman et al., 2019), slope and/or intercept bias in the MMPI-3 assessment of Latinx participants was not expected. Even so, the data analysis provided an opportunity for an important exploration of this possibility on this new measure and with its new norms. By establishing the presence or absence of bias in MMPI-3 assessment of the Latinx population, it would either validate the use of the MMPI-3 in accurately measuring personality psychopathology in this ethnic group or allow for future correction and improvement of the measure for the same purposes.
Method
Participants
The sample used is a combination of undergraduate participants from two separate studies. The treatment of all participants was in accordance with the ethical standards of the American Psychological Association and the studies were approved by the Sam Houston State University (SHSU) and University of Texas Rio Grande Valley (UTRGV) Institutional Review Boards (IRB).We report how we determined our sample size, all data exclusions, all manipulations, and all measures in the study.
Study 1
Sample 1 included 337 undergraduate participants from a southern university (SHSU). One hundred and thirty-five individuals were excluded because they did not identify their ethnicity as either White or Latinx. In addition, 19 individuals were excluded due to invalid MMPI-3 protocols (defined below), leaving 183 valid protocols for data analysis. In this sample, 100 participants identified as White/Non-Latinx (84% female, Mage = 21.4) and 83 participants identified as Latinx (80.7% female, Mage = 20.1). Participants were recruited through the university’s psychology research platform.
Study 2
Sample 2 included 221 undergraduate participants, all of which identified as bilingual Spanish/English speakers. Six individuals were excluded because they did not identify their ethnicity and four were excluded because they did not identify their ethnicity as either White or Latinx. Forty individuals were excluded due to invalid MMPI-3 protocols (defined below) and one individual was excluded due to incompletion of the PID-5-SF, leaving 170 valid protocols for data analysis (82.4% Female, Mage = 20.8); three identified as White and 167 identified as Latinx. Participants from this study were recruited at two universities in the Southwestern United States (SHSU and UTRGV). According to our demographics survey, approximately 65% of this sample was born in the United States. This study also included a survey of the participant’s level of acculturation using the Abbreviated Multidimensional Acculturation Scale (AMAS; Zea et al., 2003). Although an entire discussion of acculturation is outside of the scope of the study, it is important to note that 52% of the participants in this study reported they strongly agree with the statement that they think of themselves as a U.S. American and 32% reported they somewhat agree with the statement. In addition, 32% of participants reported they strongly agree that they are a part of U.S. American culture, and 42% said they agree somewhat with the statement.
The overall sample included 250 Latinx participants (82.0% female; Mage = 20.6) and 103 White/Non-Latinx participants (84.5% female; Mage = 21.4). Of note, regardless of possible variations in the frequency with which each participant uses the Spanish language in their daily life (if at all), all participants attend an English-speaking university and are thus considered fully proficient in the English language. The White/Non-Latinx sample was chosen to serve as our comparison group because most of the research involving personality assessment has historically been conducted in predominantly White samples. Prior to analysis, we used an a priori power analysis using a small to medium effect size (p value = .05, power = .8) for a regression model with three predictors. This analysis suggested a sample size of 141 was necessary, indicating our sample was adequately powered for this study.
Measures
Participants from both studies completed the following measures.
Personality Inventory for the DSM-5-Short Form (PID-5-SF)
The PID-5-SF (Krueger et al., 2012) is a shortened version of the PID-5 that is used to measure Criterion B of the AMPD (APA, 2013). From the original 220-item inventory, Maples et al. (2015) used item response theory to extract 100 items to measure the same domains and facets of the full inventory. It assesses 25 personality trait facets which combine to yield indices of the five broader trait domains of Negative Affect, Detachment, Antagonism, Disinhibition, and Psychoticism. Each item on the measure is rated on a 4-point scale. The response categories for the items are 0 = very false or often false; 1 = sometimes or somewhat false; 2 = sometimes or somewhat true; and 3 = very true or often true.
Minnesota Personality Inventory-2-Restructured Form-Expanded Version (MMPI-2-RF-EX)
Before the official release of the MMPI-3, some participants completed the MMPI-2-RF-EX. The MMPI-2-RF-EX is a 433-item enhanced version of the MMPI-2-RF that was used to develop and validate the MMPI-3. Research literature has shown that scale scores from both measures are psychometrically interchangeable, indicating that MMPI-3 scale scores obtained from an administration of the MMPI-2-RF-EX can be applied when using the MMPI-3 (Hall et al., 2022). Therefore, all MMPI-2-RF-EX administrations were re-scored as MMPI-3 protocols. The MMPI-3 is the newest version of the MMPI personality assessment instruments. The instrument is a self-assessment composed of 335 true-false items that measures clinically relevant variables with updated items, scales, and norms. It consists of 52 scales, including 10 validity scales. Invalid MMPI-3 protocols as outlined in the MMPI-3 Technical Manual were removed from the study (Ben-Porath & Tellegen, 2020): Cannot Say (CNS) ≥ 15 Combined Response Inconsistency (CRIN) ≥ 80 T, Variable Response Inconsistency (VRIN) ≥ 80 T, True Response Inconsistency (TRIN) ≥ 80 T, Infrequent Responses (F) ≥ 100 T, or Infrequent Psychopathology Responses (Fp) ≥ 100 T.
Procedures
Study 1
Participants in this study completed a battery of measures in person prior to the COVID-19 pandemic. Measures completed included the MMPI-2-RF-EX and the PID-5-SF. Each session lasted approximately 2 hours. The majority of participants in the present study who identified as White/non-Latinx participated in Study 1 (97%) and about 33% of the Latinx participants used in the present data analysis participated in Study 1. Participants were granted course credit upon completion of the study.
Study 2
The purpose of Study 2 was to explore the psychometrics of the U.S. Spanish MMPI-3. Approximately 67% of the Latinx participants used for data analysis in the present study participated in Study 2. This study included two time points that were 1 week apart, across which participants were randomly assigned to take either the English or Spanish version of the MMPI-2-RF-EX/MMPI-3. Only Latinx participants who took the English MMPI-2-RF-EX/MMPI-3 at either time point were included in the current study. All participants took the PID-5-SF during Time Point 1. However, given the randomized design of the study, some participants did not take the English MMPI-2-RF-EX/MMPI-3 until Time Point 2. As such, 41 participants took the MMPI-3 1 week after completing the PID-5-SF. Participants included in this study completed measures in person prior to the COVID-19 pandemic. Each appointment lasted approximately 2 hours. Individuals enrolled in eligible classes were granted course credit. Those recruited outside of the subject pool systems were compensated monetarily ($10). In order to increase the rate of participation, the monetary compensation was increased several months into the data collection ($20).
Results
Preliminary Analyses
Means and standard deviations for the MMPI-3 scales and the five PID-5-SF domains are reported by ethnicity and total sample in Supplemental Tables 1 and 2, respectively.
Correlation Analyses
We first conducted bivariate correlation analyses to address the first hypothesis, which predicted at least moderate relationships between theoretically overlapping scales of the PID-5-SF and the MMPI-3. Table 2 details the results of these analyses in the White sample, Latinx sample, and total sample, respectively.
Relationships of Theoretically Associated Scales.
Note. EID = Emotional/Internalizing Dysfunction; BXD = Behavioral/Externalizing Dysfunction; THD = Thought Dysfunction; RCd = Demoralization; RC1 = Somatic Complaints; RC2 = Low Positive Emotions; RC4 = Antisocial Behaviors; RC6 = Ideas of Persecution; RC7 = Dysfunctional Negative Emotions; RC8 = Aberrant Experiences; RC9 = Hypomanic Activation; MLS = Malaise; NUC = Neurological Complaints; EAT = Eating Concerns; COG = Cognitive Complaints; SUI = Suicidal/Death Ideation; HLP = Helplessness/Hopelessness; SFD = Self-Doubt; NFC = Inefficacy; STR = Stress; WRY = Worry; CMP = Compulsivity; ARX = Anxiety-Related Experiences; ANP = Anger Proneness; BRF = Behavior-Restricting Fears; FML = Family Problems; JCP = Juvenile Conduct Problems; SUB = Substance Abuse; IMP = Impulsivity; ACT = Activation; AGG = Aggression; CYN = Cynicism; SFI = Self-Importance; DOM = Dominance; DSF = Disaffiliativeness; SAV = Social Avoidance; SHY = Shyness; AGGR = Aggressiveness; PSYC = Psychoticism; DISC = Disconstraint; NEGE = Negative Emotionality/Neuroticism; INTR = Introversion/Low Positive Emotionality. Associations are listed as White Sample/Latinx Sample/Total Sample. .10 = small effect size; .30 = medium effect size; .50 = large effect size.
In the total sample, PID-5-SF Negative Affectivity was at least moderately correlated with all hypothesized MMPI-3 scales (rs = .31 (Suicidal Ideation [SUI]) − .75 (Negative Emotionality/Neuroticism [NEGE])) except for a small effect size for Introversion ([INTR] r = .23). The White sample showed small effect sizes for Ideas of Persecution ([RC6] r = .28) and Anger Proneness ([ANP] r = .27) and the Latinx sample yielded a small correlation with SUI (r = .28) and with INTR (r = .20). For Detachment, all correlations in both the total sample and across each ethnic group were at least moderately correlated (total sample rs = −.32 (Self Importance [SFI]) − .64 (Emotional/Internalizing Dysfunction [EID])). Antagonism was also moderately correlated with most hypothesized MMPI-3 scales in the total sample (rs = .30 (Disconstraint [DISC]) − .45 (Cynicism [CYN])). However, several correlations yielded small effect sizes, including Antisocial Behavior (RC4), Juvenile Conduct Problems (JCP), and SFI across the White, Latinx, and total samples. Similarly, although Disinhibition was moderately correlated with hypothesized MMPI-3 scales in the total sample (rs = .32 [JCP] − .65 (Impulsivity [IMP])), several hypothesized scales demonstrated small correlations across the total, White, and Latinx samples. Most notably, the associations between Disinhibition and both Compulsivity [CMP] and Behavior Restricting Fears [BRF] were of small, positive effect, despite both being predicted to have negative relationships. Finally, correlations in both the total sample and across each ethnic group were at least moderately correlated with Psychoticism (rs = .47 [RC6] − .67 (Aberrant Experiences [RC8])).
Next, Steiger’s z-tests were conducted for all hypothesized associations to test for significant differences of correlation magnitude between the two ethnic groups. No significant differences were found, which indicates correlations were generally similar in magnitude across the White and Latinx ethnic groups. Results of the Steiger’s z-tests are shown in Supplemental Table 3.
Slope/Intercept Bias Analyses
To address the second hypothesis, we used hierarchical regression to test whether the relationships between MMPI-3 scales and associated personality psychopathology constructs (as measured by the PID-5-SF) remained consistent across ethnic groups. The presence of slope and/or intercept bias was not expected due to the use of newly constructed, culturally sensitive norms. The results largely supported this hypothesis, as detailed in Table 3. Only one analysis indicated statistically significant bias using a Bonferroni correction (p < .01); intercept bias was found for PID-5-SF Negative Affectivity’s prediction of ANP scores (ΔR2 = .018, p = .007) such that ANP scores were over-predictive in the White sample. Likewise, this bias yielded a small effect size and likely does not hold clinical significance.
Regression Analyses of PID-5-SF Domains on MMPI-3 Scales.
Note. EID = Emotional/Internalizing Dysfunction; BXD = Behavioral/Externalizing Dysfunction; THD = Thought Dysfunction; RCd = Demoralization; RC1 = Somatic Complaints; RC2 = Low Positive Emotions; RC4 = Antisocial Behaviors; RC6 = Ideas of Persecution; RC7 = Dysfunctional Negative Emotions; RC8 = Aberrant Experiences; RC9 = Hypomanic Activation; MLS = Malaise; NUC = Neurological Complaints; EAT = Eating Concerns; COG = Cognitive Complaints; SUI = Suicidal/Death Ideation; HLP = Helplessness/Hopelessness; SFD = Self-Doubt; NFC = Inefficacy; STR = Stress; WRY = Worry; CMP = Compulsivity; ARX = Anxiety-Related Experiences; ANP = Anger Proneness; BRF = Behavior-Restricting Fears; FML = Family Problems; JCP = Juvenile Conduct Problems; SUB = Substance Abuse; IMP = Impulsivity; ACT = Activation; AGG = Aggression; CYN = Cynicism; SFI = Self-Importance; DOM = Dominance; DSF = Disaffiliativeness; SAV = Social Avoidance; SHY = Shyness; AGGR = Aggressiveness; PSYC = Psychoticism; DISC = Disconstraint; NEGE = Negative Emotionality/Neuroticism; INTR = Introversion/Low Positive Emotionality. Unstandardized regression coefficients are shown in the three columns identified as “b.” DV = PID-5-SF domain score; IV = MMPI-3 subscale score; IV × Ethnicity = interaction term. .010 = small effect size; .090 = medium effect size; .250 = large effect size.
Significant with the Bonferroni correction at p < .01.
Discussion
With the increasing diversity in the United States, specifically regarding the exponential growth of the Latinx population, it has become essential to validate the use of the MMPI-3 and its new norms in both White and Latinx ethnic groups to ensure equity in psychological assessment. In other words, this study ventured to contribute to an emerging body of literature on the MMPI-3 and will lend to a better understanding of how to best interpret the measure effectively across cultures. Given the established relationships between MMPI-2-RF/MMPI-3 scales and personality psychopathology constructs (e.g., Anderson et al., 2015; Anderson & Sellbom, 2021; Brown & Sellbom, 2021), testing differences in these relationships across ethnicity provides a strong start to establishing cross-cultural validity.
The first hypothesis used correlation analyses to test the relationships between theoretically overlapping MMPI-3 scales and PID-5-SF domains; this in turn would serve as a method to assess the construct validity of the MMPI-3. Most hypothesized associations were of at least a medium effect size across ethnic groups. Notably, the majority of small to negligible correlations were between hypothesized MMPI-3 scales and the Antagonism and Disinhibition domains of the PID-5-SF. However, the results of the analyses overall indicated good convergent validity for the MMPI-3 subscales in assessing personality psychopathology. Though not the primary purpose of the current study, this adds to previous work on the MMPI-2-RF (e.g., Anderson et al., 2013, 2015, 2018; Finn et al., 2014; Sellbom et al., 2013) and emerging work with the MMPI-3 (e.g., Brown & Sellbom, 2021), demonstrating the utility of the MMPI instruments in assessing personality disorder traits.
The second hypothesis predicted the absence of ethnic bias in the MMPI-3’s measurement of dimensional psychopathology between White and Latinx groups. Given research on previous iterations of the MMPI showing little to no racial/ethnic bias (e.g., Marek et al., 2015; Whitman et al., 2019), bias was similarly not expected on the MMPI-3. This hypothesis was also primarily supported. First, there were no differences in correlation magnitude across the two ethnic groups. Further, there was no evidence of slope bias and only one analysis indicated intercept bias. This bias was found for PID-5-SF Negative Affectivity’s prediction of Anger Proneness (ANP) scores, which indicated that White individuals scored higher than Latinx individuals on anger proneness regardless of their score on the PID-5-SF Negative Affectivity domain. In other words, if anything, this scale may overpredict anger among White individuals, though replication would be needed. Notably, this bias showed a small effect size and may not hold strong clinical significance.
This research answers the call of the APA to validate psychological tests in the target population to ensure culturally competent interpretations (APA, 2002). Indeed, these results support the use of the MMPI-3 as an unbiased instrument that can assess personality cross-culturally, specifically in the Latinx population. This is particularly true in consideration of the instrument’s foundation on a new norming sample that closely resembles the national demographics, consisting of 14% Hispanic individuals. Even though the history of MMPI research has indicated a lack of bias across several ethnic groups (Marek et al., 2015; Whitman et al., 2019), these updated norms have undoubtedly been a key step to ensuring continued equitable assessment in a rapidly diversifying country. Moreover, the empirical support of the instrument and its norms bolsters limited previous literature that supports the ethical use of the MMPI instruments in multiple settings and across multiple populations (Sellbom, 2019).
Although the support for both hypotheses is promising for the future contribution of the MMPI-3 in the field of cross-cultural personality assessment, there remain limitations that should be addressed. First, all participants were college students, and one of the samples collected are members of a specific population—bilingual college students in south and southeast Texas. As a result, scores across both instruments may not be representative of the national population. For example, average MMPI-3 scores related to RC4 were below 50T (the normative average), whereas scores related to Stress (STR) and Worry (WRY) were above 50T. These results mirror prior research using MMPI instruments in a college sample (e.g., Kremyar & Wygant, 2023), suggesting our results are consistent with other college samples, but may be limited in broader generalizability. Additionally, the total sample is uneven in its composition of Latinx and White individuals, as the number of Latinx participants was more than double the number of White participants. Future research should thus be conducted in a sample with a more even ethnic distribution as well as one that is composed of individuals outside of a college student participant pool to ensure results are replicable in a population that may be more generalizable. This also extends to research in a sample that is more generalizable by gender, as the majority of our sample identified as women (n = 238).
It is also notable that the term Latinx as a group classifier is heterogeneous and spans multiple ethnic and cultural backgrounds. Indeed, our participants identified six different countries of origin. Moreover, our demographics data indicated that many participants in our sample endorsed high levels of acculturation to American culture. This means our sample is largely generalizable to similar groups of Latinx individuals in the United States. Groups outside of the United States or those less acculturated (e.g., recent immigrants) may yield differing findings. In addition, although our sample is proficient in the English language and primarily acculturated, the heterogeneity of the Latinx population and Spanish language may have effects on how individuals score across measures. Future research should include or assess variations across subgroups within the Latinx culture. It also remains possible that the intersection of different demographic variables in addition to ethnicity (e.g., gender, socioeconomic status) could affect response styles. While this intersectionality would be a valuable point of focus in future research, the demographic similarities in our sample (e.g., young age, mostly female, mostly high school graduates) allowed for an isolation of ethnicity as the primary variable implicated in any group differences, which is concordant with the objectives of the present study.
Finally, the size of our sample would not have yielded adequate power to conduct analyses of measurement invariance. These analyses would be worth further study. Relatedly, our study primarily focused on self-report scales and measures most relevant to the assessment of personality psychopathology. Although our results provide initial support for the MMPI-3 as an unbiased instrument for assessment of personality psychopathology in the Latinx population, our results alone cannot conclusively determine an absolute lack of ethnic bias for the instrument. Thus, future research would benefit from both using measures that are not self-report and assessing test bias in other areas of psychopathology, particularly areas reflected in newly constructed scales for the MMPI-3 (e.g., Eating Concerns [EAT] or CMP). The use of the PID-5-SF is also a limitation, as limited research beyond its original development (Maples et al., 2015) has evaluated its factor structure. Also of note, the literature supporting the cross-cultural validity of the PID-5 and the PID-5-SF is fairly scarce and work most relevant to the current study was conducted using a sample from Spain (Aluja et al., 2019; García et al., 2022; Gutiérrez et al., 2017). Therefore, the PID-5-SF is not validated in the specific, heterogeneous population used in this study. Future research on this measure would be of benefit. A continued point of exploration will therefore be using other measures to further support the validity of the MMPI-3—both in the realm of personality psychopathology and outside of it. Likewise, it is important to note that our use of etic measures of personality functioning, rather than emic frameworks, could limit the cultural validity of the findings. As such, it is important to place continued efforts into validating assessment instruments in diverse populations.
Even with the aforementioned limitations considered, these results yield promising support for validity of the newly released MMPI-3. Its scales continue to demonstrate strong internal validity. Moreover, the new developments of the instrument, specifically the new norms reflecting the 2020 United State Census demographics, allows for unbiased interpretations of personality constructs across White and Latinx ethnic groups. These results add to a substantial dearth of literature on cross-cultural personality assessment and support efforts to validate the MMPI-3 in the target Latinx population to ensure culturally competent and ethical psychological practice.
Supplemental Material
sj-docx-1-asm-10.1177_10731911241254341 – Supplemental material for Exploring Potential Ethnic Bias Among MMPI-3 Scales in Assessing Personality Psychopathology
Supplemental material, sj-docx-1-asm-10.1177_10731911241254341 for Exploring Potential Ethnic Bias Among MMPI-3 Scales in Assessing Personality Psychopathology by Nicole Shumaker, Tessa Long, Andy Torres, Alfonso Mercado, Ryan J. Marek and Jaime L. Anderson in Assessment
Footnotes
Author Contributions
Nicole Shumaker: conceptualization, data collection and analysis, project administration, manuscript preparation; Tessa Long: data curation, project administration, funding acquisition, manuscript reviewing; Andy Torres: data curation, project administration, manuscript reviewing; Alfonso Mercado: methodology, resources, project supervision, manuscript reviewing; Ryan J. Marek: conceptualization, methodology, manuscript reviewing; Jaime L. Anderson: conceptualization, methodology, funding acquisition, supervision, resources, manuscript preparation.
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Jaime L. Anderson and Ryan J. Marek receive active funding (unrelated to this project) from the University of Minnesota Press, publisher of the MMPI instruments.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was partially supported by grant funding from the Society for Personality Assessment. MMPI-3 materials were provided by the University of Minnesota Press.
Supplemental Material
Supplemental material for this article is available online.
Data Availability
Data used in this study are available upon request from the corresponding author.
Notes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
