Beyond Description in Interpersonal Construct Validation

Abstract

The interpersonal circumplex is a well-established structural model that organizes interpersonal functioning within the two-dimensional space marked by dominance and affiliation. The structural summary method (SSM) was developed to evaluate the interpersonal nature of other constructs and measures outside the interpersonal circumplex. To date, this method has been primarily descriptive, providing no way to draw inferences when comparing SSM parameters across constructs or groups. We describe a newly developed resampling-based method for deriving confidence intervals, which allows for SSM parameter comparisons. In a series of five studies, we evaluated the accuracy of the approach across a wide range of possible sample sizes and parameter values, and demonstrated its utility for posing theoretical questions on the interpersonal nature of relevant constructs (e.g., personality disorders) using real-world data. As a result, the SSM is strengthened for its intended purpose of construct evaluation and theory building.

Keywords

construct validation interpersonal circumplex structural summary method resampling personality disorders

The interpersonal circumplex (IPC) organizes interpersonal functioning within the two-dimensional space marked by dominance and affiliation (Pincus & Ansell, 2013). The IPC’s structure is the foundation for one of the major paradigms of personality (Wiggins, 2003), interpersonal theory, and also provides a fundamental link for bridging traditionally disparate intellectual traditions such as trait (McCrae & Costa, 1989), attachment (Bartholomew & Horowitz, 1991), and evolutionary (Zuroff, Fournier, Patall, & Leybman, 2010) theories. The IPC is widely used not only in basic personality science but also in clinical science and practice (Pincus & Wright, 2011), thereby furthering much needed cross-disciplinary unification in psychology. The IPC’s highly integrative nature arises from its ability to serve as a nomological network for comprehensively organizing interpersonal variables and evaluating novel constructs and measures (Cronbach & Meehl, 1955; Gurtman, 1992; Guttman, 1970).

Ideally, quantitative summary statistics for evaluating an external variable’s patterns of association within existing nomological networks can be applied when engaging in construct and measure validation (e.g., Westen & Rosenthal, 2003). This ensures that theoretical advances are empirically defensible and protects against impressionistic or selective interpretation of the patterns of associations in the data. The structural summary method (SSM) for circumplex data (Gurtman, 1992; Gurtman & Pincus, 2003) has been developed for exactly this purpose. It builds on the circumplex structure of IPC measures, and parsimoniously summarizes profiles of correlations with other constructs outside the IPC. However, to date the method has been descriptive, providing no way to draw inferences when comparing SSM parameters across constructs or groups. Here, we describe a newly developed resampling-based method for deriving confidence intervals, which allows for SSM parameter comparisons. As a result, the SSM is strengthened for its intended purpose of construct evaluation and theory building.

The Interpersonal Circumplex

The interpersonal circle was the conceptual model initially developed by Timothy Leary and colleagues based on the close observation of interpersonal interactions in group therapy (Leary, 1957). Their theoretical model organized interpersonal behavior in a circular array around the primary axes of dominance versus submissiveness and affiliation versus separation.¹ The model was eventually formalized into a circumplex as measures were developed to precisely assess its content. A circumplex was initially defined via a circulant pattern of correlations (Guttman, 1954), such that correlations going away from the diagonal initially decrease; only to increase again beyond a certain point (see Table 1). What this implies is that in a circular array of variables, the association between any two variables is an inverse function of their angular distance from each other. That is to say, as angular distance between two variables increases, their association decreases, up to 180°. Thus, variables that share close conceptual content are also located closely on the circumplex.

Table 1.

Circulant Correlation Matrix for an Interpersonal Measure With Eight Octant Scales.

	V1	V2	V3	V4	V5	V6	V7	V8
V1	1.0
V2	ρ₁	1.0
V3	ρ₂	ρ₁	1.0
V4	ρ₃	ρ₂	ρ₁	1.0
V5	ρ₄	ρ₃	ρ₂	ρ₁	1.0
V6	ρ₃	ρ₄	ρ₃	ρ₂	ρ₁	1.0
V7	ρ₂	ρ₃	ρ₄	ρ₃	ρ₂	ρ₁	1.0
V8	ρ₁	ρ₂	ρ₃	ρ₄	ρ₃	ρ₂	ρ₁	1.0

Note. V1–V8 = hypothetical variables 1 through 8. ρ₁ > ρ₂ > ρ₃ > ρ₄.

Modern IPC measures generally use a set of eight scales (i.e., octants) that divide the circumference and content of the circle in to equally spaced 45° partitions. This represents a desirable trade-off between scale reliability and fidelity. Scales have been developed for a number of levels of interpersonal functioning including: traits (Markey & Markey, 2009; Wiggins, 1995), problems (Alden, Wiggins, & Pincus, 1990), values (Locke, 2000), efficacy (Locke & Sadler, 2007), sensitivities (Hopwood et al., 2011), strengths (Hatcher & Rogers, 2009), and covert reactions (Kiesler, Schmitt, & Wagner, 1997), among others. Figure 1 (Panel A) provides an example of the IPC as represented by the Inventory of Interpersonal Problems–Circumplex scales (IIP-C; Alden et al., 1990). As can be seen in the figure, scales that are adjacent share close conceptual content (e.g., Domineering problems and Intrusive Problems), those at right angles are conceptually unrelated (e.g., Domineering problems and Overly Nurturant problems), and those across the circle from each other are conceptually opposite (e.g., Domineering problems and Nonassertive problems). The expected correlations might be strongly positive, near to zero, and strongly negative, respectively. However, in practice, many of the circumplex measures listed here have a sizeable general factor that accounts for a positive manifold among all scales. In the IIP-C this has been labeled generalized interpersonal distress (Tracey, Rounds, & Gurtman, 1996). Despite these general factors, the basic circulant pattern can still hold, such that the largest correlations are among adjacent scales and the lowest correlations are among opposite scales.

A number of quantitative procedures have been developed for establishing whether a set of scales conforms to the circumplex pattern (see, e.g., Acton & Revelle, 2002, 2004; Gurtman, 2009; Gurtman & Pincus, 2003; Tracey, 2000; Wiggins, Steiger, & Gaelick, 1981). These range in quantitative rigor from visual analyses of plots from principal component analysis and multidimensional scaling (Wiggins et al., 1981), to nonparametric tests of scale ordering (Tracey, 1997, 2000), to confirmatory structural equation modeling approaches based on Fourier decomposition of the circulant matrix (Browne, 1992; Fabrigar, Visser, & Browne, 1997; Nagy, Marsh, Lüdtke, & Trautwein, 2009). These methods for evaluating an inventory’s “circumplexity” are important as this structure serves as both the theoretical and statistical underpinnings of the IPC model (cf. Guttman, 1970). We note most modern IPC measures have fared well when subjected to circumplex analyses, and minor deviations from perfect structure have little practical impact (Gurtman & Pincus, 2000).

Interpersonal Construct Validation and the Structural Summary Method

The SSM for circumplex data (Gurtman, 1992; Gurtman & Pincus, 2003; Wright, Pincus, Conroy, & Hilsenroth, 2009), is a suite of approaches developed for the analysis of IPC profiles, which may consist of individual octant scores (e.g., Gurtman & Balakrishnan, 1998; Pincus & Gurtman, 2003), group scores (e.g., Cain et al., 2012; Przeworski et al., 2011), or correlations with external measures (Gurtman, 1992). The latter methodology builds on earlier approaches to interpersonal construct validation such as the “vector method” (e.g., Gurtman, 1991; Wiggins & Broughton, 1991), by taking full advantage of the structure of the circumplex. The logic of the SSM is simple: Just as associations among variables within a circumplex measure are expected to conform to a certain pattern, to the extent that an external variable has specific interpersonal content, so too should it conform to that pattern (Gurtman, 1992). To illustrate, if a theoretically interpersonal variable (e.g., narcissistic grandiosity; Wright et al., 2013) were found to have a strong positive correlation with dominant interpersonal problems (e.g., IIP-C Domineering octant), it can be predicted that the next strongest correlations would be with adjacent octants (i.e., Vindictive and Intrusive octants), followed by decreasing correlations moving further away from the peak associated octant. Specifically, the expected pattern of correlations should follow a sinusoidal wave (Benjamin, 1974; Gurtman, 1992; Wiggins et al., 1981). An example of a perfect pattern of associations can be found in Figure 1 (Panel B). This predicted pattern of associations can be formally represented by the following equation for a cosine function (Gurtman, 1992):

r_{i}^{p} = e + a (\cos (θ_{i} - δ))

where $r_{i}^{p}$ is a construct’s predicted correlation with octant i, given that e is the elevation of the curve (i.e., the average correlation across all octants), a is the amplitude of the curve (i.e., the distance between the average correlation and the peak correlation), θ_i is the angle of octant i, and δ is the angular displacement of the peak of the curve from 0°. In a perfect curve (i.e., a pattern of observed correlations that perfectly conforms to a circumplex pattern) knowing the parameters of e, a, and δ are all that are required to summarize the structure of the entire profile of associations. Figure 1 (Panel B) graphically depicts the SSM parameters for an example curve with e = .1, a = .3, and δ = 90°.

Each of these parameters has a conceptual meaning useful for interpreting a profile of correlations with IPC scales (Gurtman, 1992). Elevation, or e, is calculated as the average correlation, and can be understood as an estimate of the association between a variable and the general factor of an IPC measure. As noted above, some IPC scales have a sizeable and interpretable substantive general factor. Amplitude, or a, captures the degree of differentiation in a profile of correlations, and represents the degree of interpersonal specificity in a construct.² Angular displacement, or δ, locates the peak of a curve and represents the predominant interpersonal theme or content of a profile. It should be apparent that the SSM is capable of conveying a large amount of information about a construct’s location in the interpersonal nomological net very efficiently and elegantly. However, there is one important caveat to keep in mind when interpreting these parameters—namely, that they are dependent on the observed profile’s fit to an ideal sinusoidal wave. Goodness of fit to the expected pattern, or R², is easily calculated with the following formula:

R^{2} = \frac{\sum_{i = 1}^{8} {(r_{i}^{p} - \bar{r^{p}})}^{2}}{\sum_{i = 1}^{8} {(r_{i}^{o} - \bar{r^{o}})}^{2}} = \frac{S S^{p}}{S S^{o}}

where SS is the sum of squared deviations of the (predicted or observed as denoted by superscripts) correlational profile. This statistic has been interpreted in terms of prototypicality, with values of R² > .80 representing good fit to a cosine curve and values of R² > .70 representing adequate fit to a cosine curve (Wright et al., 2009). Only in prototypical profiles are the angular displacement and amplitude parameters fully interpretable. In contrast, elevation does not hinge on prototypicality for interpretability because it reflects the mean of a profile, and is not contingent on a specific pattern of correlations. Further formulas that are useful to compute SSM parameter in practice can be found in the appendix.

The relevance of the SSM for interpersonal construct validation becomes especially apparent when it is compared with the widespread strategy to focus on single octant correlations (e.g., Desmet et al., 2007; Haggerty, Blake, & Siefert, 2010; McEvoy, Burgess, Page, Nathan, & Fursland, 2013; Miller, Price, Gentile, Lynam, & Campbell, 2012; Rodebaugh, Gianoli, Turkheimer, & Oltmanns, 2010; Sato & McCann, 2007). For example, Haggerty et al. (2010) examined the validity of the Relationship Profile Test (Bornstein et al., 2003), which is a self-report measuring destructive overdependence, dysfunctional detachment, and healthy dependence. They administered the Relationship Profile Test in conjunction with the short form of the IIP-C (Soldz, Budman, Demby, & Merry, 1995) in a sample of undergraduates, and correlated its scales with IIP-C octant scores. The Healthy Dependence scale was significantly negatively correlated with all octants except NO (rs ranging from −.37 to −.04), which led authors to conclude that healthy dependent persons show low interpersonal distress. When applying the SSM to their results, this conclusion is corroborated by a negative elevation of e = −.25. However, the SSM also shows that the correlational profile has a good fit to a cosine curve (R² = .87), is considerably differentiated (a = .15), and peaks at 76.7°. In fact, although focusing on the single octant correlations would suggest that healthy dependence is unrelated to NO, analyzing the full pattern of correlations revealed that NO is at the heart of the specific interpersonal theme of healthy dependency—interacting with others in an assured or friendly dominant way. Other examples of research that do not take advantage of the SSM include the practice of ipsatizing IPC octant scores before correlating them with the target measure, which discards any information on elevation. The point we are making here is that researchers will miss important information, or might even come to erroneous conclusions, when they do not make use of the SSM in interpersonal construct validation.

Fortunately, the correlation-based SSM is increasingly applied in evaluating the interpersonal nature of external constructs and measures (e.g., Brown & Sherman, 2014; Dinger et al., 2015; Erickson et al., 2016; Gurtman, 1992, 1993, 1999; Gurtman & Lee, 2009; Hill, Zrull, & Turlington, 1997; Hopwood et al., 2011; Hopwood, Burt, et al., 2013; Markey, Anderson, & Markey, 2013; Turan, Guo, Boggiano, & Bedgood, 2014; Williams & Simms, 2016; Wright et al., 2012; Zimmermann, Wolf, Bock, Peham, & Benecke, 2013). Combined, the 15 studies cited above evaluated 433 constructs using nine different IPC measures (whereby the IIP-C was the most prominent measure and applied in more than half of the cases). As clear guidelines for the interpretation of the sizes of elevation and amplitude are currently lacking, we reviewed the empirical distribution of elevation and amplitude across these 433 constructs. Elevation ranged from 0 to |.46|, with the 25th, 50th, and 75th percentiles being |.02|, |.11|, and |.27|, respectively. Amplitude ranged from .01 to .43, with the 25th, 50th, and 75th percentiles being .10, .16, and .23. These distributional properties roughly corroborate the heuristic cutoffs of e = |.15| and a = .15 that have been proposed by Wright et al. (2012) for what constitutes a markedly elevated and differentiated profile. However, they give researchers an even more detailed and realistic rubric with which to judge their SSM results in future studies.

Moving Beyond Description in Interpersonal Construct Validation

Although the SSM is increasingly applied in psychological research, its parameters remain poorly understood from the perspective of inferential statistics. In particular, it has not been shown thus far that point estimates of SSM parameters in empirical samples are accurate in the sense that they tend to converge with the true value in the population. It is not necessarily the case that they are accurate. For example, consider the case of amplitude, which can have values in the range of 0 to 1. When amplitude is zero in the population (i.e., when the true value of amplitude lies on the boundary of the allowable parameter region), the mean of the sampling distribution will presumably be greater than zero. That is, the majority of samples drawn from this population will presumably show nonnegligible amplitude (especially when sample size is small), suggesting that this estimator is biased. This is also noticeable from the distribution of previously published point estimates of amplitude (see above), which were greater or at least equal to .05 in 91.4 % of the cases. Thus, Monte Carlo simulation studies are needed to explore the properties of SSM parameters in more detail.

Recently, Zimmermann et al. (2013) pointed to a somewhat related limitation of the SSM: Although it parsimoniously summarizes a complex profile of correlations, it is primarily descriptive. That is, it does not answer inferential questions such as “Is elevation of construct X significantly greater than zero?” “Is amplitude of X significantly greater than amplitude of Y?” or “Does angular displacement of X significantly differ between group a and group b?” This is because there is currently no way to test SSM parameters for significance, or to compute their associated standard errors (SEs) or confidence intervals (CIs). In fact, the sampling distribution of SSM parameters is currently unknown and may depend on factors other than sample size. For example, it seems reasonable to hypothesize that amplitude is negatively related to the SE/CI of angular displacement, because measures with a more differentiated sinusoidal profile (i.e., high amplitude) should have a more “stable” interpersonal theme (i.e., their angular displacement parameter should be less affected by sampling error as compared with measures with low amplitude). This poses significant challenges for deriving formulas for SE/CI of SSM parameters in an analytical way.

A reasonable alternative to overcome this limitation is to make use of the bootstrap method (Efron & Tibshirani, 1993). The bootstrap method is a resampling-based method to empirically compute SEs and CIs, and thus is especially helpful for statistics with analytically unknown sampling distributions (such as SSM parameters). Basically, the (nonparametric) bootstrap requires generating thousands of random resamples from the original sample, with resamples being of equal size as the original sample, and individuals being sampled with replacement. In each resample, the statistic of interest is computed, and the resulting distribution across resamples is used to estimate the SE or CI of the statistic. Monte Carlo simulation studies have shown that the bootstrap method yields accurate CIs for statistics such as the area under a receiver operating characteristic curve (Ruscio & Mullen, 2012), factor loadings in exploratory factor analysis (Zhang, Preacher, & Luo, 2010), correlation coefficients corrected for range restriction (Mendoza, Hart, & Powell, 1991), or the indirect effect in statistical mediation analysis (Biesanz, Falk, & Savalei, 2010; MacKinnon, Lockwood, & Williams, 2004).

Thus, the bootstrap method offers a promising way forward for moving the SSM beyond a purely descriptive approach in interpersonal construct validation, to providing a framework for posing inferential questions to the interpersonal nomological net. For example, as outlined above, questions of differences between parameters across distinct constructs are often of substantive interest. Differences among profile parameters of the same construct’s profile across groups might be theoretically important (e.g., males vs. females, patients vs. nonpatients). And finally, being able to test whether a given profile reliably differs from a theoretically expected pattern would provide more rigorous evaluation of a construct’s location within the interpersonal nomological net.

The Current Studies

The aim of this article is to introduce, evaluate, and demonstrate a resampling-based method for the calculation of CIs of SSM parameters. Thereby, we hope to fill a major gap in the current repertoire of IPC methods, allowing for the first time inferential conclusions about (differences between) parameters within an interpersonal construct validation framework. We start with a Monte Carlo simulation study to explore whether the currently used SSM point estimates are biased, and which factors influence accuracy and precision of SSM point estimates. Next, we introduce a method for calculating 95% bootstrap CIs of (differences between) SSM parameters and evaluate their accuracy (i.e., empirical coverage) in three Monte Carlo simulation studies. This method has been implemented in the R package “ssm” that is available for free download on the second author’s website (http://www.personalityprocesses.com/ssm/). Finally, in a fifth study, we applied the method to real data demonstrating the utility of the technique we have developed. Specifically, we used CIs when comparing the interpersonal features of personality disorder (PD) scales using the IIP-C.

Study 1

The aim of Study 1 was to assess (influences on) accuracy and precision of SSM point estimates. Therefore, we conducted a Monte Carlo simulation study. In each simulation, the population consisted of 100,000 cases with scores on eight octant scales and one target measure. The distributions of octant scales and the target measure were assumed to be multivariate normal, and their associations were specified in such a way that (a) the octants’ intercorrelations conformed to a circulant matrix (see Table 1) and (b) the target measure’s profile of correlations with the octants had a known elevation, amplitude, angular displacement, and goodness of fit. Next, we drew 5,000 samples from this population, each time computing sample-specific SSM statistics (including statistics for dominance and affiliation; see the appendix). The resulting sampling distribution can be used to assess the accuracy (vs. bias) and precision (vs. error) of an SSM parameter: Its bias is defined as the difference between the known value in the population and the mean of the sampling distribution, with smaller absolute bias indicating more accurate estimates, and its standard error is defined as the standard deviation of the sampling distribution, with smaller standard errors indicating more precise estimates.

To assess factors that might influence accuracy and precision of SSM parameters, we conducted a series of simulations with systematically varying (a) sample size, (b) the correlation matrix of the eight octants, and (c) parameters of the target measure. The specific values of these factors were selected to cover the full range of conditions that interpersonal researchers commonly find in their data when using IPC instruments. Specifically, we considered seven different sample sizes (n = 25, 50, 75, 100, 150, 200, 500), two different IPC correlation matrices including an equally spaced circumplex with and without a substantial general factor,³ and 80 target measures differing in elevation (E = .0, .2), affiliation (AFF = .0, .1, .2), dominance (DOM = .0, .1, .2), and goodness of fit (R² = .6, .7, .8, .9, 1.0).⁴ Thus, Study 1 was based on 1,120 simulations. All simulations were conducted with R 2.12.2 (R Development Core Team, 2011) on the Linux computer cluster at the University of Kassel. To test the joint effects of sample size, general factor of the IPC measure, and parameters of the target measure on the accuracy and precision of SSM parameters, we conducted analyses of variances (ANOVAs). Therefore, we included the six factors and their two-way interactions as predictors, and used the amount of bias and the standard error as the outcome, respectively. Effects explaining less than 5% of total variance (i.e., η < .05) were not considered as relevant.

Study 1 Results

Bias in elevation, dominance, and affiliation was generally very small across all 1,120 conditions. The average bias was −.0007, respectively, and the most extreme bias in an individual condition was −.013. Given that the population values of these parameters differed up to .2 across conditions, we considered this amount of bias as unproblematic. However, bias in amplitude was substantial. Across all conditions, bias was consistently positive, ranged from .0002 to .136 and averaged at .021. When standardizing these values by the true population parameters of amplitude (i.e., computing relative bias), this translates into an average bias of 15.5%, ranging from 0 to 135.8%.⁵ This shows that sample estimates of amplitude tend to overestimate the true amplitude in the population under most of the simulated conditions.

Results of the ANOVA suggested that much of the variation in (relative) bias across conditions could be explained by differences in sample size (η² = .34), the general factor of the IPC measure (η² = .08), dominance (η² = .13), and affiliation (η² = .08) of the target measure, as well as all two-way interactions of these factors (η² = .29). Influences of the remaining factors (i.e., elevation and goodness of fit of the target measure) were virtually zero (η² < .001). Figure 2 illustrates the specific nature of these influences: Relative bias decreases with increasing sample size, presence of a general factor in the IPC measure, and increasing amplitude (in the sense of joint effects of dominance and affiliation) of the target measure in the population. Thus, when using an IPC measure without a general factor, relative bias in amplitude will exceed the commonly accepted margin of 5% even with sample sizes as large as n = 500 as long as amplitude in the population is rather small (A ≤ .1). Conversely, when using an IPC measure with a general factor, relative bias in amplitude will be already acceptable with samples n ≥ 150 given that amplitude in the population is at least moderate (A ≥ .14).

To further illustrate the biased nature of sample estimates of amplitude, we considered the most extreme case of A = 0, which was not included in the 1,120 simulated conditions. Figure 3 presents the respective sampling distributions of amplitude under selected conditions of sample size and presence of a general factor in the IPC measure. It becomes apparent that with decreasing sample size the probability of receiving large sample estimates of amplitude (despite amplitude being zero in the population) steadily increases. For example, with n = 50 and no general factor in the IPC measure, the expected amplitude in samples (i.e., mean of the sampling distribution) is .153. This demonstrates that a finding of a = .15 is not necessarily “marked” or “substantial,” but can, under certain conditions, simply represent the expected value when amplitude in the population is zero.

Next, we examined bias in angular displacement and goodness of fit. Average bias in angular displacement was −0.03°, ranging from −2.38° to 3.03°. Given that the population values of angular displacement differed up to 90 degrees across conditions, we considered this amount of bias as unproblematic. Average bias in goodness of fit was −.081, ranging from −.382 to .027. This amount of bias is clearly substantial, corresponding to an average relative bias of −9.5% and a range of −38.2% to 4.5%. Thus, sample estimates of goodness of fit tend to underestimate true goodness of fit in the population (at least in the range of the simulated conditions). Results of the ANOVA suggested that the most important factors influencing (relative) bias are sample size (η² = .33), dominance (η² = .11), affiliation (η² = .07), and goodness of fit (η² = .22) of the target measure, as well as all two-way interactions of these factors (η² = .19). In particular, the amount of bias decreases with increasing sample size, increasing amplitude, and lower goodness of fit of the target measure in the population. This is understandable, as R² = 1 is a boundary value and samples drawn from such a population are likely to show lower values.

The second major question of this study was to explore influences on the precision versus error of SSM parameter estimates. Table 2 presents the results of respective ANOVAs. The standard error of elevation, dominance, affiliation, and amplitude was exclusively a function of sample size and presence of a general factor in the IPC measure. Expectably, standard errors generally decreased with increasing sample size. In addition, the standard error of elevation further decreased when a general factor in the IPC measure was absent, whereas the standard error of dominance, affiliation, and amplitude further decreased when a general factor in the IPC measure was present.

Table 2.

Effects of Various Conditions on the Precision (vs. Error) of Structural Summary Method Parameters.

	e	aff	dom	a	δ	R ²
General factor	.51	.18	.18	.18	.11	.05
Sample size	.38	.79	.78	.79	.49	.51
Elevation	0	0	0	0	0	0
Affiliation	0	0	0	0	.11	.10
Dominance	0	0	0	0	.18	.16
Goodness of fit	0	0	0	0	0	.07
General factor × Sample size	.10	.03	.04	.01	.01	0
General factor × Elevation	0	0	0	0	0	0
General factor × Affiliation	0	0	0	0	0	0
General factor × Dominance	0	0	0	0	0	0
General factor × Goodness of fit	0	0	0	0	0	0
Sample size × Elevation	0	0	0	0	0	0
Sample size x Affiliation	0	0	0	0	.01	.01
Sample size × Dominance	0	0	0	0	.01	.01
Sample size × Goodness of fit	0	0	0	0	0	.01
Elevation × Affiliation	0	0	0	0	0	0
Elevation × Dominance	0	0	0	0	0	0
Elevation × Goodness of fit	0	0	0	0	0	0
Affiliation × Dominance	0	0	0	0	.07	.04
Affiliation × Goodness of fit	0	0	0	0	0	0
Dominance × Goodness of fit	0	0	0	0	0	0
Residuals	0	0	0	0	.01	.04

Note. N = 1,120 conditions. Entries in the table refer to η².

A more complex picture emerged for angular displacement and goodness of fit. In these cases, standard errors were additionally influenced by features of the target measure in the population, that is, by amplitude (in the case of angular displacement) and by amplitude and goodness of fit (in the case of goodness of fit). Figure 4 shows the specific nature of these influences for angular displacement. Standard error decreased with increasing sample size, presence of a general factor in the IPC measure, and increasing amplitude in the population. For example, when using an IPC measure without a general factor and the true amplitude of the target measure in the population is A = .1, the standard error for the sample estimate of angular displacement will be roughly SE_δ = 50° for n = 100. Note that a corresponding 95% CI would span across almost 200° of the circle, making the sample estimate itself rather meaningless. In other words, although sample estimates of angular displacement tend to be unbiased (see above), they lacked precision under most of the simulated conditions. Even when IPC measures have a general factor, target measures show substantial amplitude (A = .14), and the sample size is large (n = 500), the standard error will be still roughly SE_δ = 10° and thus the 95% CI will almost span across a full octant.

Study 1 Discussion

Study 1 suggests three major conclusions: First, the majority of SSM statistics proved to be accurate estimators of true values in the population when considering a wide range of conditions that are typical for applied settings. In particular, the unbiasedness assumption holds for elevation, dominance, affiliation, and angular displacement. However, the remaining two SSM parameters, amplitude and goodness of fit, are biased. This means that empirically observed values of amplitude are likely to overestimate true amplitude, and empirically observed values of goodness of fit are likely to underestimate true goodness of fit. In both cases, the amount of bias does not only depend on sample size but also on features of the IPC measure (i.e., whether a general factor is present or not) and features of the target measure in the population (i.e., amplitude or goodness of fit). The latter finding is clearly undesirable as we normally have little idea about the true parameter values in the population, and thus the exact amount of bias remains unknown in applied research.

Second, the precision of the majority of SSM parameters is solely a function of sample size and the presence versus absence of a general factor in the IPC measure. Interestingly, the presence of a general factor has opposite effects on the precision of different SSM parameters: Whereas it reduces the precision of elevation, it increases the precision in dominance, affiliation, and amplitude. This has to do with the joint influence of the parameters of the octant correlation matrix (see Table 1): When octant intercorrelations differ considerably from each other and are in part even negative (as in the matrix of the Interpersonal Adjective Scales), the sum of octant correlations with a target measure (i.e., a multiple of elevation) will be more “stable” during the sampling process as compared with the case of octant intercorrelations being positive and similar to each other (as in the matrix of the IIP-C; see Note 3). The exact influence can be specified by Equation (A6) in the appendix, suggesting that, everything else being equal, the standard error of elevation will be $f_{e}^{I I P - C} / f_{e}^{I A S} = . 737 / . 240 = 3.07$ times greater in the latter as compared with the former case. In contrast, dominance and affiliation (and thereby amplitude) are in part defined as differences between octant correlations with a target measure. These become more “stable” during the sampling process when octant intercorrelations are positive and similar to each other. In particular, Equation (A7) in the appendix suggests that in this case the standard error of any of these parameters will be only $f_{a}^{I I P - C} / f_{a}^{I A S} = . 545 / . 845 = 0.64$ times as large as in the case of highly different octant intercorrelations.

Third, the precision of the remaining two SSM parameters, angular displacement and goodness of fit, depend on further factors related to the true parameters of the target measure in the population (i.e., amplitude and goodness of fit). As hypothesized, higher amplitude in the population leads to a reduced standard error of angular displacement, that is, measures with a more differentiated sinusoidal profile show more “stable” interpersonal themes (that are less affected by sampling error). Nevertheless, in many cases samples larger than 500 persons will be needed to achieve sufficient precision in the determination of the interpersonal theme of a target measure.

Study 2

The aim of Study 2 was to assess whether and under which conditions the bootstrap method would yield accurate CIs for SSM parameters. Therefore, we repeated the Monte Carlo simulation study of Study 1 using exactly the same methods and conditions. The only difference was that, in each of the 5,000 samples that were drawn from a specified population, 95% percentile bootstrap CIs were computed for elevation, dominance, affiliation, amplitude, angular displacement, and goodness of fit using 2,000 bootstrap replicates. The accuracy of the bootstrap method was assessed by means of the empirical coverage rate, that is, the percentage of samples in which the population value of a given parameter ranged within the sample CI. For 95% CIs the empirical coverage rate should be close to 95%. Applying Bradley’s (1978) liberal criterion, we judged the bootstrap method to be accurate when the empirical coverage rate did not deviate more than 2.5% from its nominal level (i.e., when the empirical coverage rate was in the range from 92.5% to 97.5%). To test the joint effects of the various conditions on the accuracy of bootstrap CIs, we conducted several ANOVAs. Therefore, we included the factors and their two-way interactions as predictors, and used the absolute deviance from the nominal coverage rate as the outcome. Again, effects with η < .05 were not considered relevant.

Study 2 Results

For elevation, dominance, and affiliation, accuracy of the bootstrap CIs was simply a function of sample size (see Table 3). Panel A in Figure 5 visualizes these effects for elevation. Accuracy improved with increasing sample size, with mean deviance ranging from roughly 3% for n = 25 to 0.3% for n = 500. The minimum sample size falling below the threshold of 2.5% was n = 50 with a mean deviance of around 1.7%. Thus, the bootstrap method yielded accurate CIs for elevation, dominance, and affiliation when sample size was at least 50.

Table 3.

Effects of Various Conditions on the Absolute Deviance From the Nominal Coverage Rate of Bootstrap Confidence Intervals for Structural Summary Method Parameters.

	e	aff	dom	a	δ	R ²
General factor	.01	0	0	.09	.05	.02
Sample size	.77	.84	.84	.37	.55	.12
Elevation	.03	0	0	0	0	0
Affiliation	0	0	0	.05	.05	.02
Dominance	0	0	0	.08	.09	.03
Goodness of fit	0	0	0	0	0	.23
General factor × Sample size	.01	0	0	.11	.04	.02
General factor × Elevation	.02	0	0	0	0	0
General factor × Affiliation	0	0	0	.01	.01	0
General factor × Dominance	0	0	0	.02	.01	0
General factor × Goodness of fit	0	0	0	0	0	.09
Sample size × Elevation	.02	0	0	0	0	0
Sample size × Affiliation	0	0	0	.06	.04	0
Sample size × Dominance	0	0	0	.09	.06	0
Sample size × Goodness of fit	0	0	0	0	0	.07
Elevation × Affiliation	0	0	0	0	0	0
Elevation × Dominance	0	0	0	0	0	0
Elevation × Goodness of fit	0	0	0	0	0	0
Affiliation × Dominance	0	0	0	.05	.05	.01
Affiliation × Goodness of fit	0	0	0	0	0	0
Dominance × Goodness of fit	0	0	0	0	0	.01
Residuals	.13	.15	.16	.09	.05	.38

Note. N = 1,120 conditions. Entries in the table refer to η².

For amplitude, a more complex pattern emerged (see Table 3). Accuracy was not only influenced by sample size but also by the presence versus absence of a general factor in the IPC measure, amplitude of the target measure in the population, as well as several interactions of these factors. Panel B in Figure 5 visualizes the specific nature of these influences. It shows that accuracy generally increased with increasing sample size, increasing values of A, and presence of a general factor. When considering IPC measures with a general factor, the bootstrap method yielded accurate CIs for amplitude when sample size was at least 75. When considering IPC measures without a general factor, the bootstrap method required sample sizes of at least 150 to yield accurate CIs for amplitude.

The results for angular displacement resembled the results for amplitude. That is, accuracy increased with increasing sample size, increasing values of amplitude of the target measure in the population, and the presence of a general factor (see Panel C in Figure 5). However, the specific sample sizes at which mean deviance was close to or dropped below the threshold of 2.5% were slightly more restrictive: Given that an IPC measure has a general factor, the bootstrap method yielded accurate CIs for angular displacement when sample size was at least 100; for a measure without a general factor, sample sizes greater than 200 were required.

The results for goodness of fit were disappointing. When R² = 1, the population value never ranged within the sample CIs, that is, the empirical coverage rate was 0. This is because 1 is a boundary value in goodness of fit and will rarely appear in samples and resamples. As a consequence, the upper limits of the sample CIs were always smaller than 1, that is, the bootstrap method consistently failed to provide accurate results. Consequently, the most important predictors of accuracy were goodness of fit, sample size, and their two-way interaction (see Table 3; the interaction between presence of a general factor and goodness of fit was also substantial but will be disregarded for simplicity). Panel D Figure 5 shows that accuracy generally increased with increasing sample size as well as when values of goodness of fit fell below .9. In sum, the bootstrap method yielded accurate CIs for goodness of fit only when goodness of fit in the population was suboptimal.

Study 2 Discussion

Study 2 showed that the 95% bootstrap CIs of elevation, affiliation, and dominance are accurate when sample size is at least 50. Moreover, for amplitude and angular displacement, the accuracy of the CIs turned out to be a function of sample size, amplitude of the target measure in the population, and the presence of a general factor in the IPC measure: In particular, when sample size was at least 100 (for IPC measures with a general factor) or 200 (for IPC measures without a general factor), the method provided accurate results. The favorable effect of the general factor on the accuracy of the bootstrap CIs mirrors its effect on the precision of amplitude reported in Study 1. It should be noted, however, that the simulation study only covered target measures with nonnegligible amplitude in the population (i.e., A ≥ .1). Thus, these sample size recommendations may not generalize to situations in which true amplitude is smaller. More generally, the dependency on population parameters of the target variable is clearly undesirable, because it restricts the applicability of the method to circumstances in which the population values of amplitude are known to exceed a certain level. However, because researchers are normally interested in the CIs of variables that are at least moderately associated with affiliation or dominance (i.e., that are hypothesized to have substantial interpersonal content), and because the CIs of affiliation and dominance can be accurately estimated irrespective of population parameters, this drawback may be irrelevant in many cases. Finally, we found that the CIs of goodness of fit are not accurate when goodness of fit in the population is equal to or greater than .9. Because this is expected to be a common situation in practice (e.g., 50% of goodness-of-fit estimates in the published SSM literature listed above were equal or greater than .86), and because there is no way to accurately estimate the population value of goodness of fit, the bootstrap method seems unsuited for computing CIs for goodness of fit.

Study 3

The aim of Study 3 was to assess more precisely when the bootstrap method would yield accurate CIs for amplitude and angular displacement. Therefore, we conducted a third Monte Carlo simulation study following the same procedures as in Study 1 (see above). However, we more focally evaluated sample size, affiliation in the population, and the presence of a general factor in the IPC measure, which was revealed in the second study to be a critical factor in the performance of the CIs.⁶ Specifically, we varied 22 conditions of sample size (ranging from n = 50 to 1,000 in gradually increasing steps), 57 conditions of affiliation in the population (ranging from .02 to .30 in steps of .005), and 2 conditions representing a correlation matrix of the IPC measure with and without a general factor (in line with the previous two studies). Elevation and dominance were fixed to 0, and goodness of fit was fixed to 1. Thus, Study 3 was based on 2,508 simulations. Just as in Study 2, we judged the CIs of amplitude and angular displacement to be accurate when their empirical coverage rates were in the range from 92.5% to 97.5%.

Study 3 Results and Discussion

Figure 6 presents the relation between minimum values of sample size and affiliation (or dominance) in the population that yielded accurate CIs for both amplitude and angular displacement. It appears that with increasing sample size, the bootstrap method was applicable to measures with lower specific interpersonal content; and by the same token, with the more interpersonal a measure is in the population, the bootstrap method was applicable in smaller samples. Again, the presence of a general factor had a favorable effect, that is, allowed applying the method to smaller samples or measures with less interpersonal content. The difference between the two conditions could be exactly quantified using Equation (A7) in the appendix. In fact, after dividing minimum values of affiliation/dominance in the population (AFF_min) by the respective scaling factor (f_a) that corrects for influences of the IPC correlation matrix, and log transforming both (rescaled) AFF_min values and sample size (n), the two were nearly perfectly correlated, r = −.994. Predicting the natural logarithm of minimum values from the natural logarithm of sample size and back-transforming the regression coefficients into the original metric yields the following exponential function:

| A F F_{m i n} | = 2.95 f_{a} n^{- 0.587}

The two curves in Figure 6 visualize this function for the two IPC correlation matrices used in the simulations. Applying this equation shows that, in the case of an IPC measure with a general factor (f_a = .545), a relatively small sample of 100 will require a population value as large as .11, and a sample of 1,000 will only require a population value of .03, to have the bootstrap method produce accurate CIs of amplitude and angular displacement.

This function directly translates into recommendations for practice: Prior to assessing the 95% bootstrap CIs of amplitude and angular displacement, researchers should compare their sample values of affiliation and dominance with the minimum population values given by Equation (3). The more the sample value of either affiliation or dominance exceeds the respective population value, the higher is the probability that the CIs of amplitude and angular displacement are accurate. To provide researchers with a quantitative estimate of this probability, the R function we developed automatically compares the minimum population values given by Equation (3) with the bootstrap distributions of affiliation and dominance. To this end, the scaling factor f_a is computed from the model-based estimates of the IPC correlation matrix as implemented in the R package CircE (Grassi, Luccio, & Blas, 2010). As a result, the R function outputs the percentage of bootstrap resamples in which affiliation or dominance exceeds respective minimal value. This estimate directly quantifies the probability that the CIs of amplitude and angular displacement are accurate in a given data set. For example, when the sample value of affiliation equals the minimum population value (and the sample value of dominance is even lower), the probability estimate will be .50. Or, when the lower limit of the 95% CI of affiliation equals the minimum population value (and the value of dominance is again lower), the probability estimate will be .95. We recommend always reporting the exact probability estimate in future SSM studies, because it helps researchers judge the precision of their inferential conclusions.

Study 4

The aim of Study 4 was to assess whether the bootstrap method is also applicable to the difference between parameters of two correlational profiles. This is an important issue because it would widen the practical applicability of the bootstrap method considerably, allowing for testing assumptions about differences in interpersonal profiles. We considered three types of differences: differences between parameters that represent two uncorrelated target measures, differences between parameters that represent two correlated (ρ = .5) target measures, and differences between parameters that represent a single target measure assessed in two independent samples. Therefore, we conducted a fourth Monte Carlo simulation study following the same procedures as in Study 1. However, we included a second target measure or a second population, respectively, and applied the bootstrap method to the difference between parameters of the two correlational profiles. We systematically varied the type of difference (two uncorrelated measures, two correlated measures, two independent samples), sample size (n = 50, 100, 400), affiliation in the first profile (AFF₁ = .05, .12, .18), and affiliation in the second profile (AFF₂ = AFF₁, 0, −AFF₁). The specific values of sample size and affiliation were selected to cover combinations in which the bootstrap method was slightly over the boundary of its suitability based on Equation (3). In both profiles, elevation and dominance were fixed to 0, and goodness of fit was fixed to 1. The correlation matrix of the IPC measure was set to the general factor case defined in Note 4. Thus, Study 4 was based on 81 simulations. In line with the results of Studies 2 and 3, we expected that (a) CIs for the difference of elevation, affiliation, and dominance would be accurate under any of these conditions, (b) CIs for amplitude and angular displacement would never be accurate when AFF₂ = 0, and, apart from that, (c) CIs for amplitude and angular displacement would be accurate when n = 400, when n = 100 and AFF₁ ≥ .12, and when n = 50 and AFF₁ = .18.

Study 4 Results and Discussion

In general, accuracy estimates were very close to what was expected. First, CIs for the difference of elevation, affiliation, and dominance were accurate under any condition (with mean deviance of 0.9%, 1.0%, and 0.8%, respectively). Second, CIs for amplitude and angular displacement were consistently inaccurate when affiliation in the second profile was 0 (with mean deviance of 15.5%). This corroborates the finding from the former studies that at least one of the interpersonal dimensions of a target measure should exceed a certain value in the population. Third, under the remaining conditions, CIs for the difference of angular displacement were indeed accurate when n = 400 and AFF₁ = .05 (with mean deviance of 1.1%), when n = 100 and AFF₁ = .12 (with mean deviance of 0.8%), and when n = 50 and AFF₁ = .18 (with mean deviance of 1.0%). However, although the CIs for the difference of amplitude were accurate when n = 100 and AFF₁ = .12 (with mean deviance of 2.0%), and when n = 50 and AFF₁ = .18 (with mean deviance of 1.5%), they just failed our stringent criterion for accuracy when n = 400 and AFF₁ = .05 (with mean deviance of 2.9%). An inspection of the empirical coverage rates showed that CIs were too wide, that is, the population value was more often included in the sample CIs than warranted. In terms of classical hypothesis testing, this would lead to the tendency of falsely not rejecting the null hypothesis, concluding that both parameters are equal even if they are not (i.e., a Type II error). Thus, when applying the bootstrap method to the difference of amplitude, the minimum values presented in Figure 6 may lead to somewhat conservative conclusions. Apart from this limitation, the results provide evidence that the bootstrap method and the recommendations from Study 3 are broadly generalizable to the difference between two parameters.

Study 5

As the results of the simulation studies were satisfying, we sought to apply the newly developed approach to actual data. For pedagogical purposes, we chose to use very familiar constructs that have previously been investigated in relation to the IPC, and for which there are remaining questions about interpersonal differences between constructs, and whether the constructs operate similarly across gender (i.e., a grouping variable). Specifically, we chose the DSM-5 Section II PD constructs because of their familiarity, and the ongoing theoretical importance of interpersonal dysfunction in these disorders (Hopwood, Wright, Ansell, & Pincus, 2013; Tyrer, Reed, & Crawford, 2015). We chose to use the problems-based version of the IPC (i.e., the IIP-C) given its focus on maladaptive expressions of interpersonal functioning (for previous studies linking the IIP-C to Section II PDs, see e.g., Pincus & Wiggins, 1990; Soldz, Budman, Demby, & Merry, 1993).

Here we applied the R functions to data collected in a large university-based sample (N = 1,166). The short form of the IIP-C (Soldz et al., 1995) was used, and a number of the well-known self-report PD scales from the Personality Diagnostic Questionnaire–4+ (Hyler, 1994) were used as target measures. In addition to calculating the structural summary parameters and associated CIs for each of the DSM-5 Section II PD constructs, we also compared two sets of constructs for which the interpersonal features have often been compared and contrasted. Namely, we compared the schizoid and avoidant PD constructs (e.g., Alden, Laposa, Taylor, & Ryder, 2002; Livesley, West, & Taney, 1985), and the narcissistic and antisocial constructs (e.g., Blackburn, 2007). We additionally compared the profiles across gender for the Cluster B (i.e., dramatic/erratic PDs; antisocial, borderline, histrionic, and narcissistic) PDs, constructs for which gender bias or differential expression across gender has been hotly debated (e.g., Anderson, Sankis, & Widiger, 2001; Busch, Balsis, Morey, & Oltmanns, 2016; Corbitt & Widiger, 1995; Jane, Oltmanns, South, & Turkheimer, 2007; Moffitt, Caspi, Rutter, & Silva, 2001).

Study 5 Method

Participants and Procedure

Data were collected from a large North American undergraduate sample (N = 1166; mean age = 19.0 years, SD = 1.7) that was roughly balanced on gender (53.3% female). Participants completed the Inventory of Interpersonal Problems–Short Circumplex (IIP-SC) and the Personality Diagnostic Questionnaire–4 (PDQ-4) as part of a larger battery of questionnaires online in exchange for partial course credit.

Measures

The IIP-SC (Soldz et al., 1995) is a 32-item measure of interpersonal problems and associated distress. Items assess interpersonal behaviors that an individual does in excess (i.e., “I do . . . too much”) or finds difficult to do (“It is hard for me to . . .”). The IIP-SC provides coverage for the full range of interpersonal content mapped by the IPC with eight, 4-item scales. The octant scale names are provided in Figure 1A. Internal consistency of the scales is adequate in the current sample (Mdn α = .78; range = .64-.87).

The PDQ-4 (Hyler, 1994) is a 99-item true–false instrument with item content that corresponds directly to the criteria for the DSM-5 Section II PDs. PD scales were treated as continuous symptom counts for the purposes of this study given that continuous psychopathology scales are generally more reliable and valid than categorical markers (Markon, Chmielewski, & Miller, 2011). Internal consistencies ranged from .43 (obsessive-compulsive) to .71 (avoidant; Mdn = .56). Although some values were modest, it is reassuring that internal consistency appears to have limited impact on criterion validity estimates (McCrae, Kurtz, Yamagata, & Terraciano, 2010; Schmitt, 1996).

Results and Discussion

In a preliminary step, we tested whether the structure of the IIP-SC had an adequate fit to a circumplex model with equal spacing and equal communality using the R package CircE (Grassi et al., 2010). CircE provides several common fit indices including the Tucker–Lewis index (TLI), the comparative fit index (CFI), and the root mean square error of approximation (RMSEA). We found that the fit of such a model was unsatisfactory by current standards (e.g., Hopwood & Donnellan, 2010), with CFI = .824, TLI = .795, and RMSEA = .169. Follow-up analyses suggested that this was mainly due to a violation of the equal spacing requirement (i.e., a model with equal communality and unequal spacing showed acceptable fit, with CFI = .958, TLI = .931, and RMSEA = .098). As such minor deviations from perfect structure usually have little practical impact (Gurtman & Pincus, 2000), we proceeded with the main analyses.

Model-based estimates of the four parameters of the IIP-SC correlation matrix were ρ₁ = .580, ρ₂ = .323, ρ₃ = .134, and ρ₄ = .070. Table 4 presents structural summary statistics and their 95% bootstrap CIs for all PD scales. According to Equation (3), N = 1,166 and f_a = .625 requires AFF or DOM to be greater than |. 029| to have the bootstrap method yield accurate CIs for amplitude and angular displacement. The probability of fulfilling these requirements exceeded .95 in all cases except for obsessive–compulsive PD (see last column of Table 4). All PD scales were significantly associated with interpersonal distress, ranging from antisocial and schizoid PDs with relatively smaller effects, to paranoid, schizotypal, borderline, histrionic, narcissistic, and obsessive-compulsive with relatively moderate effects, to avoidant and dependent with relatively larger effects. Moreover, all but the obsessive–compulsive PD scale showed notable amplitudes.

Table 4.

Structural Summary Statistics With 95% Confidence Intervals for Personality Disorder (PD) Scales.

PD scale	e	aff	dom	a	δ	R ²	Prob
Paranoid	.250 [.218, .280]	−.094 [−.129, −.060]	.117 [.080, .152]	.150 [.115, .189]	128.9° [116.7°, 141.6°]	.802	1
Schizoid	.185 [.149, .221]	−.215 [−.252, −.178]	−.024 [−.065, .015]	.216 [.182, .254]	186.5° [176.0°, 198.4°]	.903	1
Schizotypal	.229 [.196, .262]	−.107 [−.141, −.073]	.035 [−.003, .071]	.113 [.080, .151]	161.8° [145.0°, 181.6°]	.800	1
Antisocial	.124 [.090, .159]	−.099 [−.133, −.063]	.203 [.170, .238]	.226 [.193, .263]	115.9° [107.2°, 124.1°]	.964	1
Borderline	.277 [.242, .309]	−.036 [−.067, −.006]	.097 [.058, .138]	.103 [.065, .145]	110.4° [93.9°, 127.9°]	.872	1
Histrionic	.201 [.168, .233]	.115 [.081, .147]	.147 [.111, .182]	.186 [.156, .217]	52.0° [40.2°, 63.2°]	.767	1
Narcissistic	.202 [.168, .236]	−.062 [−.093, −.029]	.179 [.145, .213]	.189 [.155, .224]	109.0° [99.4°, 118.1°]	.957	1
Avoidant	.331 [.300, .359]	−.061 [−.097, −.027]	−.174 [−.206, −.139]	.184 [.153, .216]	250.6° [238.8°, 261.5°]	.911	1
Dependent	.312 [.280, .341]	.062 [.030, .096]	−.069 [−.106, −.030]	.093 [.056, .131]	312.3° [291.4°, 335.6°]	.654	.976
Obsessive-compulsive	.228 [.193, .261]	.011 [−.021, .041]	−.005 [−.038, .032]	.012	337.4°	.117	.130

Note. N = 1,166. Prob = probability of accurate confidence intervals for amplitude and angular displacement.

A number of illuminating results emerged from the cross-construct and cross-gender comparisons on interpersonal problem profiles. First, we replicated the finding that schizoid PD differs from avoidant PD in terms of interpersonal theme (i.e., angle difference CI does not contain 0)—schizoid PD is associated with a prototypically cold interpersonal profile, whereas avoidant is prototypically cold-submissive (see Table 5). However, this method further shows that these two constructs also differed significantly in interpersonal distress (i.e., avoidant elevation > schizoid elevation; elevation difference CI does not contain 0), but neither is more specifically interpersonal (i.e., amplitude difference CI does contain 0). This pattern of results is highly concordant with the theoretical distinction between these constructs (Alden et al., 2002), which suggests that although both disorders are highly interpersonal and characterized by social-withdrawal, schizoid PD is not associated with strong desires for social contact, and therefore people with this disorder are less distressed by their isolation. Moving to narcissistic and antisocial PDs, the results in Table 5 indicate that these two constructs reliably differed only in the level of distress, not in the theme or differentiation. Consistent with theoretical formulations of these constructs, antisocial PD was associated with significantly lower distress. This small subset of findings offers a brief but hopefully clear demonstration of the utility of this method for theoretically based investigations in to the interpersonal features of constructs and measures.

Table 5.

Structural Summary Statistics With 95% Confidence Intervals for Differences of Selected Personality Disorder (PD) Scales.

PD scale	e	aff	dom	a	δ
Schizoid versus avoidant	−.146 [−.185, −.108]	−.153 [−.200, −.109]	.149 [.104, .193]	.032 [−.014, .079]	64.1° [49.1°, 79.0°]
Narcissistic versus antisocial	.079 [.041, .116]	.037 [−.001, .075]	−.024 [−.061, .012]	−.037 [−.076, .003]	7.0° [−2.9°, 17.4°]

Note. N = 1,166.

Beyond comparing constructs within a given sample, comparing differential patterning of the interpersonal profile across groups, or the manner in which group membership moderates the observed associations among scales, is also useful. As noted above, the question of differences in the expression of PD across genders has been of substantive interest. Much of this debate has focused on the disorders from the dramatic/erratic cluster of PDs. Table 6 catalogues the SSM statistics and associated CIs for each of the four Cluster B disorders (antisocial, borderline, histrionic, and narcissistic) by gender, and also the differences in statistics with CIs between genders. Note that these analyses require computing new group-specific probability estimates for the accuracy of CIs. Here it resulted that CIs for borderline in the male sample might be biased (i.e., the probability estimate was only .67), and thus comparisons between genders for borderline should be interpreted with caution. Nevertheless, an interesting set of results emerged. With the exception of antisocial PD, all PDs were associated with higher levels of distress in men. Furthermore, antisocial and borderline PD were associated with a more differentiated (i.e., specific) interpersonal profile in women. Finally, no differences were observed in interpersonal theme (i.e., angle).

Table 6.

Gender Differences in Structural Summary Statistics With 95% Confidence Intervals for Cluster B Personality Disorder (PD) Scales.

PD scale	G	e	aff	dom	a	δ	R ²	Prob
Ant	f	.138 [.088, .186]	−.107 [−.159, −.055]	.228 [.175, .279]	.252 [.197, .305]	115.2° [103.7°, 126.0°]	.965	1
	m	.128 [.075, .177]	−.038 [−.087, .010]	.157 [.108, .206]	.161 [.113, .213]	103.8° [86.4°, 120.4°]	.932	1
	dif	.010 [−.061, .082]	−.069 [−.140, .007]	.071 [−.001, .139]	.090 [.014, .164]	−11.4° [−31.8°, 9.3°]
Bor	f	.251 [.205, .296]	−.068 [−.112, −.024]	.126 [.067, .185]	.143 [.088, .204]	118.2° [100.8°, 138.0°]	.947	.995
	m	.320 [.271, .372]	.012 [−.036, .059]	.054 [.003, .104]	.055 [.015, .109]	77.4° [12.2°, 134.1°]	.500	.666
	dif	−.069 [−.136, −.002]	−.080 [−.146, −.015]	.072 [−.003, .151]	.088 [.012, .158]	−40.9° [−109.6°, 20.0°]
His	f	.164 [.118, .206]	.111 [.069, .156]	.175 [.125, .224]	.207 [.168, .250]	57.6° [42.6°, 70.7°]	.760	1
	m	.248 [.198, .296]	.111 [.067, .154]	.120 [.067, .169]	.163 [.116, .213]	47.4° [29.4°, 63.7°]	.771	.999
	dif	−.084 [−.145, −.022]	.001 [−.060, .063]	.055 [−.013, .127]	.044 [−.019, .107]	−10.2° [−32.3°, 11.2°]
Nar	f	.172 [.126, .217]	−.080 [−.128, −.034]	.202 [.151, .249]	.217 [.166, .269]	111.7° [99.7°, 123.2°]	.972	1
	m	.244 [.191, .297]	−.029 [−.074, .013]	.146 [.096, .192]	.149 [.102, .196]	101.2° [84.3°, 120.1°]	.902	1
	dif	−.072 [−.143, −.002]	−.051 [−.115, .014]	.056 [−.014, .124]	.068 [−.002, .135]	−10.4° [−31.1°, 11.6°]

Note. N = 1,166. G = gender; f = female; m = male; dif = difference; Prob = probability of accurate confidence intervals for amplitude and angular displacement.

Given that these are the first investigations of this kind, the sample is limited to undergraduates, and PD criteria are self-reported, special caution should be taken when interpreting these gender-based results. In addition, the structure of the IIP-SC clearly deviated from circumplexity, and it is currently unclear how this might affect the accuracy of the bootstrap method. We should also mention that we did not control the family-wise error rate (e.g., by adopting a higher than 95% level for the bootstrap CIs), that is, our results may include more false positive findings than nominally expected. However, the comparison of profiles across constructs resulted in expected differences, and thus, we can be somewhat assured that these differences are meaningful.

General Discussion

Seminal work has argued that construct validation requires that novel (and existing) constructs and scales be placed within a well-articulated “nomological net” (Cronbach & Meehl, 1955; Guttman, 1970). In few areas of psychology has this approach gained more traction than in personality psychology, given the emphasis on overarching comprehensive models and associated measures. For example, it has become commonplace for researchers to compare patterns of correlations with the five-factor model traits and facets across target measures of interest. Despite the frequency with which this approach is applied, it is most often used in an impressionistic manner, relying on the “eyeball” test or a coarse summary profile correlation. Yet the IPC, with its highly specified structure provides a precise framework from which an expected pattern of correlations should emerge, to the extent a target variable is interpersonal in nature. Gurtman (1992) quantified this using the SSM, drastically reducing the complexity of the correlational profile, and providing substantively interpretable parameters that provide a basis for interpreting this profile. To date, however, this approach has remained descriptive. We propose a resampling-based (i.e., bootstrap) method to provide the necessary quantitative tools to move this analytic approach from descriptive to inferential. In a series of five studies, we stressed the need of such an approach by showing that the currently used point estimates are often biased, evaluated the accuracy of the approach across a wide range of possible sample sizes and parameter values, and demonstrated its utility for posing theoretical questions using real-world data.

Our simulation studies showed that the bootstrap method can provide accurate CIs for SSM statistics (except goodness of fit) when sample size is at least 50. However, because the accuracy of the CI for amplitude and angular displacement is not only influenced by sample size but also by the presence versus absence of a general factor in the IPC measure and affiliation/dominance of the target measure in the population, we recommend that researchers also report the probability that these CIs are accurate. For this purpose, we incorporated the results from Study 3 into the R function, providing probability estimates for the CIs in a specific data set. As a general rule of thumb we propose that CIs of amplitude and angular displacement should not be interpreted when the probability estimate is below .50. In this case, researchers should refrain from comparing SSM parameter across constructs or groups, and from drawing any inferential conclusions. Probability estimates in the range from .50 to .95 indicate that the CIs of amplitude and angular displacement should be interpreted with caution as they might be biased to some extent. Ideally, probability estimates should exceed .95, which would indicate that the CIs of amplitude and angular displacement are fully trustworthy.

However, it should be noted that meeting this stringent condition will require samples that may be larger than one might expect. For example, the simulation data from Study 3 show that when the target variable has marked interpersonal content (i.e., either affiliation or dominance is at least |. 15| in the population) and the IPC measure has a general factor similar to the IIP-C, a sample size of 300 is needed to have a good chance (i.e., about 84%) to meet this stringent condition. In contrast, a sample size of 120 suffices to have the same chance to meet a less stringent criterion (i.e., get a probability estimate that exceeds .50). Note that, in any case, these sample sizes are much larger than would be expected by Figure 6. This is because Figure 6 does not take into account sampling error, and thus should not be used for the purpose of sample size planning. The general message here is that the SSM, including its newly developed bootstrap extension, requires larger samples to be fully interpretable than many researchers might have expected before. In this regard, the SSM approach joins other popular approaches, such as latent variable modeling, where sample size considerations are of central importance (e.g., Tanaka, 1987).

From a methodological point of view, we consider the bootstrap extension to be an important advance within the SSM approach to interpersonal construct validation. Reporting confidence intervals has been recommended practice in psychology journals for more than 15 years (Wilkinson & Task Force on Statistical Inference, 1999), and will now be possible for studies applying the SSM. This is especially needed as the point estimates of amplitude are biased under many circumstances. Although this bias decreases with increasing sample size, it is obvious that reporting the CI of amplitude is more desirable as it includes the population value with a given probability. Another desirable feature of the bootstrap method is that it performed quite well in capturing the theoretically expected interdependence between amplitude and the precision of angular displacement. Study 1 showed that the sampling error in angular displacement is indeed negatively related to the amount of amplitude, and thus the CI of angular displacement should be much narrower in a measure with high as compared with low amplitude. This is exactly what the bootstrap CIs reproduce. In sum, we strongly recommend reporting CIs in future studies that apply the SSM to assess the interpersonal nature of target constructs or measures. To maximize the accessibility of this method for applied researchers, we provide the most recent version of the R package “ssm” together with instructions and example data from Study 5 on the second author’s website (http://www.personalityprocesses.com/ssm/).

Theory and method are closely intertwined. There is an ever-present tension between the two because of the fact that one often places a limit on the other. The accumulating literature relating constructs to the IPC nomological net has been constrained by an inability to directly compare the interpersonal features of constructs to draw quantitatively defensible inferences. The current method loosens the bindings, freeing up researchers to contrast constructs and groups within the interpersonal nomological net. We demonstrated this using the well-known constructs of the DSM-5 Section II PDs and the IIP-C. As expected, using the bootstrap approach, we found that schizoid and avoidant PD differed in distress and theme, but were equally interpersonally differentiated (i.e., equivalent amplitude). In contrast, narcissistic and antisocial PD symptoms were only different in terms of distress, such that narcissism has a stronger average octant association (i.e., elevation), also in line with theoretical expectations. These findings demonstrate that using real data that relatively well established theoretical predictions emerge in the expectable ways. More novel are the results that emerge from the gender comparisons, which suggest that the link between the cluster B PD symptoms and distress is stronger in men. At the same time, the interpersonal themes of these constructs do not differ significantly across genders, suggesting that theoretical differences in expression are unlikely to manifest in the interpersonal style of individuals with significant PD symptoms.

Many more applications are possible. For example, researchers can now test predictions about the interpersonal content of constructs based on a priori hypotheses. Common is the case in practice where the resulting angle from an SSM is within a few degrees of the predicted angle, leaving it up to the researcher to “call it safe or call it out.” Additionally, many scales are developed to measure constructs that are thought to be interpersonally heterogeneous. This often results in a modest, flat, or complex profile on the SSM. By using this method, the empirical overlap among items can be determined. This can lead to a clarification of a modest profile, or can serve as the basis for refining scales to meet a specific interpersonal profile by trimming individual items that significantly differ from the others in terms of interpersonal content (see, e.g., Gurtman, 1993). Doubtlessly, researchers will come up with additional applications for this method in the service of scale construction, construct evaluation, and theory building.

Introducing the bootstrap into the SSM represents a stimulating methodological breakthrough. However, much work needs to be done. We will close with highlighting some lines of methodological research we would like to see in the coming years. First, it is an open question how the bootstrap approach relates to the circular statistics approach as outlined by Wright et al. (2009). The circular statistics approach has been proposed as an alternative to the SSM when the focus is on the comparison of octant mean scores between groups, and it allows for the computation of CIs for angular displacement. However, comparing mean scores between two groups may be reconceptualized as a special case of the correlational summary, that is, correlating octant scales with a binary variable. Thus, it would be interesting to see if both approaches converge in their estimation of CIs for angular displacement. Second, and somewhat related, future Monte Carlo simulation studies should test if the bootstrap method is also accurate when the target measure is binary or nonnormally distributed, or when the correlational structure of the IPC measure deviates from circumplexity in the population. This would strengthen the evidence that the bootstrap method is generalizable to situations that are commonly found in SSM research practice. Third, it may be worthwhile to test the percentile bootstrap we implemented in our R script against other resampling based methods, for example, the bias-corrected and accelerated bootstrap, the jackknife, or the Monte Carlo method. These methods have been successfully used in the field of mediation analysis (MacKinnon et al., 2004; Preacher & Hayes, 2008), although the debate on which method is most accurate is continuing (Biesanz et al., 2010). The most attractive alternative approach might be the Monte Carlo method as it does not require access to the original data but would work with information on means, standard deviations, and intercorrelations (MacKinnon et al., 2004; Preacher & Selig, 2012). Finally, a highly desirable long-term goal would be extending the method to be able to handle measurement error, that is, to account for scale differences in reliability and provide CIs for the true population values of SSM parameters (e.g., Charles, 2005). We look forward to future research applying and extending the SSM, and thereby stimulating interpersonal theory building and the advance of circumplex statistics.

Footnotes

Appendix

The following equations can be used to compute elevation (e), amplitude (a), and angular displacement (δ) from observed correlations ( $r_{i}^{o}$ ) of eight IPC octant scales (located at θ_i) with a target measure (see Gurtman, 1992):

The following equations provide estimates of how the correlation matrix of an interpersonal circumplex measure (defined by the parameters ρ₁, ρ₂, ρ₃, and ρ₄; see Table 1) influences the standard error of elevation and amplitude:

Authors’ Note

Both authors contributed equally to this article (within the 95% confidence interval). The opinions expressed herein are solely those of the authors and do not represent those of the funding source. Parts of this article were presented 2012 at the 15th annual meeting of the Society for Interpersonal Theory and Research (SITAR) in Montreal. The authors thank Pamela Sadler and Aaron Pincus for helpful comments on an earlier draft of this article.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Aidan G. C. Wright received support from the National Institute of Mental Health (L30 MH101760).

Notes

References

Acton

G. S.

Revelle

(2002). Interpersonal personality measures show circumplex structure based on new psychometric criteria. Journal of Personality Assessment, 79, 446-471.

Acton

G. S.

Revelle

(2004). Evaluation of ten psychometric criteria for circumplex structure. Methods of Psychological Research, 9, 1-27.

Alden

L. E.

Laposa

J. M.

Taylor

C. T.

Ryder

A. G.

(2002). Avoidant personality disorder: Current status and future directions. Journal of Personality Disorders, 16, 1-29.

Alden

L. E.

Wiggins

J. S.

Pincus

A. L.

(1990). Construction of circumplex scales of the Inventory of Interpersonal Problems. Journal of Personality Assessment, 55, 521-536.

Anderson

K. G.

Sankis

Widiger

T. A.

(2001). Pathology versus statistical infrequency: Potential sources of gender bias in personality disorder criteria. Journal of Nervous and Mental Disease, 189, 661-668.

Bartholomew

Horowitz

L. M.

(1991). Attachment styles among young adults: A test of a four-category model. Journal of Personality and Social Psychology, 61, 226-244.

Benjamin

L. S.

(1974). Structural analysis of social behavior (SASB). Psychological Review, 81, 392-425.

Biesanz

J. C.

Falk

C. F.

Savalei

(2010). Assessing mediational models: Testing and interval estimation for indirect effects. Multivariate Behavioral Research, 45, 661-701.

Blackburn

(2007). Personality disorder and antisocial deviance: Comments on the debate on the Psychopathy Checklist–Revised. Journal of Personality Disorders, 21, 142-159.

10.

Bornstein

R. F.

Languirand

M. A.

Geiselman

K. J.

Creighton

J. A.

West

M. A.

Gallagher

H. A.

Eisenhart

E. A.

(2003). Construct validity of the Relationship Profile Test: A self-report measure of dependency-detachment. Journal of Personality Assessment, 80, 64-74.

11.

Bradley

J. V.

(1978). Robustness? British Journal of Mathematical and Statistical Psychology, 31, 144-152.

12.

Brown

N. A.

Sherman

R. A.

(2014). Predicting interpersonal behavior using the Inventory of Individual Differences in the Lexicon (IIDL). Journal of Research in Personality, 51, 23-28.

13.

Browne

M. W.

(1992). Circumplex models for correlation matrices. Psychometrika, 57, 469-497.

14.

Busch

A. J.

Balsis

Morey

L. C.

Oltmanns

T. F.

(2016). Gender differences in Borderline Personality Disorder features in an epidemiological sample of adults age 55-64: Self versus informant report. Journal of Personality Disorders, 30, 419-432. doi:10.1521/pedi_2015_29_202

15.

Cain

N. M.

Ansell

E. B.

Wright

A. G. C.

Hopwood

C. J.

Thomas

K. M.

Pinto

A. N.

. . . Grilo

C. M.

(2012). Interpersonal pathoplasticity in the course of major depression. Journal of Consulting and Clinical Psychology, 80, 78-86.

16.

Charles

E. P.

(2005). The correction for attenuation due to measurement error. Clarifying concepts and creating confidence sets. Psychological Methods, 10, 206-226.

17.

Corbitt

E. M.

Widiger

T. A.

(1995). Sex differences among the personality disorders: An exploration of the data. Clinical Psychology: Science and Practice, 2, 225-238.

18.

Cronbach

L. J.

Meehl

P. E.

(1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281-302.

19.

Desmet

Vanheule

Groenvynck

Verhaeghe

Vogel

Bogaerts

(2007). The Depressive Experiences Questionnaire: An inquiry into different scoring procedures. European Journal of Psychological Assessment, 23, 89-98.

20.

Dinger

Barrett

M. S.

Zimmermann

Schauenburg

Wright

A. G. C.

Renner

Zilcha-Mano

Barber

J. P.

(2015). Interpersonal problems, dependency and self-criticism in major depressive disorder. Journal of Clinical Psychology, 71, 93-104.

21.

Efron

Tibshirani

R. J.

(1993). An introduction to the bootstrap. London, England: Chapman & Hall.

22.

Erickson

T. M.

Newman

M. G.

Siebert

E. C.

Carlile

J. A.

Scarsella

G. M.

Abelson

J. L.

(2016). Does worrying mean caring too much? Interpersonal prototypicality of dimensional worry controlling for social anxiety and depressive symptoms. Behavior Therapy, 47, 14-28.

23.

Fabrigar

L. R.

Visser

P. S.

Browne

M. W.

(1997). Conceptual and methodological issues in testing the circumplex structure of data in personality and social psychology. Personality and Social Psychology Review, 1, 184-203.

24.

Grassi

Luccio

Blas

(2010). CircE: An R implementation of Browne’s circular stochastic process model. Behavior Research Methods, 42, 55-73.

25.

Gurtman

M. B.

(1991). Evaluating the interpersonalness of personality scales. Personality and Social Psychology Bulletin, 17, 670-677.

26.

Gurtman

M. B.

(1992). Construct validity of interpersonal personality measures: The interpersonal circumplex as a nomological net. Journal of Personality and Social Psychology, 63, 105-118.

27.

Gurtman

M. B.

(1993). Constructing personality tests to meet a structural criterion: Application of the interpersonal circumplex. Journal of Personality, 61, 237-263.

28.

Gurtman

M. B.

(1999). Social competence: An interpersonal analysis and reformulation. European Journal of Psychological Assessment, 15, 233-245.

29.

Gurtman

M. B.

(2009). Exploring personality with the Interpersonal Circumplex. Social and Personality Psychology Compass, 3/4, 601-619.

30.

Gurtman

M. B.

Balakrishnan

J. D.

(1998). Circular measurement redux: The analysis and interpretation of interpersonal circle profiles. Clinical Psychology: Science and Practice, 5, 344-360.

31.

Gurtman

M. B.

Lee

D. L.

(2009). Sex differences in interpersonal problems: A circumplex analysis. Psychological Assessment, 21, 515-527.

32.

Gurtman

M. B.

Pincus

A. L.

(2000). Interpersonal Adjective Scales: Confirmation of circumplex structure from multiple perspectives. Personality and Social Psychology Bulletin, 26, 374-384.

33.

Gurtman

M. B.

Pincus

A. L.

(2003). The circumplex model: Methods and research applications. In Schinka

Velicer

(Eds.), Comprehensive handbook of psychology, Vol. 2: Research methods in psychology (pp. 407-428). New York, NY: Wiley.

34.

Guttman

(1954). A new approach to factor analysis: The Radex. In Lazarsfeld

P. F.

(Ed.), Mathematical thinking in the social sciences (pp. 258-348). New York, NY: Free Press.

35.

Guttman

(1970). Integration of test design and analysis. In DuBois

P. H.

(Ed.), Proceedings of the 1969 Invitational Conference on Testing Problems (pp. 53-65). Princeton, NJ: Educational Testing Service.

36.

Haggerty

Blake

Siefert

C. J.

(2010). Convergent and divergent validity of the relationship profile test: Investigating the relationship with attachment, interpersonal distress, and psychological health. Journal of Clinical Psychology, 66, 339-354.

37.

Hatcher

Rogers

(2009). Development and validation of a measure of interpersonal strengths: The Inventory of Interpersonal Strengths. Psychological Assessment, 21, 554-569.

38.

Hill

R. W.

Zrull

M. C.

Turlington

(1997). Perfectionism and interpersonal problems. Journal of Personality Assessment, 69, 81-103.

39.

Hopwood

C. J.

Ansell

E. B.

Pincus

A. L.

Wright

A. G. C.

Lukowitsky

M. R.

Roche

M. J.

(2011). The circumplex structure of interpersonal sensitivities. Journal of Personality, 79, 708-740.

40.

Hopwood

C. J.

Burt

S. A.

Keel

P. K.

Neale

M. C.

Boker

S. M.

Klump

K. L.

(2013). Interpersonal problems associated with multidimensional personality questionnaire traits in women during the transition to adulthood. Assessment, 20, 60-67.

41.

Hopwood

C. J.

Donnellan

M. B.

(2010). How should the internal structure of personality inventories be evaluated? Personality and Social Psychology Review, 14, 332-346.

42.

Hopwood

C. J.

Wright

A. G. C.

Ansell

E. B.

Pincus

A. L.

(2013). The interpersonal core of personality pathology. Journal of Personality Disorders, 27, 271-295.

43.

Hyler

S. E.

(1994). PDQ-4 and PDQ-4+: Instructions for use. Unpublished manuscript, Columbia University.

44.

Jane

J. S.

Oltmanns

T. F.

South

S. C.

Turkheimer

(2007). Gender bias in diagnostic criteria for personality disorders: An item response theory analysis. Journal of Abnormal Psychology, 116, 166-175.

45.

Kiesler

D. J.

Schmidt

J. A.

Wagner

C. C.

(1997). A circumplex inventory of impact messages: An operational bridge between emotion and interpersonal behavior. In Plutchik

Conte

(Eds.), Circumplex models of personality and emotions (pp. 221-244). Washington, DC: American Psychological Association Press.

46.

Leary

(1957). Interpersonal diagnosis of personality. New York, NY: Ronald Press.

47.

Livesley

W. J.

West

Tanney

(1985). Historical comment on the DSM-III schizoid and avoidant personality disorders. American Journal of Psychiatry, 142, 1344-1347.

48.

Locke

K. D.

(2000). Circumplex scales of interpersonal values: Reliability, validity, and applicability to interpersonal problems and personality disorders. Journal of Personality Assessment, 75, 249-267.

49.

Locke

K. D.

Sadler

(2007). Self-efficacy, values, and complementarity in dyadic interactions: Integrating interpersonal and social-cognitive theory. Personality and Social Psychology Bulletin, 33, 94-109.

50.

MacKinnon

D. P.

Lockwood

C. M.

Williams

(2004). Confidence limits for the indirect effect: Distribution of the product and resampling methods. Multivariate Behavioral Research, 39, 99-128.

51.

Markey

Anderson

J. M.

Markey

(2013). Using behavioral mapping to examine the validity of the IPIP-IPC. Assessment, 20, 165-174.

52.

Markey

P. M.

Markey

C. N.

(2009). A brief assessment of the interpersonal circumplex: The IPIP-IPC. Assessment, 16, 352-361.

53.

Markon

K. E.

Chmielewski

Miller

C. J.

(2011). The reliability and validity of discrete and continuous measures of psychopathology: A quantitative review. Psychological Bulletin, 137, 856-879.

54.

McCrae

R. R.

Costa

P. T.

(1989). The structure of interpersonal traits: Wiggins’s circumplex and the five-factor model. Journal of Personality and Social Psychology, 56, 586-595.

55.

McCrae

R. R.

Kurtz

J. E.

Yamagata

Terraciano

(2010). Internal consistency, retest reliability, and their implications for personality scale validity. Personality and Social Psychology Review, 15, 28-50.

56.

McEvoy

P. M.

Burgess

M. M.

Page

A. C.

Nathan

Fursland

(2013). Interpersonal problems across anxiety, depression, and eating disorders: A transdiagnostic examination. British Journal of Clinical Psychology, 52, 129-147.

57.

Mendoza

J. L.

Hart

D. E.

Powell

(1991). A bootstrap confidence interval based on a correlation corrected for range restriction. Multivariate Behavioral Research, 26, 255-269.

58.

Miller

J. D.

Price

Gentile

Lynam

D. R.

Campbell

W. K.

(2012). Grandiose and vulnerable narcissism from the perspective of the interpersonal circumplex. Personality and Individual Differences, 53, 507-512.

59.

Moffitt

T. E.

Caspi

Rutter

Silva

P. A.

(2001). Sex differences in antisocial behaviour: Conduct disorder, delinquency and violence in the Dunedin Longitudinal Study. Cambridge, England: Cambridge University Press.

60.

Nagy

Marsh

H. W.

Lüdtke

Trautwein

(2009). Representing the circles in our minds: Confirmatory factor analysis of circumplex structures and profiles. In Teo

Khine

M. S.

(Eds.), Structural equation modelling in educational research: Concepts and applications (pp. 287-315). Rotterdam, Netherlands: Sense.

61.

Pincus

A. L.

Ansell

E. B.

(2013). Interpersonal theory of personality. In Suls

Tennen

(Eds.), Handbook of psychology: Vol. 5. Personality and social psychology (2nd ed., pp. 141-159). Hoboken, NJ: Wiley.

62.

Pincus

A. L.

Gurtman

M. B.

(2003). Interpersonal assessment. In Wiggins

J. S.

(Ed.), Paradigms of personality assessment (pp. 246-261). New York, NY: Guilford.

63.

Pincus

A. L.

Wiggins

J. S.

(1990). Interpersonal problems and conceptions of personality disorders. Journal of Personality Disorders, 4, 342-352.

64.

Pincus

A. L.

Wright

A. G. C.

(2011). Interpersonal diagnosis of psychopathology. In Horowitz

L. M.

Strack

(Eds.), Handbook of interpersonal psychology: Theory, research, assessment, and therapeutic interventions (pp. 359-381). New York, NY: Wiley.

65.

Preacher

K. J.

Hayes

A. F.

(2008). Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behavior Research Methods, 40, 879-891.

66.

Preacher

K. J.

Selig

J. P.

(2012). Advantages of Monte Carlo confidence intervals for indirect effects. Communication Methods and Measures, 6, 77-98.

67.

Przeworski

Newman

M. G.

Pincus

A. L.

Kasoff

Yamasaki

A. S.

Castonguay

L. G.

(2011). Interpersonal pathoplasticity in individuals with generalized anxiety disorder. Journal of Abnormal Psychology, 120, 286-298.

68.

R Development Core Team. (2011). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from http://www.R-project.org

69.

Rodebaugh

T. L.

Gianoli

M. O.

Turkheimer

Oltmanns

T. F.

(2010). The interpersonal problems of the socially avoidant: Self and peer shared variance. Journal of Abnormal Psychology, 119, 331-340.

70.

Ruscio

Mullen

(2012). Confidence intervals for the probability of superiority effect size measure and the area under a receiver operating characteristic curve. Multivariate Behavioral Research, 47, 201-223.

71.

Sato

McCann

(2007). Sociotropy-autonomy and interpersonal problems. Depression and Anxiety, 24, 153-162.

72.

Schmitt

(1996). Uses and abuses of coefficient alpha. Psychological Assessment, 8, 350-353.

73.

Soldz

Budman

Demby

Merry

(1993). Representation of personality disorders in circumplex and ﬁve-factor space: Explorations with a clinical sample. Psychological Assessment, 5, 41-52.

74.

Soldz

Budman

Demby

Merry

(1995). A short form of the Inventory of Interpersonal Problems Circumplex scales. Assessment, 2, 53-63.

75.

Tanaka

J. S.

(1987). “How big is big enough?” Sample size and goodness of fit in structural equation models with latent variables. Child Development, 58, 134-136.

76.

Tracey

T. J. G.

(1997). RANDALL: A Microsoft FORTRAN program for a randomization test of hypothesized order relations. Educational and Psychological Measurement, 57, 164-168.

77.

Tracey

T. J. G.

(2000). Analysis of circumplex models. In Tinsley

H. E. A.

Brown

(Eds.), Handbook of applied multivariate statistics and mathematical modeling (pp. 641-664). San Diego, CA: Academic Press.

78.

Tracey

T. J. G.

Rounds

J. B.

Gurtman

M. B.

(1996). Examination of the general factor with the interpersonal circumplex structure: Application to the Inventory of Interpersonal Problems. Multivariate Behavior Research, 31, 441-466.

79.

Turan

Guo

Boggiano

M. M.

Bedgood

(2014). Dominant, cold, avoidant, and lonely: Basal testosterone as a biological marker for an interpersonal style. Journal of Research in Personality, 50, 84-89.

80.

Tyrer

Reed

G. M.

Crawford

M. J.

(2015). Classification, assessment, prevalence, and effect of personality disorder. Lancet, 385, 717-726.

81.

Westen

Rosenthal

(2003). Quantifying construct validity: Two simple measures. Journal of Personality and Social Psychology, 84, 608-618.

82.

Wiggins

J. S.

(1995). Interpersonal adjective scales: Professional manual. Odessa, FL: Psychological Assessment Resources.

83.

Wiggins

J. S.

(2003). Paradigms of personality assessment. New York, NY: Guilford Press.

84.

Wiggins

J. S.

Broughton

(1991). A geometric taxonomy of personality scales. European Journal of Personality, 5, 343-365.

85.

Wiggins

J. S.

Steiger

J. H.

Gaelick

(1981). Evaluating circumplexity in personality data. Multivariate Behavioral Research, 16, 263-289.

86.

Wilkinson

, Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604.

87.

Williams

T. F.

Simms

L. J.

(2016). Personality disorder models and their coverage of interpersonal problems. Personality Disorders, 7, 15-27. doi:10.1037/per0000140

88.

Wright

A. G. C.

Pincus

A. L.

Conroy

D. E.

Hilsenroth

M. J.

(2009). Integrating methods to optimize circumplex description and comparison of groups. Journal of Personality Assessment, 91, 311-322.

89.

Wright

A. G. C.

Pincus

A. L.

Hopwood

C. J.

Thomas

K. M.

Markon

K. E.

Krueger

R. F.

(2012). An interpersonal analysis of pathological personality traits in DSM-5. Assessment, 19, 263-275.

90.

Wright

A. G. C.

Pincus

A. L.

Thomas

K. M.

Hopwood

C. J.

Markon

K. E.

Krueger

R. F.

(2013). Conceptions of narcissism and the DSM-5 pathological personality traits. Assessment, 20, 339-352.

91.

Zhang

Preacher

K. J.

Luo

(2010). Bootstrap confidence intervals for ordinary least squares factor loadings and correlations in exploratory factor analysis. Multivariate Behavioral Research, 45, 104-134.

92.

Zimmermann

Wolf

Bock

Peham

Benecke

(2013). The way we refer to ourselves reflects how we relate to others: Associations between first-person pronoun use and interpersonal problems. Journal of Research in Personality, 47, 218-225.

93.

Zimmermann

Wright

A. G. C.

Zanger

(2015). Clarifying the relation between the vector method and the structural summary method: An algebraic note on interpersonal construct validation. Manuscript in preparation.

94.

Zuroff

D. C.

Fournier

M. A.

Patall

E. A.

Leybman

M. J.

(2010). Steps toward an evolutionary personality psychology: Individual differences in the social rank domain. Canadian Psychology, 51, 58-66.