Organizational Research Methods

The formulae for attenuation correction in meta-analysis treat reliabilities as if they were independent of each other. The current study puts this assumption of independence to the test by empirically examining the correlation among predictor and criterion reliability estimates across studies. Interdependence of reliabilities would result in either overestimation or underestimation of population correlations depending on the direction of the relationship between the reliabilities. We conducted two studies to examine the extent to which predictor and criterion reliabilities correlate across studies. Study 1 is based on 628 pairs of reliability estimates from 518 studies published in the Academy of Management Journal and the Journal of Applied Psychology between 2004 and 2011, while Study 2 is based on 564 pairs of reliability estimates from 347 studies included in a meta-analysis on perceived organizational support (POS) and some of its antecedents and outcomes. The findings in both studies show substantial correlations between predictor and criterion reliability coefficients across studies. Our article discusses important implications from these findings for future research and for the future conduct of meta-analyses.

We aim to develop a meaningful single-source reference for management and organization scholars interested in using bibliometric methods for mapping research specialties. Such methods introduce a measure of objectivity into the evaluation of scientific literature and hold the potential to increase rigor and mitigate researcher bias in reviews of scientific literature by aggregating the opinions of multiple scholars working in the field. We introduce the bibliometric methods of citation analysis, co-citation analysis, bibliographical coupling, co-author analysis, and co-word analysis and present a workflow for conducting bibliometric studies with guidelines for researchers. We envision that bibliometric methods will complement meta-analysis and qualitative structured literature reviews as a method for reviewing and evaluating scientific literature. To demonstrate bibliometric methods, we performed a citation and co-citation analysis to map the intellectual structure of the Organizational Research Methods journal.

This article investigates in two ways the use and reporting of marker variables to detect common method variance (CMV) in organizational research. First, a review of 398 empirical articles and 41 unpublished dissertations that employ marker variables indicates that authors are not reporting adequate information regarding marker variable choice and use, are choosing inappropriate marker variables, and are possibly making errors in their assessment of CMV effects. Second, two data sets are presented that investigate the properties of six prospective markers to assess the degree to which they capture specific, measurable causes of CMV and the conclusions these markers produce when applied to substantive relationships. Results from the review and empirical investigation are used to expand the set of conditions scholars should consider when determining whether to employ a marker technique over other alternatives for detecting and controlling CMV and how best to do so.

We investigated the effects that careless responding and acquiescence response bias have on analyses conducted to assess construct dimensionality. Using job satisfaction/dissatisfaction as the focal construct, we measured and controlled for careless responding and acquiescence bias in data obtained from an online survey of employees (N = 666) from different organizations and occupational groups. We found that the negative correlation between factors defined by job satisfaction and dissatisfaction items, respectively, was attenuated by careless responding and acquiescence bias, and was not significantly different from –1.0 when both were controlled. Moreover, the correlations between the two factors and measures of other constructs were similar when careless responding and acquiescence were controlled. These findings challenge recent research purporting to demonstrate that job satisfaction and dissatisfaction are distinct constructs. Recommendations for future investigations of construct dimensionality are provided.

Studies of social networks in organizations confront analytical challenges posed by the multilevel effects of hierarchical relations between organizational subunits on the presence or absence of informal network relations among organizational members. Conventional multilevel models may be usefully adopted to control for generic forms of non-independence between tie variables defined at multiple levels of analysis. Such models, however, are unable to identify the specific multilevel dependence mechanisms generating the observed network data. This is the basic difference between multilevel analysis of networks and the analysis of multilevel networks. The aim of this article is to show how recently derived multilevel exponential random graph models (MERGMs) may be specified and estimated to address the problems posed by the analysis of multilevel networks in organizations. We illustrate our methodological proposal using data on hierarchical subordination and informal communication relations between top managers in a multiunit industrial group. We discuss the implications of our results in the broader context of current theories of organizations as connected multilevel systems.