Applied Psychological Measurement

A simulation study was conducted to evaluate the dependability of Stout's T index of unidimensionality as used in his DIMTEST procedure. DIMTEST was found to dependably provide indications of unidimensionality, to be reasonably robust, and to allow for a practical demar cation between one and many dimensions. The proce dure was not affected by the method used to identify the initial subset of unidimensional items. It was, how ever, found to be sensitive to whether the multidimen sional data arose from a compensatory model or a partially compensatory model. DIMTEST failed when the matrix of tetrachoric correlations was non-Gramian and hence is not appropriate in such cases.

Type I error rates for the likelihood ratio test for de tecting differential item functioning (DIF) were investi gated using monte carlo simulations. Two- and three-parameter item response theory (IRT) models were used to generate 100 datasets of a 50-item test for samples of 250 and 1,000 simulated examinees for each IRT model. Item parameters were estimated by marginal maximum likelihood for three IRT models: the three-parameter model, the three-parameter model with a fixed guessing parameter, and the two-parameter model. All DIF comparisons were simulated by ran domly pairing two samples from each sample size and IRT model condition so that, for each sample size and IRT model condition, there were 50 pairs of reference and focal groups. Type I error rates for the two-param eter model were within theoretically expected values at each of the α levels considered. Type I error rates for the three-parameter and three-parameter model with a fixed guessing parameter, however, were different from the theoretically expected values at the α levels considered.

A quadratic curve test equating method for equating different test forms under a random-groups data collec tion design is proposed. This new method extends the linear equating method by adding a quadratic term to the linear function and equating the first three central moments (mean, standard deviation, and skewness) of the test forms. Procedures for implementing the method and related issues are described and discussed. The quadratic curve method was evaluated using real test data and simulated data in terms of model fit and equating error, and was compared to linear equating, and unsmoothed and smoothed equipercentile equat ing. It was found that the quadratic curve method fit most of the real test data examined and that when the model fit the population, this method could perform at least as well as, or often even better than, the other equating methods studied.

Using the characteristic curve method for dichoto mously scored test items, the sampling distributions of equating coefficients were examined. Simulated data for broad-range and screening tests were analyzed using three equating contexts and three anchor-item configurations in horizontal and vertical equating situ ations. The results indicated that the sampling distribu tions were bell-shaped and their standard deviations were uniformly small. There were few differences in the forms of the distributions of the obtained equating coefficients as a function of the anchor-item configura tions or type of test. For the equating contexts studied, the sampling distributions of the equating coefficients appear to have acceptable characteristics, suggesting confidence in the values obtained by the characteristic curve method.

It is widely believed that measures of gain, growth, or change, expressed as simple differences between pretest and posttest scores, are inherently unreliable. It is also believed that gain scores lack predictive validity with respect to other criteria. However, these conclu sions are based on misleading assumptions about the values of parameters in familiar equations in classical test theory. The present paper examines modified equa tions for the validity and reliability of difference scores that describe applied testing situations more realisti cally and reveal that simple gain scores can be more useful in research than commonly believed.

Research has demonstrated that people can and often do consciously manipulate scores on personality tests. Test constructors have responded by using social desir ability and lying scales in order to identify dishonest re spondents. Unfortunately, these approaches have had limited success. This study evaluated the use of appropri ateness measurement for identifying dishonest respon dents. A dataset was analyzed in which respondents were instructed either to answer honestly or to fake good. The item response theory approach classified a higher number of faking respondents at low rates of misclassification of honest respondents (false positives) than did a social de sirability scale. At higher false positive rates, the social desirability approach did slightly better. Implications for operational testing and suggestions for further research are provided.

The network-flow algorithm (NFA) of Armstrong, Jones, & Wu (1992) and the average growth approxi mation algorithm (AGAA) of Luecht & Hirsch (1992) were evaluated as methods for automated test assem bly. The algorithms were used on ACT and ASVAB item banks, with and without error in the item parameters. Both algorithms matched a target test information function on the ACT item bank, both before and after error was introduced. The NFA matched the target on the ASVAB item bank; however, the AGAA did not, even without error in this item bank. The AGAA is a noncorrecting algorithm, and it made poor item selec tions early in the search process when using the ASVAB item bank. The NFA corrects for nonoptimal choices with a simplex search. The results indicate that reason able error in item parameters is not harmful for test as sembly using the NFA or AGAA on certain types of item banks.