Comparing Measurement Precisions of Short Forms: Measuring Self-Care Physical Function in Veterans

PURPOSE: This study compared measurement precisions of the four-item and the eight-item short forms (SFs) generated from a FIM™–Minimum Data Set (MDS) self-care physical function item bank with the original tools. We hypothesized that the assessment tools with more items would have better measurement precision, and the short forms with the same number of items would have similar measurement precision.

RATIONALE: This study aimed to provide practitioners efficient and flexible assessment tools by generating varied SFs from the item bank. Generating SFs could reduce assessment burdens for practitioners. However, SFs have been criticized for lacking measurement precision compared with the original tool, since the shorter version of the instrument may introduce more error than the original one. Therefore, the purpose of this study is to compare the measurement precision of the full item bank (26 items), full-length tests (each 13 items), and 4- and 8-item SFs.

DESIGN: This study used a national retrospective veterans dataset for secondary data analysis.

PARTICIPANTS: The mean age of the participants was 67.1 yr (standard deviation [SD] = 11.3) and most were male (n = 2,377, 96.2%), White (n = 1,576, 65.6%), married (n = 1,064, 42.5%), admitted for initial rehabilitation (n = 2,362, 94.5%), and pre–living setting was at an acute medical/surgical care unit in the same rehabilitation facility (n = 1,113, 44.5%). Participants had one of the following diagnoses: stroke (n = 1,066, 42.6%), lower-extremity amputation (n = 472, 18.9%), knee replacement (n = 568, 22.7%), and hip replacement (n = 394, 15.8%).

METHOD: We obtained a random sample of 2,500 veterans who completed both the FIM and the MDS within 6 days through October 2008 to September 2010. The average length of days between the administrations of the FIM and the MDS was 3.2 days (SD = 2.1) with a range from 0 to 6 days.

ANALYSIS: The four- and eight-item SFs were developed based on conceptually sound SF item response theory (IRT) developmental procedures. The advantage of IRT-based SFs was that item difficulties could be selected to match the range of person abilities (i.e., low, medium, and high). This study examined person strata, ceiling–floor effects, person fit statistics, test standard error plots, and 95% confidence interval of anchored person measures between the item bank and the varied SFs.

RESULTS: The six 4- and 8-item SFs (i.e., Item Bank_8, Item Bank_4, FIM_8, FIM_4, MDS_8, and MDS_4) were generated across a range of difficulty levels from the item bank composed of FIM and MDS physical self-care items. Overall, SFs with the same number of items had similar person strata and test error (e.g., person strata = 3.47, 3.37, and 3.16 for the Item Bank_8, FIM_8, and MDS_8 SFs, respectively). The three 8-item SFs all had higher correlations with the item bank (r =.82∼.95), higher person strata, and less test error than the corresponding 4-item SFs (r = .80∼.90). The three 4-item SFs did not meet the criteria of standard error less than 0.3 for any theta values. MDS_4 item SF had the largest ceiling effects (18.9%), and FIM_4 item SF had the largest floor effects (6.72%).

DISCUSSION: In general, SFs with the same number of items demonstrated similar precision. When the number of items increased, the error of the test decreased and person strata increased. A good balance between precision and efficiency appears to be 8-item SFs (FIM or MDS).

IMPACT STATEMENT: This study suggested that the practitioners could use an administration form that compromises between efficiency and precision. Furthermore, they can select a form (i.e., FIM or MDS SF) with which they are familiar.