Validating the Accuracy of the Demands and Abilities Transforming Algorithm (DATA) for Systematic Job Matching

PURPOSE: “Job matching is the collaborative, data-based decision-making process used by transition teams to determine the best fit between an individual’s abilities and preferences and the job’s environmental and occupational demands” (Case-Smith, Cleary, & Persch, 2015, para. 1). Job matching practices within the American special education and vocational rehabilitation systems have been found to be collaborative and multidimensional, yet highly variable (Persch et al., 2015). Systematic Job Matching was created to match the worker’s abilities to the job demands, increasing the likelihood that the worker would be successful at that particular job (Persch, 2014). Typically, a professional evaluates each combination of abilities and demands to choose a job that will best fit the individual. Systematic Job Matching uses the Demands and Abilities Transforming Algorithm (DATA) to make evaluative judgments based on the abilities of the worker and the job demands. This process is meant to make the job matching process more accurate, efficient, and consistent. A previous study was completed that showed initial indications that the algorithm was accurate (Persch & Cleary, 2016). The purpose of this study was to further test the accuracy of the DATA to confirm that the algorithm can inform job matching decisions for professionals. The continuing development of Systematic Job Matching should help individuals with developmental disabilities be placed into jobs that fit them well.

DESIGN AND METHOD: The study used a clinical simulation design. The participants were professionals who worked with individuals with developmental disabilities and were involved in the job matching process. Data were collected from written surveys. Participants were presented with simulated decision-making scenarios. Each participant was presented with single data points from the Vocational Fit Assessment (VFA)–Job and VFA–Worker. Based on these data alone, the participants were asked whether they supported or opposed the job match. These responses were then compared with the algorithm. If the algorithm matched the response, the trial was considered a success. If the algorithm did not match the response, the trial was considered a failure.

RESULTS: The accuracy of the algorithm compared to the professional responses was 79.8%. The sensitivity of the algorithm was 89.0% and the specificity was 66.2%. The false negative rate was 11.0% and the false positive rate was 33.8%.

CONCLUSION: These results are similar to our previous work (Persch & Cleary, 2016) and affirm the accuracy of the DATA at approximately 80%. This is a substantial improvement over the inconsistent and highly variable practices used by professionals. Use of the Demands and Abilities Transforming Algorithm as part of Systematic Job Matching will increase the reliability, effectiveness, and efficiency of successful job matches for individuals with developmental disabilities.

References

Case-Smith, J., Cleary, D., & Persch, A. (2015). Current practices in job matching for individuals with intellectual and developmental disabilities. American Journal of Occupational Therapy, 69(Suppl. 1), 6911500099. https://doi.org/10.5014/ajot.2015.69S1-PO4089

Persch, A. C. (2014). The model of Systematic Job Matching. Doctoral dissertation, The Ohio State University.

Persch, A. C., & Cleary, D. S. (2016, April). A novel decision support system for systematic job matching of individuals with developmental disabilities. Poster presented at the AOTA Annual Conference & Expo, Chicago.

Persch, A. C., Cleary, D. S., Rutkowski, S., Malone, H. I., Darragh, A. R., & Case-Smith, J. D. (2015). Current practices in job matching: A Project SEARCH perspective on transition. Journal of Vocational Rehabilitation, 43, 259–273. https://doi.org/10.3233/JVR-150774