For a long time, psychologists have been interested in creating better tests to predict job performance. Recently, there have been attempts to improve the Armed Services Vocational Aptitude Battery (ASVAB), which is used to select and classify enlisted personnel. This research investigates the ability of the Enhanced Computer-Administered Test (ECAT), a computerized battery of new measures, to increment the validity of the ASVAB in predicting three criteria of mechanics' job performance: a hands-on performance test (HOPT), a job knowledge test (JKT), and final course grade (FCG). Examinees included 698 automotive mechanics and 443 helicopter mechanics.
One spatial ability test, assembling objects (AO), was the only promising new predictor of mechanical HOPT performance among those we examined. AO added the greatest incremental validity to the ASVAB for predicting HOPT scores of both automotive (.012 or 1.6%) and helicopter mechanics (0.15 or 2.2%). When criteria other than hands-on tests were used, increments were much smaller, and they pinpointed different subtests as incrementing the ASVAB. For the prediction of automotive mechanics' JKT scores, sequential memory, a complex memory test designed to be a nonverbal measure of general intelligence, added .003 (0.4%) to the validity of ASVAB. Spatial Reasoning also had an incremental validity of .003 for predicting JKT scores. For prediction of FCG, the best new subtest added essentially nothing. These results indicate that sole reliance on academic criteria, such as JKT or FCG, can result in choice of different predictors or underestimate the incremental validity of new predictor tests against hands-on performance. The incremental validity of AO for predicting hands-on mechanical performance is likely to be between .01 and .02.