Program Applicants as a Comparison Group in Evaluating Training Programs

The authors begin with a thorough assessment of the many nonexperimental employment and training program evaluation techniques based on non-random comparison groups. These techniques typically use econometric methods to estimate the effects of employment and training programs by using comparison groups from non-program "e;external"e; sources. Then, recognizing the inherent drawbacks in these methods, Bell, Orr, Blomquist and Cain respond by reintroducing an evaluation method first implemented in the 1960s, the use of "e;internal"e; comparison groups consisting of nonparticipating program applicants. These groups include withdrawals, screen-outs and no-shows of the programs being evaluated in order to solve the selection bias problem. By applying to the program, say the authors, nonparticipating applicants reveal themselves to have some of the same difficult-to-measure, personal characteristics that inspire participants to seek help in response to their current economic situation. The methodology of this technique is updated, then tested against the random experimental findings derived from a controlled job training experiment, the AFDC Homemaker-Home Health Aide Demonstrations. Encouraging results are presented along with useful suggestions for designers and implementers of all types of program evaluations.

The authors begin with a thorough assessment of the many nonexperimental employment and training program evaluation techniques based on non-random comparison groups. These techniques typically use econometric methods to estimate the effects of employment and training programs by using comparison groups from non-program "e;external"e; sources. Then, recognizing the inherent drawbacks in these methods, Bell, Orr, Blomquist and Cain respond by reintroducing an evaluation method first implemented in the 1960s, the use of "e;internal"e; comparison groups consisting of nonparticipating program applicants. These groups include withdrawals, screen-outs and no-shows of the programs being evaluated in order to solve the selection bias problem. By applying to the program, say the authors, nonparticipating applicants reveal themselves to have some of the same difficult-to-measure, personal characteristics that inspire participants to seek help in response to their current economic situation. The methodology of this technique is updated, then tested against the random experimental findings derived from a controlled job training experiment, the AFDC Homemaker-Home Health Aide Demonstrations. Encouraging results are presented along with useful suggestions for designers and implementers of all types of program evaluations.