Linear Models for Optimal Test Design

Over my nearly forty years of teaching and conducting research in the ?eld of psychometric methods, I have seen a number of major technical advances that respond to pressing educational and psychological measu- mentproblems. Thedevelopmentofcriterion-referencedassessmentwasthe ?rst, beginning in the late 1960s with the important work of Robert Glaser and Jim Popham, in response to the need for assessments that considered candidate performance in relation to a well-de?ned body of knowledge and skills rather than in relation to a norm group. The development of criterion-referenced testing methodology with a focus on decision-theoretic concepts and methods, content validity, standard-setting, and the recog- tionofthemeritsofbothcriterion-norm-referencedandcriterion-referenced assessments has tremendously in?uenced current test theory and testing . The second major advance was the introduction of item response-theory (IRT) and associated models and their applications to replace classical test theory (CTT) and related practices. Beginning slowly in the 1940s and 1950s with the pioneering work of Frederic Lord, Allan Birnbaum, and GeorgRasch,bythe1970sthemeasurementjournalswerefullofimportant research studies describing new IRT models, technical advances in model parameter estimation and model ?t, and research on applications of IRT models to equating, test development, the detection of potentially biased test items, and adaptive testing. The overall goal has been to improve and expand measurement practices by overcoming several shortcomings of cl- sicaltesttheory:dependenceoftest-itemstatisticsandreliabilityestimates on examinee samples, dependence of examinee true score estimates on the particular choices of test items, and the limitation in CTT of modeling ex- viii Foreword aminee performance at the test level rather than at the item level.

Over my nearly forty years of teaching and conducting research in the ?eld of psychometric methods, I have seen a number of major technical advances that respond to pressing educational and psychological measu- mentproblems. Thedevelopmentofcriterion-referencedassessmentwasthe ?rst, beginning in the late 1960s with the important work of Robert Glaser and Jim Popham, in response to the need for assessments that considered candidate performance in relation to a well-de?ned body of knowledge and skills rather than in relation to a norm group. The development of criterion-referenced testing methodology with a focus on decision-theoretic concepts and methods, content validity, standard-setting, and the recog- tionofthemeritsofbothcriterion-norm-referencedandcriterion-referenced assessments has tremendously in?uenced current test theory and testing . The second major advance was the introduction of item response-theory (IRT) and associated models and their applications to replace classical test theory (CTT) and related practices. Beginning slowly in the 1940s and 1950s with the pioneering work of Frederic Lord, Allan Birnbaum, and GeorgRasch,bythe1970sthemeasurementjournalswerefullofimportant research studies describing new IRT models, technical advances in model parameter estimation and model ?t, and research on applications of IRT models to equating, test development, the detection of potentially biased test items, and adaptive testing. The overall goal has been to improve and expand measurement practices by overcoming several shortcomings of cl- sicaltesttheory:dependenceoftest-itemstatisticsandreliabilityestimates on examinee samples, dependence of examinee true score estimates on the particular choices of test items, and the limitation in CTT of modeling ex-viii Foreword aminee performance at the test level rather than at the item level.

Over my nearly forty years of teaching and conducting research in the ?eld of psychometric methods, I have seen a number of major technical advances that respond to pressing educational and psychological measu- mentproblems. Thedevelopmentofcriterion-referencedassessmentwasthe ?rst, beginning in the late 1960s with the important work of Robert Glaser and Jim Popham, in response to the need for assessments that considered candidate performance in relation to a well-de?ned body of knowledge and skills rather than in relation to a norm group. The development of criterion-referenced testing methodology with a focus on decision-theoretic concepts and methods, content validity, standard-setting, and the recog- tionofthemeritsofbothcriterion-norm-referencedandcriterion-referenced assessments has tremendously in?uenced current test theory and testing . The second major advance was the introduction of item response-theory (IRT) and associated models and their applications to replace classical test theory (CTT) and related practices. Beginning slowly in the 1940s and 1950s with the pioneering work of Frederic Lord, Allan Birnbaum, and GeorgRasch,bythe1970sthemeasurementjournalswerefullofimportant research studies describing new IRT models, technical advances in model parameter estimation and model ?t, and research on applications of IRT models to equating, test development, the detection of potentially biased test items, and adaptive testing. The overall goal has been to improve and expand measurement practices by overcoming several shortcomings of cl- sicaltesttheory:dependenceoftest-itemstatisticsandreliabilityestimates on examinee samples, dependence of examinee true score estimates on the particular choices of test items, and the limitation in CTT of modeling ex-viii Foreword aminee performance at the test level rather than at the item level.

Brief History of Test Theory and Design.- Formulating Test Specifications.- Modeling Test-Assembly Problems.- Solving Test-Assembly Problems.- Models for Assembling Single Tests.- Models for Assembling Multiple Tests.- Models for Assembling Tests with Item Sets.- Models for Assembling Tests Measuring Multiple Abilities.- Models for Adaptive Test Assembly.- Designing Item Pools for Programs with Fixed Tests.- Designing Item Pools for Programs with Adaptive Tests.- Epilogue.