Abstract
The recent increase
in the use of constructed-response items in educational assessment
and the dissatisfaction with the nature of the decision that the
judges must make using traditional standard-setting methods created
a need to develop new and effective standardsetting procedures
for tests that include both multiple-choice and constructed-response
items. The Body of Work (BoW) method is an examinee-centered method
for setting cut-scores that applies a holistic approach to student
work in order to estimate the cut-scores that differentiate examinees
according to their level of performance in situations where both
item formats are used. A detailed review of Version 1 and the
recent modification, Version 2, are first presented followed by
a critical evaluation of the two versions in terms of Berk's (1986)
10 criteria for defensibility. The results reveal that the BoW
method appears to be a promising method for setting cut-scores
that could be used on a wider scale in Canada. However, as with
other methods, the experience gained from using the BoW method
in the field will probably lead to further modifications in an
attempt to increase efficiency without sacrificing accuracy.
Copyright © AJER, the Faculty of Education, and the University
of Alberta, 2006.
Last revised: May 16, 2006.
Designed by G.H. Buck