Publications and Research
Document Type
Article
Publication Date
2-29-2016
Abstract
Mixed format tests (e.g., a test consisting of multiple-choice [MC] items and constructed response [CR] items) have become increasingly popular. However, the latent structure of item pools consisting of the two formats is still equivocal. Moreover, the implications of this latent structure are unclear: For example, do constructed response items tap reasoning skills that cannot be assessed with multiple choice items? This study explored the dimensionality of mixed format tests by applying bi-factor models to 10 tests of various subjects from the College Board’s Advanced Placement (AP) Program and compared the accuracy of scores based on the bi-factor analysis with scores derived from a unidimensional analysis. More importantly, this study focused on a practical and important question—classification accuracy of the overall grade on a mixed format test. Our findings revealed that the degree of multidimensionality resulting from the mixed item format varied from subject to subject, depending on the disattenuated correlation between scores from MC and CR subtests. Moreover, remarkably small decrements in classification accuracy were found for the unidimensional analysis when the disattenuated correlations exceeded 0.90.
Included in
Educational Assessment, Evaluation, and Research Commons, Higher Education Commons, Industrial and Organizational Psychology Commons
Comments
This article was originally published in Frontiers in Psychology: Quantitative Psychology and Measurement, available at https://doi.org/10.3389/fpsyg.2016.00270
This work is distributed under a Creative Commons Attribution 4.0 International License (CC BY 4.0).