6533b851fe1ef96bd12a9630

RESEARCH PRODUCT

Integrating the Analysis of Mental Operations Into Multilevel Models to Validate an Assessment of Higher Education Students’ Competency in Business and Economics

subject

description

The Standards for Educational and Psychological Testing indicate that validation of assessments should include analyses of participants’ response processes. However, such analyses typically are conducted only to supplement quantitative field studies with qualitative data, and seldom are such data connected to quantitative data on student or item performance. This paper presents an example of how data from an analysis of mental operations collected using a sociocognitive approach can be quantitatively integrated with other data on student and item performance to validate in part an assessment of higher education students’ competency in business and economics. Evidence of forward reasoning and paraphrasing as mental operations is obtained using the think-aloud method. As part of the validity argument and to enhance credibility of the findings, the generalized linear models are expressed as multilevel models in which the analyses of response processes are aligned with quantitative findings from large-scale field studies.