Developing Automated Feedback on Spoken Performance : Exploring the Functioning of Five Analytic Rating Scales Using Many-facet Rasch Measurement

2022

In this study, we used the Many-facet Rasch measurement (MFRM) to explore the quality of ratings as well as the functioning of five analytic rating scales developed for automated assessment of L2 speech. This study is part of a multidisciplinary research project that develops automatic speech recognition (ASR), automated scoring and automated feedback for L2 Finnish and Swedish. The data include the analytic ratings (task completion, fluency, pronunciation, range, accuracy) gathered from human raters (n=14) who assessed L2 Finnish learners’ (n=64) speech samples using Moodle. The four-facet Rasch analysis showed that the raters performed and the rating scales functioned well, although task …