Intervention in teachers' differential scoring judgments in assessing L2 writing through communities of assessment practice

Abstract

In writing assessment, the inconsistency of teachers' scorings is among the frequently reported concerns regarding the validity and the reliability of assessment. The study aimed to find out to what extent participating in a community of assessment practice (CAP) can impact the discrepancies among raters' scorings. Adopting one group pretest-posttest design, patterns in the teachers' scoring judgments were explored based on both quantitative and qualitative data. The results indicate significant increase in the degrees of agreement in the teachers' differential scorings showing changes in their severity tendencies for structural variety, lexical accuracy, organization and mechanics criteria while their scoring judgements on structural accuracy, task achievement, and lexical variety criteria had low levels of agreement.