The Simpson paradox of school grading in Italy

Abstract Data from the 2003 OECD-PISA Survey for Italy reveal a striking difference in the relationship between students’ competence (as measured by PISA score in Mathematics) and school grades across regions: a competence level granting bare sufficiency in the North yields excellence grades in the South. This has spurred a lively debate on education policy in the country, based on the inference drawn from this evidence that grading practices are excessively different in the two areas. We show in this note that this inference overlooks a Simpson paradox hidden in the data. After a more careful analysis, the above inference is seen to be wrong. The crucial omitted variable is the school-leve…

School grading and institutional contexts

We study how the relationship between students' cognitive ability and their school grades depends on institutional contexts. In a simple abstract model, we show that unless competence standards are set at above-school level or the variation of competence across schools is low, students' competence valuation will be heterogeneous, with weaker schools inflating grades or flattening their dependence on competence, therefore reducing the information content and comparability of school grades. Using data from the OECD-PISA 2003 Survey, the model is applied to a sample of four countries, namely Australia, Germany, Italy, and the Netherlands. We find that in Australia, schools' heterogeneity does …

``Grading in Heterogeneous schools''