Liberal and Conservative Differential Item Functioning Detection Using Mantel-Haenszel and SIBTEST: Implications for Type I and Type II Error Rates

2004

The aim of this work was to determine, in terms of Type I and Type II error rates, the risks of applying various statistical procedures for evaluating differential item functioning. To this end, the authors carried out a simulation study in which the Mantel-Haenszel and SIBTEST procedures were applied in conjunction. The variables manipulated were sample size and distribution of ability between groups. Results indicated that, although there was a high rate of agreement between the procedures, the joint Type I and Type II error rate may vary substantially from that obtained when each of the procedures was applied separately. Furthermore, the authors analyzed empirical data to obtain informat…