Determination of Differential Bundle Functioning (DBF) of Numeracy and Literacy Tests Administered to Grade 3 Learners in South Africa

Abstract

The study is based on data gathered for the Assessment Modelling Initiative (AMI), part of the District Development Support Programme (DDSP) project conducted by the Human Sciences Research Council (HSRC) in 2002, in conjunction with the Department of Education (DoE) and provincial departments of education of the Eastern Cape, KwaZulu-Natal, Northern Cape, and Limpopo. The project was funded by the United States Agency for International Development (USAID) and managed by Research Triangle Institute (RTI) International. The Joint Education Trust (JET) developed the Grade 3 numeracy test and adapted the literacy test from the International Association for the Evaluation of Educational Achievement (IEA) survey. The aim of AMI was to develop assessment materials in literacy and numeracy for learners in Grade 3 that could be used by teachers in classroom instruction and for learner assessment. The purpose of the analysis was to investigate Differential Item Functioning (DIF) and Differential Bundle Functioning (DBF) in these tests and to determine item difficulty, discrimination, reliability, and evidence of construct validity. Participants were 4,000 male and 4,000 female Grade 3 learners selected using the stratified random sampling method from 13,425 Grade 3 learners who participated in the DDSP project. The Simultaneous Item Bias Test (SIBTEST) was used for detecting DIF and bias. SIBTEST was also used to detect bundle bias or DBF. There were significant gender differences in mean scores of both numeracy and literacy tests. Female participants performed consistently better than male participants in both tests. Both the numeracy and the literacy tests possessed appropriate psychometric properties of difficulty, discrimination, reliability, and some evidence of construct validity. Significant estimates of beta (test for DIF and DBF) were obtained for literacy but not for numeracy. The literacy test exhibited DIF at item level and DBF at bundle level. Although a myriad of factors could have caused DIF and DBF, the use of tests translated into various languages may have introduced bias. Further analysis should be done to identify other factors that could have caused DIF and DBF.