Measuring the outcomes of a literacy programme in no-fee schools in Cape Town

Abstract

Background: Most funders require non-governmental organisations to evaluate the effectiveness of their programmes. However, in our experience, funders seldom fund evaluation endeavours and organisational staff often lack evaluation skills.Aim: In this outcome evaluation of Living through Learning’s (LTL) class-based English-medium Coronation Reading Adventure Room programme, we addressed two evaluation questions: whether Grade 1 learners who participated in the programme attained LTL’s and the Department of Basic Education’s (DBE) literacy standards at the end of the programme and whether teacher attributes contributed to this improvement.Setting: The evaluation was conducted in 18 different no-fee schools in Cape Town. Participants comprised 1090 Grade 1 learners and 54 teachers.Methods: We used Level 2 (programme design and theory) and part of Level 4 (outcome) of an evaluation hierarchy to assess the effectiveness of the programme.Results: Evaluation results showed that most schools, except three, attained the 60% performance standard set by the LTL on all quarterly assessments. Most schools, except two, attained the 50% performance standard of the DBE for English first language on all quarterly assessments. We also found that in terms of teacher attributes, only teacher experience in literacy teaching was significant in predicting learner performance in literacy in the first term of school.Conclusion: We explain why our results should be interpreted with caution and make recommendations for future evaluations in terms of design, data collection and levels of evaluation.