By Nompumelelo Mohohlwane
Evidence-based research, monitoring and evaluation is foundational to rational decision-making when informing policies, programmes and interventions. This is imperative for public policy to have an impact on service delivery, and especially so in the education sector. The information derived from this evidence base determines the value or merit of programmes or policy options by identifying standards, performing empirical investigations using various techniques and integrating these findings into conclusions and recommendations for the sector. In the short-term, this is helpful for programme managers to improve performance and accountability. In the long-term, the knowledge generated for the sector could inform broader programme and policy design, and practices beyond the programme being evaluated.
By Professor Brahm Fleisch
Over the past, decade, there has been a resurgence of interest in evidence-informed education policies and programmes in South Africa. Specifically, there is a growing recognition that rigorous research designs, particularly designs that include ‘counterfactuals’ can provide strong findings into what works to improve learning outcomes.
Any organisation that is committed to improving the quality of evaluations understands that one of the most critical steps is getting the design right. Our own experience in M&E, particularly in designing evaluations has evolved over the years. We see evaluation design as the structure of the evaluation that will provide information/data to answer evaluation questions. The design is determined by the purpose of the evaluation, the programme theory of change, the evaluation questions and of course, the budget.
Those who work in education are acutely aware of the complexity of the problems in the South African education system. No silver bullet solution exists. Despite this, donors like the Zenex Foundation continue to implement initiatives to improve education with the expectation that evaluation should help us learn.
I am a strong proponent of M&E as a key lever in evidence-based grantmaking. It definitely adds a cost to the donor and I am sharing my perspective on four key issues to do with budgeting for, and the costs of, monitoring and evaluation (M&E):
The increase in the call for proposals for evaluation in recent years is likely an indication of growing understanding of the importance of evaluation. More specifically, there appears to be growing interest in understanding how programmes are actually implemented and the ‘impact’ of particular programmes. And whilst in theory, this bodes well for ensuring accountability and improving programming to maximise benefits, the process of responding to calls for proposals brings prevalent challenges to the surface.
Why are education evaluations becoming increasingly important?
There is no doubt that improving education is a high priority for all key stakeholders in South Africa. This is evident through the huge financial investment and numerous initiatives underway to improve education delivery and outcomes. The South African Government allocated R320 billion for education in 2017. This is approximately 6.4% of its Gross Domestic Product (GDP), which is higher than other developing countries. In addition, there has been a significant increase in private sector support for education development, estimated at R25 billion.
The Department of Basic Education (DBE) has commissioned a number of evaluations in recent years. In the process, we have learned a great deal about how to improve policy and programme design as well as about how to conduct better evaluations in the education sector. Notable evaluations include an impact evaluation of the introduction of the Grade R programme (2013), implementation evaluations the Funza Lushaka Bursary Programme (2016), the National School Nutrition Programme (2016) and the National Curriculum Statement Grade R to 12: Focusing on the Curriculum and Assessment Policy Statements (2017), as well as the Early Grade Reading Study (EGRS), all commissioned in partnership with the Department of Planning, Monitoring and Evaluation (DPME).
Rob Worthington is a Director at Kwantu, which is a Cape Town, based social enterprise that specialises in helping NGOs and Governments to use software to manage key processes in their programmes and operations. This approach makes it easier to collect, analyse and present data for management, learning and evaluation. This article is an introduction to Monitoring, Evaluation and Learning Systems, their importance, the different types that exist and some things that organisations should consider when deciding on MEL systems.
The value of monitoring is that we can better understand the context and beneficiaries in which we implement projects. We can see the project in action, identify gaps in implementation and capacity issues. Monitoring gives insight on the nuances in context and can help to manage risks in the project. These lessons can then be shared in the sector.