Introduction
The Programme to Improve Learning Outcomes (PILO), works on developing methodologies that contribute to significant and sustainable improvement in the education system. After five years of providing district officials, teachers and School Management teams (SMTs) with tools, routines and data management practices to better manage curriculum implementation, PILO embarked on an intervention to embed and institutionalise these new practices. This initiative took place between 2021 and 2023 in KwaZulu-Natal (KZN). Zenex in recognising the importance of understanding the dynamics of system change, its barriers and successes, has funded an evaluation of this embedding component.
Evaluation Focus and Design
JET carried out the evaluation which focussed on the:
- Extent to which embedded outcomes were achieved at provincial, district and school levels;
- Elements that had been officially adopted;
- Enabling and constraining factors, and barriers towards non-adoption;
- PILO-embedded activities being appropriate and sufficient to achieve the embedding outcomes.
The evaluation took place in King Cetshwayo District schools. JET collected data through document analysis; analysis of PILO’s tools and monitoring data; interviews and focus groups (district officials, principals, teachers); observations of training, meetings and practices at various levels in the system, and surveys with teachers and school principals to triangulate findings from the qualitative approaches. While there were various logistical and evidential challenges affecting some data collection efforts, JET was in a position to record the findings below.
Findings
Embedding – province level:
- Outcomes were the slowest at this level with strategy plans, PILO tools and the use of province budgets being only partially met;
- The use of data and integration of PILO into key performance areas (KPAs) did not occur.
Embedding – district level:
- Greater uptake of integrated planning, an improved culture of collaboration, and the practical use of data was to some extent achieved. Capacity building and district support of schools were partially met. Subject Advisors’ support and accountability between schools and districts were not effective.
Embedding – school level:
- Mixed results with a high variation and unevenness;
- Improvement in the culture of collegiality and support, and principals and SMTs prioritised curriculum management;
- The use of the PILO Monitoring, Reporting and Responding (MRR) methodology and tools was partially implemented;
There was little evidence of the presence and quality implementation of the Professional Learning Communities (PLCs) and no evidence of teachers/department heads’ engagement in professional supportive conversations.
Elements of the PILO Programme Adopted by the System
There was evidence of greater uptake of the Project at the district and school levels than at the province level. Here, traction was only gained when elements were officially adopted by the province, for example, the training of officials and the associated resources, as evidenced in the KZNDoE’s operational plan and Annual Teaching Plans (ATPs).
Other elements showed some uptake but were uneven, such as the use of PILO processes (offering the system important routines). In this regard, the province’s failure to adopt the curriculum tracker and other monitoring tools resulted in limiting benefits. The evaluation found that activities such as professional supportive conversations and the PLC are unlikely to be adopted as these are too complex to be embedded in the system.
Recommendations
- Departmental buy-in and active participation from inception is critical and this should include:
- Explicit agreement by the Department to officially adopt the programme as an integral part of the provincial policies, strategies and plans.
- A gradual plan to shift funding responsibility from external partners to the Department throughout the project, with provincial contributions taking place from inception.
- Flexibility to allow for simpler adaptation of methodologies and tools in accordance with provincial capacity and resources;
- Support of capacity development, particularly in project management, monitoring and data management is a vital element.
Conclusion
Over the past few years, Zenex has moved increasingly into large-scale, multi-stakeholder systemic initiatives working with the government. This particular project has provided invaluable information and knowledge about what it means to embed programmes in government contexts: what constraints may arise and what the low-hanging fruits may be regarding the actual embedding process. Additionally, the Evaluation Report points out the important principles that underlie the embedding process and why it is critical to activate these principles practically from the outset of any such collaborative project. By documenting these lessons, the report provides valuable guidelines which can potentially inform Zenex’s future work in the design of systemic programmes, as well as possible incorporation into existing programmes where feasible.