A short, generic quality assessment (QA) tool is currently being designed for organisations wishing to determine the quality of the early learning programme attended by children who have been assessed on the ELOM. This tool adds to other supplementary measures that assist in understanding possible factors that may influence ELOM scores. These include measures of the home learning environment, socio-economic status, programme dosage (hours and years of enrolment) and height for age.
The tool is for use with group learning programmes delivered directly to children aged 3 -5 years. It is not suitable for use in programmes where the parent mediates the early learning experience. It focuses only on direct implementation of the learning programme/curriculum activities and not on other important aspects of service quality such as compliance with local regulations, parent involvement, nutrition, support and referrals provided to children, and overall programme leadership and management. Data on structural dimensions of quality such as teacher education and qualifications, physical infrastructure, group sizes and ratio should also be collected, but this tool focuses on the observation of process dimensions.
The small set of chosen indicators are:
- Easily observable (direct observation or documentary records) and with differentiated levels for scoring
- Strongly associated with overall quality scores and child outcomes related to readiness to benefit from Grade R
- Aligned with the SA National Curriculum Framework aims – towards Grade R competencies and Programme sections of the Children’s Act ECD Norms and Standards.
The QA tool requires direct observation in the classroom (including reference to records) for a minimum of two hours at a time of day that free choice or small group activities (indoor playtime) and at least one large group activity (morning ring, story or music time) can be observed. This is in line with all the well-known quality tools (CLASS, ISSA tool, ECERS suite of measures, MELE, and TIPPS). See here for a summary of the focus areas and scale used for well-known quality measures.
Assessors using this QA tool will require practical training to ensure correct use, reduce subjective interpretation and ensure a high inter-rater reliability (different assessors should as far as possible, score the same behaviour in the same way). Training will involve provision of examples and practical application of the tool in preschool classrooms.
The proposed indicators are informed by a review of the literature and draw on the key classroom focus areas of established observational measures which will support construct validity. Once it has been administered in the context of the 2020 South African Early Years Index Study, psychometry will be undertaken. Prior to its use in this study, inter-rater reliability was established.
Key dimensions of quality: what the literature tells us
While definitions of high quality ECCE vary somewhat according to context, including resource availability and cultural beliefs about teaching and learning, there is general agreement that the physical and psychological environment, curriculum, learning and teaching approaches, teacher child interactions, programme management and community integration are all important in defining quality.
The accumulated evidence of numerous quality studies indicates that the following classroom inputs are associated with good development outcomes:
- Implementation of a holistic age appropriate curriculum. To promote school readiness there should be a targeted focus on specific school readiness skills (early mathematics and literacy) with clear learning goals, rather than a general whole child curriculum which includes these skills. Effective learning activities should be cumulative and sequenced to align with children’s developmental stages (Center on the Developing Child, 2016; Phillips et al., 2017; UNESCO, 2017).
- Presence of varied learning materials in the classroom (Aboud, 2006; Montie, Xiang & Schweinhart, 2006; Trawick-Smith et al, 2015; UNESCO, 2017).
- Opportunities for child-initiated activities alone and with peers, as well as adult-led group individual and small and whole group activities (Jenkins & Duncan, 2017; Phillips et al, 2017; Montie et al. 2006; Sylva et al., 2007).
- Sensitive, mediated practitioner/child interaction targeted to the developmental level and needs of individual children (AKF, 2010; Sylva et al., 2007; UNESCO, 2017), and designed to address areas that need strengthening. This requires careful planning and assessment of individual abilities (Grisham-Brown, Hallam & Brookshire, 2006).
- Play promotes learning and development, and a continuum of different types of play provide for this, from play that is freely chosen by children, through adult-guided play where adults scaffold child-led play, to adult-structured activities where the teacher designs, sets rules and scaffolds play with a particular learning objective (Edwards and Cutter- Mackenzie, 2013; Jensen, Pyle, Zosh, Ebrahim et al., 2019; Pyle and Danniels, 2017; Zosh et al, 2017, 2019). Highly teacher-controlled, direct instruction methods, such as large group worksheet-based academic activities should be avoided as they have been linked with stress and reduced motivation in preschool children (Elkind, 1986; Stipek et al., 1995).
- Rich language and literacy experiences as the basis for learning and later reading (Lonigan et al., 2000; Opel et al 2009; Storch & Whitehurst, 2002)
- Warm, supportive and encouraging relationships with programme staff (Aga Khan Foundation, 2010; Hamre. & Pianta, 2005) will facilitate learning as well as the development of social and emotional skills associated with successful school transition (Shala, 2013). This is a key aspect of CLASS, TIPPS, and the Arnett Child Caregiver Interaction Scale (Arnett, 1989) often used with the ECERS.
It is notable that at least one large study of children in public pre-K programmes (Sabol, Hong, Pianta & Burchinal, 2013) found that summary Quality Rating Information System were not significant predictors of children’s learning. However, indicator 4 above (teacher–child interactions) as measured using CLASS scores had the most powerful effect on learning outcomes (twice as powerful a predictor as variables included in the QRIS ratings).
Choice of focus areas was also informed by recent reports on the Biersteker adaptation of ECERS which distinguished the items on which high scoring practitioners most differed from low scoring practitioners. These included availability of inside equipment (including blocks, educational toys and games), good efficient routines and appropriate daily programme, music and movement /gross motor activities, planning, observing and assessing children regularly, and talking with children about their activities. There was little difference on maths and language/story activities in these samples probably because the items were insufficiently sensitive to differences.
Short-listed focus areas
Following the literature, well known measures of quality and local experience, a short list of indicators was drafted to address the following focus areas:
- Presence in the learning environment of a variety of activities supported by materials and books manipulatives, books and fine motor materials
- Implementation of an age appropriate curriculum (including a targeted focus on skills needed for successful school transition – maths, language and social/emotional competencies).
- A play-based pedagogy including free play, teacher supported free play, guided play and adult planned playful activities (curriculum / teaching strategies).
- Planning of scheduled and ongoing assessment to inform programme planning and individualised support for children (intentional teaching).
- Teaching strategies (adult-child interactions) that scaffold learning (language interactions with children).
- Social and emotionally supportive relationships (including fostering of independence and self-regulation).
In addition, early learning programmes will be asked to furnish information on practitioner qualifications, experience, support and programme dosage, attendance and duration of enrolment. This is all contained in other ELOM support tools.
Format of the tool
The ELOM Learning Programme Quality Assessment Tool is designed as a short and global measure, so it does not provide detail about all the equipment and activities offered but focuses more generally on the variety of activities and material and teaching strategies. Maths and Language as key predictors for Grade R are the exception to this.
Items are rated on a three point scale (inadequate, basic, good) to provide some range of scores, but also to take into account that more nuanced and extended scales require experienced and ECD-qualified assessors. This simple format with specific examples for each category per indicator will be tested, and is preferable to a rating with both ends specified which gives much more opportunity for subjectivity in rating.
Explanatory notes have been added to indicators. A ‘not observed’ category has been added for those items that may not be seen during the limited period of observation (in this case the score will be pro-rated).
Aboud, F. E. (2006) Evaluation of an early childhood preschool programme in rural Bangladesh. Early Childhood Research Quarterly 2, 46 – 60.
Aga Khan Foundation (AKF). (2010) Improving learning achievement in early primary in low-income countries: A review of the research. Geneva, Switzerland.
Arnett, J. (1989). Caregiver interaction scale. Princeton, NJ: Educational Testing Service.
Association for Childhood Education International (ACEI). 2011. ACEI global guidelines assessment (GGA), 3rd edition. Washington, DC: Author.
Biersteker,L., Dawes, A., Hendricks, L., & Tredoux, C. (2016). Predictors of center-based early childhood care and education program quality: A South African study. Early Childhood Research Quarterly, 36, 334 – 344.
Britto, P., Yoshikawa, H. & Boller, K. (2010) Quality of early childhood development programs in global contexts. Rationale for investment, conceptual framework and implications for equity. Social Policy Report, 25 ( 2).
Burchinal,M., Zaslow,M. & Tarullo, L. (2016). (Editors).Quality thresholds, features, and dosage in early care and education: Secondary data analyses of child outcomes. Monograph: Society for Research in Child Development.
Center on the Developing Child at Harvard University. (2016). From best practices to breakthrough impacts: A science-based approach to building a more promising future for young children and families. Retrieved from www.developingchild.harvard.edu.
Cutter Mackenzie, A. & Edwards, S. (2013) Toward a model for early childhood environmental education: Foregrounding, developing, and connecting knowledge through play-based learning. The Journal of Environmental Education, 44:3, 195-213,
Early, D. M., Maxwell, K. L., Burchinal, M., Alva, S., Bender, R. H., & Bryant, D. (2007). Teachers’ education, classroom quality, and young children’s academic skills: results from seven studies of pre-school programs. Child Development, 78(2), 558–580.
Elkind, D. (1986). Formal education and early childhood education: An essential difference. The Phi Delta Kappan 67, no. 9 (1986):631-636. Cited in Jenkins & Duncan (2017).
Edwards, S. & Cutter Mackenzie, A. (2013). Pedagogical play types: what do they suggest for learning about sustainability in early childhood education? International Journal of Early Childhood, 45, 327 – 346.
Grisham-Brown, J., Hallam, R. & Brookshire, R. (2006) Using authentic assessment to evidence children’s progress toward early learning standards. Early childhood Education Journal, 34(1), 45-47.
Hamre, B. & Pianta, R.C. ( 2005) Can instructional and emotional support in the first‐grade classroom make a difference for children at risk of school failure? Child Development 76 (5) 949-967.
Harms, T., Clifford, R. M., & Cryer, D. (2015). Early childhood environment rating scale ( ECERS), 3rd edition. New York: Teacher’s College Press.
High Scope (n.d.). The preschool programme quality assessment (PQA). Ypsilanti, Michigan: High Scope.
International Step by Step Association (ISSA) (n.d.). Competent educators of the 21st century: Principles of quality pedagogy. Leiden, Netherlands.
Jenkins, J. M. & Duncan, G.R. (2017). Do pre-kindergarten curricula matter? The current state of scientific knowledge on pre-kindergarten effects. In Phillips, D., Lipsey, M., Dodge, et al.. Washington DC: Brookings Institute p 37 – 44
Jensen, H., Pyle, A., Zosh, J. M., Ebrahim, H. B.,Scherman, A. Z., Reunamo, J., & Hamre, B. K. (2019).Play facilitation: the science behind the art of engaging young children (white paper). Denmark: The LEGO Foundation
Lieber, J., Butera, G. Hanson, M., Palmer, S., Horn, E., Czaja, C. et al. (2009). Factors that influence the implementation of a new pre-school curriculum: Implications for professional development. Early Education and Development 20, (3), 456-481.
Lonigan, C. J., Burgess, S. R., & Anthony, J. L. (2000). Development of emergent literacy and early reading skills in preschool children: evidence from a latent-variable longitudinal study. Developmental Psychology, 36(5), 596.
Montie, J. E., Xiang, Z., & Schweinhart, L. J. (2006). Preschool experience in 10 countries: Cognitive and language performance at age 7. Early Childhood Research Quarterly, 21(3), 313-331.
National Association for the Education of Young Children. (2018). Early learning accreditation standards and assessment items. Washington DC: NAEYC.
OECD. (2007). Starting Strong II: Early childhood education and care. OECD, Paris.
Opel, A., Ameer, S. S., & Aboud, F. E. (2009). The effect of preschool dialogic reading on vocabulary among rural Bangladeshi children. International Journal of Educational Research, 48(1), 12-20.
Opel, A., Zaman, S. S., Khanom, F., & Aboud, F. E. (2012). Evaluation of a mathematics program for preprimary children in rural Bangladesh. International Journal of Educational Development, 32(1), 104-110.
Phillips, D., Lipsey, M., Dodge, K., Haskings, R., Bassok, D. et al. (2017). Puzzling It Out: The current state of scientific knowledge on pre-kindergarten effects- A consensus statement. Washington DC: Brookings Institute.
Pianta, R. C., La Paro, K. M., & Hamre, B. K. (2008). Classroom assessment scoring system: manual pre-K. Education Review// Reseñas Educativas
Pyle, A. & Danniels, E. (2017). A continuum of play-based learning: The role of the teacher in play-based pedagogy and the fear of hijacking play. Early Education and Development 28, 3, 274–289.
Rao, N., Sun, J., Wong, J.M.S., Weekes, B., Ip, P., Shaeffer, S., Young, M., Bray, M., Chen, E. & Lee, D. (2014). Early childhood development and cognitive development in developing countries: a rigorous literature review. London: Department for International Development.
Sabol, T. J., Hong, S.L., Pianta, R.C.& Burchinal, M. R. (2013) Can Rating Pre-K Programs Predict Children’s Learning? Science, 341, issue 6148, pp 845 – 846,
Schweinhart, L.J., Barnes, H.V. & Weikart, D.P. (2005). Significant benefits: The High/Scope Perry pre-school study through age 27. In Frost, N. (Ed)Child welfare: major themes in health and social welfare, 9–29. Abingdon, UK: Routledge.
Seidman, E., Raza, M., Kim, S., & McCoy, J. M. (2014). Teacher instructional practices and processes system–TIPPS: Manual and scoring system. New York: New York University.
Shala, M. (2013). The impact of preschool social-emotional development on academic success of elementary school students Psychology (4).
Stipek D., Feiler, R. Daniels, D. & Milburn S. (1995), Effects of different instructional approaches on young children’s achievement and motivation. Child Development 66 (1), 209-223.
Storch, S. A., & Whitehurst, G. J. (2002). Oral language and code-related precursors to reading: Evidence from a longitudinal structural model. Developmental psychology, 38(6), 934.
Sylva, K., Taggart, B., Siraj-Blatchford, I., Totsika, V., Ereky-Steven, K., Gilden, R., et al. (2007). Curricular quality and day-to-day learning activities in pre-school. International Journal of Early Years Education, 15(1), 49–65.
Trawick-Smith, J., Wolff, J., Koschel, M. et al. (2015) Effects of toys on the play quality of preschool children: Influence of gender, ethnicity, and socioeconomic status. Early Childhood Education Journal 43: 249.
UNESCO (2007). Good governance of early childhood care and education: Lessons from the 2007 EFA global monitoring report. UNESCO Policy Briefs on Early Childhood No 40. Paris UNESCO
UNESCO (2017). Overview MELQO (Measuring early learning quality and outcomes). Paris: UNESCO.
UNICEF & DBE. (2019). National early childhood development monitoring and evaluation framework: South Africa. Pretoria: Khulisa Management Services.
Zosh, J. M.; Hirsh-Pasek, J., Hopkins, E. J., Jensen, H., Liu,C., Neale, C. , Solis, L. & Whitebread, D. (2018). Accessing the inaccessible: Redefining play as a spectrum. Frontiers in Psychology, 9, Article 1124.
 While health and safety is a critical element of a good ECD service, local audits have not found a significant association of this with learning programme quality.
 QRIS ratings covering a range of areas such as parent participation, health and safety, facilities, nutrition, staffing, teaching strategies, classroom environment and curriculum.