Evaluating CPD: the challenge


Meaningful evaluation

Meaningful evaluation of CPD can be challenging. Attempting to track the impact of CPD activity through a final impact on pupil progress is not an easy process. That is perhaps one reason why schools often pay too little attention to CPD evaluation.

The research evidence about evaluation practices in relation to CPD suggests the following:

  • It rarely focuses on long-term or indirect benefits.
  • It rarely differentiates between different kinds of benefits, such as impact on moral purposes, relevance to phase of development, change, thinking or emotional intelligence.
  • It is often based on individual self-report which relates to the quality and relevance of the experience and not its outcomes.
  • It usually occurs simultaneously, after the learning experience, rather than formatively so that it can be used to enhance that experience.
  • It rarely attempts to chart benefits to the school or department (possibly because these are often not explicitly contained within the initial purpose of the CPD).

 Goodall et al, 2005

Children outdoors jumping into the air with their hands raised
Reflections:
questions & activities

Analysis of practice

Choose one of the module case studies in Section 1 to analyse. Look for evidence to address the following questions.

Questions:


  • 01.
    How has the school attempted to evaluate CPD activity? What evaluation tools or techniques has the CPD leader used to assess the effectiveness of CPD?
  • 02.
    What specific challenges or difficulties has the CPD leader encountered or recognised in evaluating CPD activity?
  • 03.
    Looking at the research evidence from Goodall et al (2005) about the common weaknesses in school CPD evaluation, which of these apply in the case study school?
Pupils looking at hamsters

What might effective evaluation look like? The next topic will consider potential models for evaluation. First, you need to consider the underlying principles for effective evaluation.

Ten key principles for effective CPD evaluation

Building on work undertaken by the TDA, school CPD leaders in South Yorkshire identified the following ten key principles for effective evaluation.


Ten key principles for effective CPD evaluation
1 Evaluation of CPD should be a collaborative process between the individual participant and key staff such as team leader and CPD leader. 6 Intended outcomes with timescales should always be identified as part of the planning stage.
2 Evaluation outcomes should inform the strategic leadership of CPD at whole school level, including the governing body. 7 Required support or changes to the organisation may be needed to allow learning to be implemented. These should be identified in the planning stage, or early in the evaluation stage.
3 The planning and evaluation of CPD should link closely to the school’s performance management process. 8 The evidence-base and the success criteria of the impact evaluation should be agreed in advance. A range of evidence should be considered, including contributions by colleagues or pupils.
4 There should be an agreed timescale for evaluating outcomes in the short-, medium- and long term, accepting that some outcomes may take longer to become evident than others. Longer-term professional development, such as Master’s level or foundation degrees for support staff, should have formative reviews at agreed stages. 9 The evaluation of impact should, where possible, include a cost-benefit analysis of the CPD.
5 Evaluation should focus on what participants learn, how they use what they have learned, and the impact on pupils and the organisation. 10 The process of evaluating the impact of CPD should itself be reviewed regularly to ensure it is effective and proportionate.

Rotherham Metropolitan Borough Council, undated


Reviewing a series of case studies of CPD practices in London schools, Earley and Porritt emphasised the importance of evaluation in assessing the overall impact of development activity. They expressed it as follows:

Child's lettered blocks spelling the word 'school'
QuoteTraditional impact evaluation tends to take place at the end of a development activity. We believe that all initial planning as to the potential impact of CPD should be undertaken before CPD activity starts. By 'impact' we mean, for example, stating specific changes in a teacher’s classroom strategies or clarifying a changed approach towards children in the playground by a lunchtime supervisor. In terms of learning outcomes, we must agree at the outset the differences in how children learn as a result of a proposed CPD activity – for example, 'pupils will move from using closed questions to the use of higher order questioning'. This is a simple concept to agree, yet requires a significant change in the CPD practice of many organisations.Quote Earley & Porritt, 2010, p7

Defining 'impact'

Explaining the definition of 'impact' in this context, Earley and Porritt put it as follows:

Impact is the difference in staff behaviours, attitudes, skills, and practice as a result of the professional development in which staff have engaged. Ultimately, impact is also the difference in the learning and experience of the children and young people as a result of the change in staff practice. Bringing about an improved outcome in the learning and experience of the children is what enables us to say that professional development of staff has been effective.

Earley & Porritt, 2010, p8


"Good evaluation does not need to be complex; what is necessary is good planning and paying attention to evaluation at the outset of the professional development program, not at the end."

Guskey, 2002, p48

Row of lockers in a corridor

The following account of practice looks at how one school's leadership development programme helped it to secure dramatic improvements.

shadow

Account of practice

Developing a formal leadership development programme

Helen Gregory, Deputy Headteacher, Esher Church of England High School, Surrey
Colleagues gathered around a computer and discussing

Introduction

Helen Gregory has been deputy head at Esher Church of England High School (EHS) for the last eight years, during which time she has been part of a leadership team that has secured a dramatic transformation. From an undersubscribed school with a local authority support plan in place to address concerns, EHS has become an 'outstanding' (Ofsted, 2009), oversubscribed, national teaching school.

The journey started with a focus on student achievement and monitoring student progress. The initial key target areas were the C/D borderline and A*–C grades at GCSE, but this quickly developed and became focused on all individuals and target groups. At the heart of this journey was a clear focus on the core belief in what Helen describes as a "relentless drive for improvement", an initial rigorous use of the school's specialist status as a performing arts college, and the recognition that "developing leadership at all levels was the gateway to school improvement".


Success in improving outcomes for pupils meant that the school was judged a high-performing specialist school in 2009 and was able to explore a second specialism. After much discussion, the decision was made to become a leadership partner school. As Helen puts it: "The penny dropped: we recognised that developing leadership at all levels was the key driver for improving standards, not only at Esher High School but also across the wider educational community."

All members of the leadership team at EHS are actively involved in delivering leadership training programmes and in coaching colleagues. There is recognition that there are many demands on the time of senior leaders in school and with all the responsibilities in existence, there is a need to remember that the outcomes for pupils are paramount. Helen notes that "the number one priority is to keep teaching and learning as outstanding." The school leadership remains determined to sustain high-quality provision. This is essential for pupils and to retain a credibility in the educational community.

EHS has developed its own leadership development model, Stepping Stones, which reflects a belief in promoting pathways at all levels. It is based on the assertion that leadership is a key element in all aspects of the teacher’s role.


Stepping Stones is an in-house strategy that to some extent reflects national programmes and leads into and supports the National College's modular curriculum. EHS is one of five schools in Surrey that form the Surrey Teaching School Network. As a group, the schools have been granted a licence to deliver the National College’s modular curriculum. EHS is taking responsibility for Level 1. The Stepping Stones programme also covers:

  • initial teacher training (ITT)
  • newly qualified teachers
  • early entrant teachers (EET)
  • Developing Middle Leaders (as a precursor to the National Professional Qualification in Middle Leadership)
  • Aspiring Senior Leaders (as a precursor to the National Professional Qualification in Senior Leadership) alongside a coaching programme

All levels of training include a leadership strand with activities based on practical leadership tasks. The challenge is to apply the theory to a real-world situation. Each practical challenge includes the need to illustrate impact, thus developing the skills of self-evaluation and reflection at every stage.


The commitment to leadership development is also about talent-spotting, nurturing potential leaders and ensuring that "everybody at any point in our school can see a leadership development opportunity." Staff don’t have to wait to get on a programme: "We find an opportunity for staff to grow," says Helen. The approach to the development of leadership at EHS is a fundamental part of the succession planning strategy. The promotion of leadership is seen as vital. The school is dedicated to an approach of growing its own leaders and the development of a 'house style' in terms of leadership is central to the work of the school.

Helen was able to illustrate the success of the talent-spotting strategy through reference to the NQT programme delivered two years ago. Six graduates of the programme are now in leadership positions. Staff were coached, nurtured and encouraged to develop leadership aspirations. There is a motto at EHS: 'If you're good enough, you're old enough'. Encouraging an aspirational culture is seen as very important.


Alongside Stepping Stones, EHS delivers a range of leadership opportunities which is offered to alliance partners. Programmes include an in-house Master's delivered by a teacher practitioner, the Improving Teacher Programme (ITP) and Outstanding Teacher Programme (OTP) and a Chair of Governors Leadership Programme, which has proved very popular. A faith leadership course is being developed, working in collaboration with the National College and the diocese.

The school is now building capacity among its alliance through the appointment of specialist leaders of education (SLEs). Successful deployment of SLEs is an essential part of the development of cross-phase working. SLEs are being deployed to respond to a wide range of needs including coaching, quality assurance in other educational settings, and the provision of training. Alliance schools are also contributing to the delivery of the leadership programmes. There is a need to build an expanding delivery model. There is a growing awareness that building capacity across the partnership will provide immediate benefits to both primary schools and secondary schools. The growing understanding of cross-phase needs is undoubtedly helping the whole community. Helen said:

QuoteIt's not just about us, it's about bringing everybody together and recognising that by building capacity everybody benefits. If we are involved in supporting another school we will bring our strategic partners together to see how best that provision can be delivered. We will explore the capacity, experience and expertise from a group of schools to find the best 'fit' response. We have seen this work successfully in a range of primary school settings.Quote

EHS maintains a real commitment to research and development (R&D) and staff at have been working hard to develop leadership skills so that they can lead on R&D work. All R&D work is based on themes derived from school self-evaluation and the school improvement plan, says Helen:

The strategy of identifying themes through the self-evaluation process ensures that the work does not stand in isolation. There is a strategic approach with all R&D themes drawn from identified school priorities.

Coaching is used to support group leaders and they receive regular support sessions throughout the course of the project. The school has learned from its mistakes when it was assumed that staff had the capacity to lead small-scale research projects with small teams of people. This simply was not the case. R&D group leaders needed support and guidance, and are now led by an in-house R&D specialist, who guides the learning and leading process. Of the six projects that were launched at EHS last year, four led to significant changes in classroom practice relating to marking and assessment. The findings from the remaining two groups raised further questions for the school to explore and have formed the basis for research groups in 2012–13.


Monitoring of impact is a key factor in all aspects of the school's work. This is particularly true when it comes to monitoring the impact of training. With such a diverse range of provision, the challenge has been to develop a system that could provide good-quality data to inform development. Helen has a lead role in developing partnerships with other secondary and primary schools across the alliance. She recognises that the school must continually work on developing its evaluation methods and that the information created from the process needs to be meaningful and informative. Evaluation feedback is constantly used to inform the planning for the next session. Helen notes:

You need to be brave enough to ask the questions and continually test your core purpose. You need to be prepared to test whether the training has had an impact and that kind of test may need to happen sometime after the training has taken place.

It is Helen's role to ensure that these questions are asked and that findings are used to inform future planning.

An intensive evaluation process has developed, which takes many forms, including:

  • post-course participant feedback
  • facilitator feedback discussions
  • monitoring of recruitment levels
  • measurement of longer term impacts on participants and on their classes
  • questioning of schools and participants three months after the course

Helen notes that the most informative feedback was obtained when she revisited course participants three months after the course had been delivered. The intention here was to look for evidence of impact actually in participants' schools. Where impact was noted, the endorsement of the course is received. On the odd occasion where a lack of impact was noted, Helen used this information to review the content of the course to ensure that amendments were secured where necessary.

Whilst gathering information three months after the course could be seen as a challenge, Helen found that the coaches and facilitators were keen to engage in the process in order to secure good-quality training that made a difference.


Helen ensures that her communication with colleagues is frank and honest. Where courses are effective, this is celebrated. Where there is a need for improvement, this is identified and changes are monitored. "You need to have a brutal approach to evaluation, everything must be evaluated. This leads to self-reflection on all sides and at all levels," says Helen, adding: "If you really want to know [whether you are being effective] then you need to take the time to drill down to see if you can see a longer term impact of what you are doing."

Helen has used evidence from this approach to evaluation to analyse the impact of the ITP. Evidence was collected three months after the course to ascertain whether teachers had moved from ‘satisfactory’ to ‘good’ in their daily lesson delivery.


Finally, Helen noted the importance of staying in touch with a sense of moral purpose. School improvement is not just about individual schools: it is about the wider learning community.

End rule showing end of an article
Reflections:
questions & activities

Analysis of practice

Now that you have read the account of practice from Esher Church of England High School, analyse the account to look for evidence of the effectiveness of evaluation of the school's leadership development programme.

Questions:


  • 01.
    What evidence is there that evaluation has been built into the programme from the beginning?
  • 02.
    How has evaluation been used to improve the programme?
  • 03.
    What improvements would you make to the way the programme is evaluated?
shadow

Activity

CPD evaluation

Task 1

Think of the most recent significant CPD activity that you took part in either as a CPD leader or a participant. Consider how that CPD activity was evaluated.


Task 2

Now assess the evaluation against the 10 principles for evaluation devised by Rotherham Metropolitan Council.

You can download a sheet below to complete this activity. It contains the 10 principles in a table format, with space for you to record your reflections against each principle.

Make a note of what worked well as an evaluation process and what you would want to improve, judged against the principles.