Evaluation models


Evaluation

In this topic, you will look at specific models for evaluation, building on the principles you examined in the previous topic.

One of the most enduring models for the evaluation of training and development comes from the work of Kirkpatrick, originally developed in the 1950s. As you will see, Kirkpatrick’s work is still evident in models developed much more recently, including work in the education sector in the 21st century.

Kirkpatrick's model considers the effectiveness of training on four levels.

An explanation of the model is provided below. Consider the reflection points at each level of the pyramid.

Kirkpatrick’s model shown as layers on a pyramid, beginning with reactions on the bottom level, moving up to learning, then behaviour, with results at the top

Building on the Kirkpatrick model

Since it was first developed, there have been many adaptations to the Kirkpatrick model. Some have suggested adding a fifth level:

Was the training worth the expense? Did it provide an acceptable return on investment (ROI)? 

This is especially relevant in the commercial sector where ROI is more easily measurable. You will consider value-for-money issues in the next topic.

American academic Thomas Guskey (Kreider & Bouffard, 2005) suggested inverting the Kirkpatrick model as a tool to plan development. In other words, start by considering level 4, 'results', and asking yourself what outcome you want the development activity to deliver. Then, consider what behaviour change is required to deliver that change (level 3), and what learning has to be transferred to support that new behaviour (level 2). Finally, decide how the development activity should make people feel to create an environment in which that learning can happen (level 1).


In earlier work, Guskey (2002) offered other adaptations to the original Kirkpatrick model to make it more specific to the education sector. He added two ideas in particular.

First, he added a new level which he called 'organisation support and change'. This recognises the key role that schools must play to nurture new learning and behaviours and enable them to take root. Guskey recognised that individuals often require support to implement learning within an organisational culture that encourages experimentation with new ways of working.

Second, Guskey introduced an explicit reference to student outcomes to the evaluation framework. This reflects the productivity or ROI levels added to the Kirkpatrick model by business commentators.


Watch this video about Colegrave Primary School in Newham, East London. It describes how a team of teachers adopted an enquiry-based approach to their CPD.

As you watch the video, capture your reflections in response to the questions in the table below. You can pause the video to give you sufficient time to make notes.

When you have finished, add your completed table to your notes and also record any overall reflections.


Download this response sheet to capture your reflections while you watch the video. Instructions are on the sheet.

Once completed, save the completed sheet for future reference with any overall reflections you may have.


Consider the following scenario.

Imagine that you are responsible for leading professional learning in a school. You have a meeting with an individual member of staff and his line manager. The purpose of the meeting is to consider how (together) you will evaluate a specific professional development activity that the individual staff member is about to undertake.

  • What questions will you ask the staff member, the line manager and yourself to ensure that you will be able to evaluate the activity effectively?

Record your ideas for questions.


Next, imagine the same group of people meeting together after the individual has completed the learning activity.

  • What questions will you ask the staff member, the line manager and yourself to encourage effective evaluation of the activity?

Record your notes.

shadow

Practical task

Question framework activity

Task 1

Apply part of the question framework ‘Suggested question frameworks for discussions’, you have just considered above, to interview a colleague in your own school. This might be:

  • the current CPD leader
  • an individual who has just completed, or is about to embark upon, a CPD activity
  • a team leader whose staff have just completed, or are about to embark upon, a CPD activity

Select the questions relevant to your interviewee and use them as a framework for conversation with them. Move beyond the questions to explore specific issues if appropriate: the questions should act as a prompt rather than a constraint.


Task 2

Record your reflections, taking into consideration these questions:

  • How well are staff able to answer focused questions about the evaluation of the impact of CPD? What enables them to, or prevents them from, focusing on the impact in a meaningful way?
  • What does this suggest in terms of how evaluation of CPD should be planned and implemented?