Skip to content

Evaluating QI capability building using the Kirkpatrick model: case example of the Lean Fundamentals MOOC

A case study exploring an evaluation tool recommended by FutureNHS.

25 Apr 2024
12:00 – 13:00

NHS IMPACT is NHS England’s new approach to improvement. It aims to support all NHS organisations to have the skills and techniques to deliver continuous improvement. Building improvement capability is one of NHS IMPACT’s five core principles.

Tools and resources to support NHS Impact are shared on FutureNHS and include tools to help evaluate improvement work. The Kirkpatrick model is recommended to evaluate the impact of improvement training and capability building programmes.

In this session, Dr Iain Smith will describe how the Kirkpatrick model was used to evaluate a large-scale programme to develop Lean process improvement capability in health and care. An adapted version of the Kirkpatrick framework was used to assess: participant reaction; knowledge change; confidence change; improvement results; and overall return-on-investment.

An interactive live exercise will be used to demonstrate the Kirkpatrick framework for evaluating learning. Audience participation in a practical exercise illustrating the influence of variation on process outcomes will be evaluated across several Kirkpatrick levels. Delegates will be engaged via online Zoom polling to evaluate their reaction to the exercise; their self-rated knowledge change, and self-rated confidence to explain process variation.

With over 25 years NHS experience as a quality improvement leader, Iain has worked in various roles to build capability to improve the quality and safety of healthcare. As an Associate Director at South Tyneside and Sunderland NHS Foundation Trust, Iain leads the trust’s planning, improvement and programme office to develop the trust’s improvement methodology in line with NHS Impact principles. Iain is a mathematics graduate and also hold postgraduate qualifications in innovation and transformational change as well as a PhD in healthcare facilities and service systems design using Lean. In addition to his full time role in the NHS, Iain is also an Associate Editor for the BMJ Leader journal and an Associate Researcher with Newcastle University Business School.

Supporting resources

  1. Building lean improvement skills at scale: an evaluation of a massive open online course in the English NHS
  2. Recovering staff, recovering services: massive-online support for recovering a paediatric service using Lean and compassionate communication
  3. Mass production methods for mass vaccination: improving flow and operational performance in a COVID-19 mass vaccination centre using Lean
  4. Operations management on the front line of COVID-19 vaccination: building capability at scale via technology-enhanced learning
  5. Capability building for large-scale transformational change: learning from an evaluation of a national programme