Hi Sahana, this is something we have lots of experience of at Matter of Focus and I can share some tools that other organisations we work with have used to evaluate quality improvement training and practicums. Do drop me an email if you would like to discuss, ailsa@matter-of-focus.com.

Hi Sahana – a lot of evaluations I’ve seen use the Kirkpatrick model for evaluating training programmes. You may have come across this already – it is helpful for breaking the different levels of outcomes (although it doesn’t provide tools). From the evaluations that I’ve read about of QI training programmes, it reinforces for me how important it is that there is a clear and evidence-based theory of change for the programme that helps surface expectations and assumptions by a range of stakeholders about what the outcomes are expected to be; and you let training participants know what is expected of them after the training in terms of evaluation, what support is available and how outcomes will be tracked.

Also, I’ve just been on an excellent webinar (organised by Emma and the other convenors of this group) discussing the VMI evaluation which has a lot of interesting learning related to your question. They discussed using Social Network Analysis to interesting effect to look at the extent to which people receiving training were going back to their place of work and sharing and using their knowledge. This strikes me as a great way to look at the impact of QI training, although it isn’t a tool that will be right for all circumstances. Another key learning shared in this discussion was the importance of understanding organisational/cultural readiness – which can help you to understand how and why some people will go away and apply their training to great effect, and others will struggle https://www.eventbrite.co.uk/x/measuring-the-impact-of-qi-reflections-from-evaluating-a-major-programme-tickets-130764386801

Hi Sahana, I agree with Jo that Kirkpatrick is commonly used for evaluation of educational interventions . I have not heard of the social network analysis- sound interesting. Here is a link to a an evaluaiton resource we developed which I hope will be helpful: https://ihub.scot/media/7486/guide-to-programme-evaluation-in-quality-improvement-v10.pdf

Hi Sahana,

Just to let you know we are also looking to run a session on evaluation of QI training in the new year, this is to present a toolkit that was developed as part of one of the Q Exchange projects – https://q.health.org.uk/idea/2018/transfering-learning-from-qi-training-for-better-impact-on-care/ – which used the Kirkpatricks model. We will confirm dates and times in the New Year.
Best Wishes,
Emma

That’s good news Emma! Will look out for the announcement as I’m very keen on the topic of evaluating QI programmes!