Meet the team: QITrainingforImpact
Quality Improvement Lead
South Eastern HSC Trust
- Northern Ireland
Clinical Lecturer in Public Health Medicine Consultant in Public Health Medicine
Public Health Agency and Queen's University Belfast
- Northern Ireland
Lecturer in Population Health and Healthcare
Queen's University Belfast
- Northern Ireland
Clinical Lead in Prison Dentistry
South Eastern Health and Social Care Trust
- Northern Ireland
This member's profile is private - please log in to view all the information.
- Mark McCrory, Lecturer in Management, Ulster University
The need for system transformation and innovation is widely recognised in health and social care in Northern Ireland (NI). The Department of Health published the Quality 20:20 Attributes Framework to develop the knowledge, skills and capacity of organisations in Quality Improvement (QI). Since then organisations in NI have been investing significantly in training staff to support and drive QI in practice.
Thus far, resource has been dedicated to the delivery of training rather than evaluating whether the training has had the intended impacts and how. In a study with more than 140 companies, Bersin (2006) found that organisations struggled to practically determine the value of training and were only spending about 2.6% of their total training budget on evaluation.
Evaluation is recognised as important but for many of the QI training programmes delivered, evaluation has been restricted to levels 1 and 2 of Kirkpatrick’s model (reaction and learning). Whilst important, this does not tell us whether the programmes are leading to participants changing their on-the-job behaviour (level 3 – training transfer) and driving QI within their teams and services (level 4 – results).
Again, this is not unusual. An evidence scan published by the Health Foundation (2006) reported that most evaluations of QI training focus on perceived changes in knowledge rather than delving deeper into the longer-term outcomes for professionals and patients. We have received similar feedback from the Q community as can be seen below.
The aim is to develop an evaluative framework for QI development programmes delivered across Northern Ireland with a specific focus on post programme impact. From this, we hope to identify the enablers and barriers experienced by participants when driving QI from which resources can be developed to guide future programmes to support participants apply their QI learning post programme.
Our project will consist of an evaluation of programmes already delivered across NI. Access across the region can be achieved as we are representing a regional Community of Practice for QI evaluation across the HSC Trusts, service users, both NI universities and the Health Leadership Centre.
We will draw on various evaluation methods but primarily, Brinkerhoff’s (2003) Success Case Method (SCM). The SCM is particularly useful in looking back at programmes already delivered to identify what has worked and how. The approach is based on comparing successful and unsuccessful cases through story-telling.
First, we will co-design a survey which can be used to identify successful cases i.e. QI training participants who have adopted the behaviours and habits of an ‘improver’ (Health Foundation 2015) and have embedded these within their practice (level 3 of Kirkpatrick), and who can demonstrate impact and outcomes within their teams and services through QI (level 4). The survey will be distributed to all the previous participants of QI training programmes in NI. Next, based on the survey results, we will conduct in-depth, semi-structured interviews with multiple success cases as well as non-success cases (hopefully between 25 to 30 interviews in total). Here we will document, with evidence, the success case stories, as well the enablers and barriers experienced by these participants. With the non-success cases, we will concentrate on barriers to achieving impact.
Impact and learning
The first major impact for the Q community will be an evaluative framework which allows healthcare organisations to efficiently measure impact of QI training at levels 3 and 4.
Secondly, our project will identify enablers and barriers experienced by participants post training. Research has shown that the work environment is one of the key elements of training transfer including: culture; available support/resources; and follow-up through additional learning opportunities (Grossman and Salas, 2011). We expect these issues to emerge but this project will help identify specifically how they emerge in a QI context as well as other unexpected issues.
From this we will produce a ‘transfer resource’ which we will share with the Q community, especially relevant to those involved in delivering, supporting or commissioning QI training, and with the wider healthcare sector who want to support transfer. We will share:
- Our progress through social media, our project website and Q newsletter,
- Our findings via our report to the Health Foundation, publication including paper based short reports distributed at suitable events; digital interactive reports including infographics via our project website and social media as well as contribution to the ‘Q talks’ programme.
The HSCQI have a website and newsletter which we would avail of. We will also seek to collaborate with Q members (e.g. through team members links with multiple SIGs) and Q Exchange projects involved in QI training.
Review with colleagues involved in QI training will allow us to demonstrate the benefit of the evaluative framework. With future QI training participants we will replicate the Success Case Method survey we design post programme to demonstrate the benefits of the ‘transfer resources’ provided.
How you can contribute
- Tell us how you have measured the impact of QI training
- Share your experiences of successful and unsuccessful impacts of training on care practices and outcomes
- If you have completed QI training, what has supported you to maintain the habits of an improver within your organisation?
- What are the barriers to transfering your QI learning and skills to the workplace?
This is a great project because…
The project is well structured, thoughtful and builds on existing evidence. Not enough is known about what makes QI training effective beyond initial feedback and understanding the impact it can have will provide useful learning for the Q community.
By the time of the event we encourage the project team to think more about…
Progressing work on identifying the programmes they will be evaluating will add significantly to the project - helping to make the learning potential for other Q members more tangible.