Skip to content

Q Exchange

Impact evaluation framework for patient information videos

Develop a validated impact evaluation framework for patient information videos in clinical pathways that captures both patient experience and health economic benefits.

Read comments 4
  • Proposal
  • 2019

Meet the team

Also:

  • Nick Hex Associate Director YHEF York University
  • Louise Hall Evaluation Lead SW AHSN

What is the challenge your project is going to address and how does it connect to your chosen theme?

The first question asked of any new innovation is “what is the evidence it works?”  Torbay Hospital has being using short focussed patient facing information videos (presented as web based library and app) to leverage significant pathway redesign – reducing need for pre-assesssment OPD visits (22% reduction in angiogram), reduction in length of consultations (cardiology / podiatry), improved satisfaction with consent (urology) and reduced need for follow up (30% reduction post shoulder surgery). The Trust is now planning to scale this approach across a minimum of 20 new pathways with the aim of achieving a 4% reduction in OPD appointments, but wishes to understand in greater depth and robustness the impact this approach will have, in terms of patient (and staff) experience and service efficiency. This will not only provide reassurance re the current planned investment, but enable future work to be targeted at specific points in pathways that have been evidenced a having highest impact.

What does your project aim to achieve?

On behalf of the Trusts partnership venture (Health and Care Videos) the South West Academic Health Science Network (SW AHSN) has commissioned the development of a health economics evaluation framework from York University. This will specifically focus on the introduction of video based information into cardiology pathways. The work commissioned only encompasses the development of the framework.

This project aims to

  • validate the framework through data collection and comparing the modelled impact with actual achieved
  • add qualitative evaluation eg patient and staff experience tools to the framework
  • model and test transferability of the framework to other clinical pathways
  • assess this approach  as a potential foundation for the impact of other digital interventions

The intended output is a validated tool that can be made available to other Trusts undertaking clinical service redesign to use.

How will the project be delivered?

The cardiology service redesign undertaken in TSDFT has been used to develop a set of assumptions with respect to health economic benefits (appointments avoided / staff time saved). YHEC are modelling these and developing an evaluation framework (commissioned by SW AHSN).  Validated qualitative questionnaires will  assess patient experience and acceptability of the video based patient information.

This project will focus on collecting baseline and post implementation data from two sites – Torbay and Truro where video based information redesign of the cardiac angiogram pathway has been implemented.  Data sources will be patient administration systems, surveys and direct measurement (consultation time). Comparison of this observed impact will enable validation of the YHEC model.

The validated evaluation framework will be developed into a web based tool that can be used to model predicted benefits of video based patient information in other clinical pathways.

What and how is your project going to share learning throughout?

The transferrable learning from this project will include

  • understanding the  potential benefits of video based patient information in enabling clinical pathway redesign (and in particular impact on number / duration of OPD appointments).
  • factors that need to be considered in evaluating the health economic and patient experience impact of digital interventions
  • a web based toolkit that can be used to model the health economic impact that is transferable across clinical pathways and providers

Awareness of the tool will be raised through collaboration between TSDFT,  SW AHSN and YHEC and utilising networks, such as the AHSN network, Devon STP partnership and the Q Community. The results of the evaluation of the cardiology pathway will be disseminated through appropriate mechanisms eg presentations, abstracts, posters and potentially published articles.

How you can contribute

  • Would such a framework be of value to others involved in digital innovation implementation?
  • What are the impacts that are of primary interest?
  • Are there other Q Members who would be interested in using the validated tool?

Plan timeline

2 Dec 2019 Identify data analyst (Torbay and Truro)
2 Dec 2019 Recruit researchers at both sites (Truro and Torbay)
14 Dec 2019 Design and creation of data collection tools
31 Dec 2019 Data collection from 2 clinics both sites
31 Dec 2019 Extract baseline pre-introduction of pathway redesign
31 Dec 2019 Input of data from month 1 of project / flash report
31 Jan 2020 Data collection from 2 clinics both sites
31 Jan 2020 Input of data month 2 / flash report
31 Jan 2020 Interim data analysis and project report
29 Feb 2020 Data collection from 2 clinics both sites
29 Feb 2020 Input of data month 3 / flash report
29 Feb 2020 Prototype web based evaluation tool developed
31 Mar 2020 Data collection from 2 clinics both sites
31 Mar 2020 Final data analysis and project report
31 Mar 2020 Finalised version of web based evaluation tool
31 Mar 2020 Input of data month 4 / flash report
31 Mar 2020 Monthly project meetings (face to face / virtual)

Comments

  1. Hi Matthew. Very interesting.

    We are suggesting a similar project, looking at using video for increasing uptake and completion of pulmonary rehabilitation:

    https://q.health.org.uk/idea/2019/use-of-videos-to-promote-pulmonary-rehabilitation-in-copd/

    I think there are lots of potential applications.

    1. Thanks Steven

      I've added a comment to your idea and perhaps oppotunity here for a collaboration?

  2. Hi Matthew,

    What an interesting idea and really good to read about the early impact the videos have already had (side note: it would be great to hear about this approach, how the videos were developed and feedback etc... Perhaps one for a Q blog?).

    You've talked about gathering qualitative feedback and I think that will be vital. It will be important to assess patient feedback and other measures such as FU, confidence etc based on those who had accessed the videos and those who didn't. This might lend itself well to a randomised control trial, perhaps?

     

    Clinician feedback will be vital too. I wonder if they feel the videos release some pressure in clinics and means they can focus on the questions the patients have, rather than feeling rushed trying to get everything done..

     

    Anyway, just a few thoughts and good luck!

    Dimple

     

    1. Dear Dimple,

      Thank you for your comments and suggestions - hadn't though about a blog before, but will look into that.

      We have looked at randomised control trial approach in the past, but its very tricky to identify comparison groups and control confounding variables in such a complex system. Plus reassuringly clinical services just want to get on and use the videos so we are looking at real world validation / health service research approaches.

      I've added a comment to Stevens COPD idea that hopefully gives you some feel for how we have been using the videos and their impact (including as you say saving healthcare staff from providing the repetitive information and enabling them to focus on that particular patient).

      Thanks again for your interest.

Comments are now closed for this post.