Skip to content

Q Exchange

Translating predictive occupancy data into real-time discharge action

We aim to determine the enablers and barriers to operational managers using real time predictive data on hospital occupancy to improve hospital flow.

Read comments 21 Project updates 1
  • Winning idea
  • 2022

Meet the team

Also:

  • Julie Dixon (University Hospitals of Leicester NHS Trust)
  • Carolyn Tarrant (Leicester University)
  • Mark Simmonds (Nottingham University Hospitals NHS Trust)
  • Adrian Boyle (Royal College of Emergency Medicine)
  • David McGregor (Abtrace)
  • Umar Amad (Abtrace)

What is the challenge your project is going to address and how does it connect to the theme?

The NHS remains in a critical state in regards to both emergency flow and delivering elective care. These two challenges are interlinked:

(i) Emergency attendances and admissions increase the occupancy of the hospital which leads to

(ii) less beds being available for elective cases, especially if discharging patients is protracted.

Operational managers are very good at understanding hospital occupancy and develop many methods of finding all spare bed capacity but are limited by good predictive data. Often hospital teams aim just to get through the next 12-24 hour period. Our team has been working for the last year on predictive modelling which has shown very impressive results in being able to predict hospital occupancy 24 hours in advance. This data alone will not be enough to produce meaningful change and we therefore wish to amalgamate improvement methodologies with our data modelling to give the managers the best chance of making accurate decisions.

What does your project aim to achieve?

This has progressed as collaborations have developed

  1. Understand operational managers (at bronze/silver command level) experience with using real-time data in practice at multiple hospitals and whether it impacts on their decision making.
  2. Observe the impact of using predictive data on hospital occupancy on operational decision making at large NHS trusts.
  3. Use qualitative findings from the work to develop a framework that would aid operational managers utilisation of predictive data

Our overall aim is to ensure that predictive data has a real time application. This has benefits for improving operational manager decision making, increasing clinical staff confidence and direct patient benefit as a result of flow being improved (it is well known that crowding increases mortality). In particular any learning regarding potentially disadvantaged patient groups will be taken forward as this is a particular issue the local health communities of the involved hospitals.

How will the project be delivered?

A small working group of Clinical Academics, Data Analysts, Business Intelligence operatives and machine learning experts has already come together to demonstrate the theoretical application of this model. This group will be expanded with the addition of hospital senior and middle managers, improvement specialists and qualitative researchers.

The team involved have experience of working in NHS trusts and have a track record in delivering projects to time and target. A number of the team, either as Q members or other funded projects, have direct links to the vision the Health Foundation has for the improvement of patient outcomes and reduction of inequality. We would be working with Abtrace, a digital health start-up lead by NHS doctors and data scientists. Their technology uses electronic health records and machine learning models to improve clinical decision making and tackling operational pain points on Hospitals and GP Practices.

How is your project going to share learning?

This project’s objectives include producing a translatable framework for the enablers and barriers to the utilisation of big data into clinical practice and operational management. This framework  will be shared via traditional (paper based) & digital (audio-visual) platforms supported by real time footage of the deployment of this data during a clinical shift. Through a number of the groups social media channels and links to wider networks this knowledge can be shared with the wider improvement community. The hospital trusts are fully committed to this endeavour meaning learning will be prioritised by the trust board.

The two (at this stage) confirmed hospital trusts (University Hospitals of Leicester NHS trust and Nottingham University Hospitals NHS trust) are similar sizes and use EPRs deployed in over 100 other hospitals allowing for rapid translation of findings.

We believe our project is synergistic with (but different from) other applications so we can collaborate on lesson’s learned.

How you can contribute

  • Has your trust undertaken similar projects whereby predictive data has been put into action by local teams? We'd love to hear your experiences?
  • Are you an operations manager or director who would like to share your experiences of running big organisations and where barriers arises in using data to it's maximum ability?
  • Do you have interviewing skills and are based in the East Midlands. We'd love to be able to undertake a wide range of qualitative work as part of the project.

Plan timeline

22 Mar 2022 Complete and Submit Idea
3 May 2022 Team Meetings and preparation of relevant research/logistical governance
17 May 2022 Celebrate idea with Q community
17 May 2022 Discuss with related Q exchange projects to confirm USP.
14 Jun 2022 Hopefully celebrate success as a Q exchange winner!
14 Sep 2022 Submission of governance/logistical permissions & employment of project staff
14 Nov 2022 Completion of Focus Groups and Interviews with operational managers
14 Mar 2023 Completion of observations of real world decision making
14 May 2023 Development of programme/operational guides (based on feedback)
14 May 2023 Repeat Focus Groups and Interviews to consolidate learning
13 Jun 2023 Dissemination of guides (depending on best format)

Project updates

  • 13 Dec 2022

    What have you learned so far?

    We have been pleased by the interest in this project and the study team has grown to include the President of the Royal College of Emergency Medicine, a PhD student interested in Hospital flow and a Lecturer in Operations and Supply Chain Management. The question we initially posed has been subject to a great amount of debate ranging from the nature of what ‘data’ is to, and perhaps more importantly, how it is practically used. It has become clear during initial discussions that there isn’t a clear evidence base for how data is used at operational meetings, and in fact, specifically in a health care perspective how decisions are reached. The input of an academic from outside of healthcare has been useful in order to remind us there is a large amount of literature (Theory of Constraints, Resource Based View, Queuing Theory) that hasn’t always been robustly applied to healthcare. One of the unintended outcomes of this piece of work is to write an editorial/review piece on this topic.

    Current Challenges

    It has taken a bit of time to refine a research question. The current aim is to highlight relevant themes that may assist hospitals in utilising predictive data effectively at operational meetings. We hypothesise there may be common themes that staff identify which are enablers or barriers to the effective use of data at operational meetings. These themes may then lead to methodologies that assist hospitals in utilising predictive data effectively at operational meetings.

    Unfortunately we have had delays in appointing a research associate to the study (although have recently resolved this issue) but more importantly there have been some protracted discussions about the work’s research governance. The original plan to have ethics from the university has not gone ahead as planned and we are now awaiting confirmation of sponsorship from the lead hospital site to enable an Health Research Authority submission to be made. Hopefully this will enable interview and observations to start in early 2023. The project is currently therefore 2-3 months behind schedule.

    How can the community get involved in your project?

    We are still interested in ideas from the Q Exchange community – particularly given the current challenges acute and emergency care faces. If any individuals or organisations have undertaken qualitative work of their operational meetings could they please reach out to us?

Comments

  1. I'll be watching the project with interest Damian.

    For me one of the key issues "bed managers" face is that -  even when staff are provided with a suite of relatively accurate information (predictors and/or holistic real time management tools) - due to the "constraints" on the systems in which we are working, the actual choices that managers are able to make, based on the data directly or indirectly, can be very limited.

    I don't think the primary rate limiting factor in terms of effective bed management (for some organisations) is necessarily the lack of good predictive "demand" data.

    If a system is already operating at full capacity / maximum occupancy, the choices are limited regardless of whether or not the actual demand is in keeping with a predicted level.

    That's why I think the two-pronged approach taken by Kettering / NHSX on their projected has really interesting applications (with further development) as a "real time" decision aid.

    However, it does also start to bring into sharper focus some of the ethical dilemmas that are are being faced by Bed Managers, Ops and Clinical Teams on a daily basis.  The Kettering Project likens the problem to Tetris.  I think that doesn't really do the problem it justice. To my mind, it is more like trying to address the ethical dilemmas that "the trolley problem" (no pun) highlights.

    Cheers.

     

     

    1. Hi Sygal and Matthew

      Many thanks for your comments.

      I believe, a large piece of organisational development and culture work required alongside the analytical work

      100% agree with you and I think this work will be the first part of this process of encouraging all sites to look at how data is used and what the communication enablers and barriers are at operational levels.

       

    2. Guest

      Sygal Amitay 16 Jun 2022

      Hi Damian,

      Sounds like a wonderful project. I'd be following with great interest.

      I think real-time information is really important, but that teaching/training bed managers/decision makers in how to use it is just as important!

      Our trust is currently embarked on a similar project with predicting the medically safe for discharge date, and there is, I believe, a large piece of organisational development and culture work required alongside the analytical work. Have you considered this aspect of the project?

      Best wishes, Sygal

    3. Congrats on the funding.

  2. You should take a look at:

    https://www.nhsx.nhs.uk/ai-lab/explore-all-resources/develop-ai/improving-hospital-bed-allocation-using-ai/  - further info: https://github.com/nhsx/skunkworks-bed-allocation

    If you haven't already. Some of work here, and the advanced application and implications are worth considering and linking with your project it would seem.

    Likewise, the work undertaken by GooRoo (incojuntion with NHS England and NHS Improvement) is very important: https://gooroo.co.uk/wp-content/uploads/2019/08/Planning_beds_bed_occupancy_and_risk.pdf

    Hope not teaching to suck eggs - but wouldn't want to duplicate things that are already out there.

    Cheers, Matthew

    1. Matthew - thank you so much for your interest in this project and sharing the resources! We'd not seen the github repository for the Kettering work and will be looking at this with earnest. The GooRoo document is really useful, our hope is this particular project will take some of the learning from this on (i.e. on the relevant information you need) and work out how operational managers actually use and act on this (as opposed to what we think they should and might do...)

  3. Our project https://q.health.org.uk/idea/2022/how-to-achieve-improvement-with-digital-tools-and-new-applications/ is likely to make use of data display boards. Maybe we could share information on code and boards. I'm good at process mapping and data mapping if you need any help in this area.

  4. Really like the idea and would be interested to know what data is used by operational managers to take action currently or is this just based on feeling? How will the new data set change this decision making process?

    In addition to this predictive data set, you mentioned about a framework - will this be a set of actions to follow, in different scenarios? If so, do you suspect any challenges with applying this type of standard work with ops managers? Sorry if I have got the wrong end of the stick with this one.

    Lastly, do you plan to share this with system partners or use system partners data sets to enhance predictions or support wider decision making?

    1. Thanks Thomas

      So currently the hospital teams (dependant on time of day) essentially know:

      (please note this is not the complete picture and is very simplistic. One of the things we'd like to learn in the project is how hospital managers use available data)

      Number of patients in the Emergency Department (A)

      Number of patients in the Emergency Department awaiting admission (B)

      Number of available hospital beds (empty and full) based on standard staffing levels (C)

      Number of patients in a hospital bed (D)

      Number of patients awaiting discharge out of the hospital (E)

      Essentially occupancy is C/D. If B > C-E you know that bed waits in ED will be prolonged.

      The challenge is that you can roughly guess what might happen over the next 12-24 hours in terms of arrivals to the ED or discharges but that's often based on experience and what happened over the preceding 24 hours (very changeable staffing levels during COVID made this much more complex).

      We have a model which will provide managers with some predictions of what is likely to happen with discharges and occupancy that might help inform decisions (do you stop or restart elective surgery for example?). We wonder, exactly what you are suggesting I think, how this will be received and enacted? While the data may be excellent its translation into actual practice more tricky and it's this real world deployment of a data we'd like to examine.

  5. This sounds like a really useful project, let me know if we can support you at RCEM.

    1. Thanks Sam - would be good to be able to work with RCEM for both clinical input and perhaps some Patient viewpoints. You'd be very welcome to join our study team!

  6. It looks a really promising project and we have similar aims in Oxford University Hospitals. I am currently one of a similar group of stakeholders so would love to touch base on this, irrespective of the success of your bid. I could also put you in touch with the medical team at NUH who have been working on impressive trust modeling around "Safe for Discharge".

    My thoughts currently are that for greatest impact this data needs to be made meaningful for front-line clinicians to help them better manage uncertainty at the point of care. Antibiotic stewardship is a good example of where this could have profound impact as currently there is wide variation in practice and hence length of hospital stays. As this recent BMJ Q&S article advises, using the wisdom of crowds leads to better decisions and less risk averse behaviours: https://qualitysafety.bmj.com/content/31/3/163.

    I anticipate a limiting factor will be the quality of data collected around decision to admit with "rubbish in, rubbish out". Our current live Q Exchange bid should help with enuring the digitial and clinical workflows are more closely aligned: https://q.health.org.uk/idea/2022/how-to-achieve-improvement-with-digital-tools-and-new-applications/

    1. Thanks Hesham

      You raise a good point about who the data is for. In our case, while obviously we aren't averse to frontline clinicians understanding the data, we think there is a separation of different forms of data for different audiences. From a ED perspective real-time data dashboards are vital to the running of a busy department i.e. I need to know how many patients I have, what their acuity is and what needs to happen to them.

      However this is slightly different from an trust management/operational team (which may well involve clinicians) who need sight of patients movement throughout the trust (I suppose moving from the individual patient in the ED to an area demand model for the whole hospital)

      I think one of the errors systems have made previously is thinking that all types of data are useful for all types of staff. Our early leaning is that this isn't the case.

      I see your project working on what this alignment looks like across a range of staff. I suppose ours is saying what do these particular group of staff (operational managers) need to make their job more effective. Hopefully we are coming at the same problem from two different angles!

  7. Can you predict discharge numbers specifically for ED or is the prediction hospital wide?

    1. The prediction is number of discharges from a clinical directorate (acute and emergency medicine) based on data on number of admissions to the hospital via the Emergency Department. Knowing the 'inflow' also enables you to measure (and predict) this directorates occupancy.

      Part of the ongoing work will also consider prediction of admission but from an operation stand point determining hospital occupancy and discharges was felt to have been greater use to operational managers and clinicians.

  8. Hi Damian

    This sounds like a really interesting project. Q member Sygal Amitay shared learning about her work as an analyst on an in-patient flow project here that may provide useful learning for you https://q.health.org.uk/insight/moving-past-backlogs-and-waiting-times/#Four

    A similar idea was drafted for Q Exchange in 2019 - it might be worth getting in touch with Navonil Mustafee to see if this idea went anywhere, and if there is learning you could take into your proposal? https://q.health.org.uk/idea/2019/shaping-demand-for-urgent-care-using-data-driven-nudges/

    Best of luck with the idea!

    Jo

    1. Thanks Jo - that's immensely helpful. Will start working through these resources!

  9. Guest

    David McGregor 3 Mar 2022

    Hi Damian,
    as an emergency physician currently working with your industry partner in the initial stage of this project I can see how this approach uses the data that we routinely collect at scale to improve patient care. I wonder if you should mention the recent RCEM analysis showing that at least 1 patient out of 67 came to avoidable harm in crowded EDs (2020-2021)? This helps make clear the immediate patient focus of the proposal and the potential for improvement.

  10. I like the idea of stating the methodological approach to improvement - pushing thoughts towards a continuing process. As the future environment around emergency care will be changing (because it always does!) it would be good to emphasise that an 'adaptive device' approach continues to learn from recent history and so changes as the system changes. May also be good to mention the NICE "Evidence Standards Framework for Digital Health Technologies".

  11. Hi Damian,

    This sounds very exciting and from my perspective fits the theme of digital and improvement neatly. I wonder whether you could outline how and which improvement methodologies will be utilised to a) develop the idea further, and b) how QI and continued organisational learning approaches could be used to support iterative and systematic learning and improvement following the implementation / pilot phase?

    Best,

    Thomas

    1. Thanks Thomas

      Good questions - and will certainly help shape this proposal!

      a) I think having been through the 'academic' phase of the project we now need to work a lot harder on co-production. At this stage perhaps not with patients (although the public's view on how they would feel about decisions potentially being taken based on data streams would be interesting - this is no different really than algorithms used by Amazon etc!). We definitely with managers and clinical staff. This is where there is definitely a dearth of good evidence and information. There are many machine learning and AI tools but implementing them meaningfully is a challenge. A good example from the early phase of our project is the time in which the information is delivered to teams. We have a basic model that can predict the number of discharges 24 hours in advance of them actually occurring. However this information may be difficult to get to the team (because of data draw down and validation) until mid morning - at what time does this information become redundant for the team who are on and is there a pinch point at which it has maximal impact?
      b) Work to do on sustainability of system learning. I think it is too easy for organisations to think of this as a quick fix as our suspicion is this will become truly iterative - the more data that is obtained the better the model becomes but that will impact on the decision making made and therefore potentially alter the model! It will be really important that we set up a longer term improvement ethos as we assess its impact otherwise it will just become just another 'tool' with a short  shelf life.

      Thanks for your comments!

Comments are now closed for this post.