I note that this data tool is produced by Logan Tod & Co; I provide this feedback on the understanding that the tool is free for the NHS to use.

1. I like the detail about the assumptions and the detail about the data sources. You might want to add an assumption about how you’ve modelled people dealing with the backlog – here, it appears as if it will be steadily over time, which would not necessarily be the same as if Trust’s prioritise according to operational/clinical risk: preventing more breaches and clinical risk/acuity. I’m not sure how you would model that, because it would be very dependent on the referral pathways and behaviours. For example, it may be that there would be a smaller number of people referred during COVID-19 but those who have been are more urgent and so would need to be seen earlier and so the COVID backlog is removed quickly, but a large proportion of BAU remains. Regardless, it’s worth noting how you’ve modelled dealing with backlog.

Usability –
2. The tool is fairly intuitive, although I do have experience with PowerBI so it would be worth testing with non-PowerBI users. A method would be to assign a task – e.g. select ophthalmology and what conclusions do they draw from the data, then ask them to make it include all specialties. (You can use the info from point 8 to design theses tests.) Observe (possible virtually through screen sharing) how users interact with the tool. NB usability is not the same as user experience; asking users what they think would count as user experience.

3. The font is quite small. Using the new Microsoft Edge browser, I tried to zoom in on the text, it zoomed in and enlarged the ‘Power BI label in the left corner’ but didn’t enlarge the graphic. I’m not sure if there is a way to overcome this as I only use Power BI in-house; we don’t publish it to the web.

4. It is best to co-locate function with visualisation – e.g. put the capacity at the top, where the button for capacity scenario is.

5. Try labelling the graphic rather than using a legend; it takes longer for the audience to consult the legend and interpret the data otherwise. NB. Sometimes it doesn’t work because of placement, particularly if filters change.

6. How can you make it quicker for your audience to get the (correct) info they need? Explicitly provide summary information without requiring the audience to interpret the visualisation, this has the side benefit of minimising misinterpretation. Typically a data narrative would help with this, but understand that’s not necessarily workable here. Instead, I would put in something for specific time points – e.g. a table with a column for incomplete pathway categories and then the next column is how much of the working-capacity that category accounts for. This would explicitly show that in 2 months’ time 50% of working-capacity is likely to be BAU patients, 10% COVID backlog and 40% pre-existing backlog. You can get predetermined time points that you select using a drop-down or you could let the user select the desired time point.

Data layout
7. For visualisations I consider what audience am I doing this for, how do I expect them to use it, what experience/knowledge do they have of data and what do I want them to take away. Plan this for each page and visualisation. For this tool the audience could be Trust management, specialty management, commissioners and NHSE.

8. Of course, there is a trade-off in terms of the number of audiences you are catering for and how well the model will meet each of the audience member’s needs.

For example:
The pie chart is relevant for taking a whole-trust view. Note that if you’re clicking between specialties, the axis on the incomplete pathways graph changes. If people are trying to interpret quickly and don’t notice this, means they may misinterpret the data, change the axis ranges so they are the same for each specialty graph,

If you’re interested in a specific specialty, the pie chart takes a lot of space to visualise something which is not all that relevant to analysing a specific specialty’s data.

A way to overcome this could be having a multi-page tool for different audience requirements or just accepting the trade-off.

Hope that’s useful and apologies if I’m teaching ‘my grandmother to suck eggs’.

Rachel

HI Matthew,
There’s also some other modelling out there, an example is: https://www.hsj.co.uk/quality-and-performance/mapped-nhs-waiting-times-after-the-covid-peak/7027654.article