On the 20 November 2023, the Official Statistics Section (OSS) of the Royal Statistical Society and the Health Statistics User Group (HSUG) held a hybrid meeting in Errol Street to discuss how statistics producers can work together and with different users to create coherent statistics that are communicated in a transparent and accessible way to the public.
The first part of the afternoon focused on the theme of coherence of statistics, numbers and narrative, while in the second half the panellists and the audience exchanged their views and ideas around communications. We have summarised below the key points from the discussion, and a detailed report of the discussions has been produced and is available on the HSUG website
www.hsug.org.uk in the Webinar Materials section.
- Coherence
There is lack of coherence of health data across the UK, and also sometimes within England due to multiple producers.
Challenges for producers
The first thing that needs to be acknowledged is that health is a policy domain devolved to each individual country of the UK, and as such policy decisions create different requirements for outputs. Politicians, the media and public often focus on front end metrics, in part because they are perhaps easier to comprehend and report on, as well as being useful to describe different aspects of the health services. At the same, it is important that statistics, including front end metrics, are brought into context, including consideration of the societal determinants.
More and more now, producers of statistics are not the ones collecting the data which causes additional challenges. Producers can harmonise surveys but they are not in control of operational data which has been defined to meet operational needs. This is true both for the health system as well as administrative data obtained from other government departments. In the specific domain of health, which was the focus of the event, it is also worth noting that the people collecting the data are health professional and as such can be trusted for accurate recording of health information. However, they are not social scientists and their primary focus is not on collecting additional socio-demographic information, which are key data for producers of statistics as they are used for drawing representative population inferences. In addition, data producers may change their collection systems overtime in response to changing operational needs. This can create an enormous task to change systems or adapt methods if any changes are to be made to data definitions for instance. So how best can statistical producers connect with data producers? And would it be appropriate or justifiable for statistical producers to dictate how the data producers design their system and collect information?
Last but not least of the challenges discussed in the afternoon was around users and their differing needs, with the acknowledgment that it is always easy to prioritise these needs. When society is changing, there is a further need and expectation to adjust and improve statistics, which results in a tricky balancing act between maintaining continuity to protect the timeline and ensuring that data definitions are up to date to meet the current needs.
Problems for users
Discussion with the audience also highlighted some of the challenges that users of our statistics faced. There was an acknowledgement of the difficulty in putting together coherent UK statistics from a range of sources, with outputs and data being sometimes provided in different and not necessarily consistent formats. This is compounded by the fact that apparently comparable figures sometimes may differ because of definitions or way the underlying data were collected, or that sometimes different names and terms are used, making it harder to understand what is comparable and what is not.
Possible solutions
Throughout the first session, while discussing the challenges above, a number of possible avenues for solution emerged:
- Co-production (across producers) and working more closely with users from the start rather than consulting at the end of the process, though with being wary of giving more voice or weight to perhaps not-fully representative groups.
- Need to recognise that extra time is required for collaboration – but is important because no one department or nation has ability to harmonise on their own. Early agreement on standards for new data is good if achievable, and it is always easier to harmonise upfront at the data collection stage that later on.
However, as producers of statistics we should be mindful and remember that the primary objectives of the departments producing the data is to deliver a service to their users, and not to collect data for statistical purposes.
- Joint user engagement will help harmonisation – the earlier that can harmonise the better.
- Need to acknowledge that the legal situation following devolution is that it is OK to do things differently in how the service is organised. You need to accept that and work from that basis.
- Even if it is not possible for producers to harmonise operational data, or harmonise data across the four countries, they can:
- aim to harmonise the adjustments needed to bring the data together;
- look at coherence in outcome measures.
- Output methods developed during the pandemic could be used for users to ‘self -serve’ more.
With offering the facility for users to ‘self-serve’, producers have a responsibility to explain the context and the different policies underlying the data, including explanations on differences in data definitions, terms used, quality, and to make these as accessible as possible.
- Communicating risks and uncertainty in health statistics for the users
The second session focused on communication and how we ensure that our statistics and messages land as intended.
Problems
A core aspect that is present throughout the
Code of Practice for Statistics is ensuring the users are at the centre of how statistics, and especially official ones, are produced. This means:
- ensuring the statistics are those required by the users, that is they answer the questions and address the issues that matter to the public.
- communicating to the wider public and letting them know that maybe they are not on occasions asking the right questions.
- Communicating in a format that both meets the needs for detailed stories while being short and concise.
- Balancing the needs of statistics to cover both individual experience and population experience.
Some of the challenges in communicating statistics and risks specifically stem from the varying degrees in statistical expertise in the general population but also across different user groups, including politicians and health professionals. If provided solely with outputs, the users, policy makers, politicians, media may not have the time or necessary background to understand how to interpret the outputs of models and their strengths/ limitations.
In addition, when it comes to risks and making decisions from statistics, it is often a trade-off rather than a clear-cut choice, between what the statistics or model outputs say or tell us and what is the alternative or what is feasible to do.
There may also be a gulf between public expectations from the health service, which could be supported by or resulting from data and evidence, and what is actually happening or what solutions are feasible.
Possible solutions
Following on the idea of co-production to help with creating coherent statistical system, producers should consider working more closely with users, and in particular journalists, to understand how the public view data and how it is used and used that information as feedback mechanism to improve their outputs.
Acknowledging that different methods of presentation will be required for different users, and for dealing with individual decisions compared to understanding population issues, producers may gain with working with the press and the media, and learning from their techniques, e.g. data journalism.
There is a tendency to be technocratic when publishing statistics, so there may be an opportunity here to include more storytelling, which in turn required to understand the relevance of personal experience - putting information in context of things that people know.
This could include using data to frame discussions and common understandings from a local perspective as well as from national/population level – not just for the here and now, but also in terms of the longer-term future. There already exists a framework to do that as part of the way statistics are produced, by using more regularly the ‘voice of statistician’ section in statistical publication to highlight what is relevant in a publication not just providing the figures, or by publishing blogs to bring the work into context. Opportunities are growing, particularly with new technologies and AI to help the users navigate the large amount of data and evidence being produced and by learning from the pandemic on how to communicate regarding wider determinants of health.
While new mode of accessing and interacting with data have arisen, democratising access to information, there is a need to be aware of possible accessibility barriers from difference in the technical kit people, internet access to varying accessibility needs in terms of understanding and processing content. For instance format that works for one person may not be work for another, so a variety of format should be considered (graphs, pictures, words, infographics etc) , as there is one method that is easiest for all.
In terms of users engagement, there is currently quite good engagement with expert users, but producers need to engage more broadly; Regional Forums offer an opportunity to reach people more widely and should be leveraged more. Public discussion of figures can be helpful and should be welcome, in particular open criticism/scepticism of figures needs to be understood and addressed, rather than simply dismissed or ignored.
Finally, producers have a role to play, alongside charities and third-sector organisations, such as the RSS, in training policy makers, MPs and the media in the interpretation of figures to ensure they understand both the uses and the limitations of data and model outputs. This includes raising the importance of transparency around inherent uncertainty in figures, which should be presented and always explained.
And lastly, but very definitely not the least important for effective communication, there is the need to be open and honest, making clear what we can say and cannot say from any statistics or data.
The organisers would like to extend their thanks to all the discussants who shared their time and perspective:
Catherine Bromley (Deputy Head of Profession for statistics at Public Health Scotland, previously at OSR and former member of the HSUG)
Ed Humpherson (Office for Statistical Regulation)
Sakthi Karunanithi (Director of Public Health, Lancashire County Council)
William Perks (Head of Health, Social Services and Population Statistics at the Welsh Government as well as one of the heads of profession for statistics)
David Schley (Sense about Science)
Julie Stanborough (Deputy Director for Health and Social Care Analysis; Office for National Statistics, as well as chair of the UK Health Statistics Steering Group)
Lucy Vickers (Chief Statistician, Department for Health and Social Care)
Michelle Waters (Office for National Statistics)
Heidi Wilson (Office for National Statistics)
Recording of the event is available on the RSS youtube channel