On Thursday 21 November the RSS Social Statistics Section held a meeting at the Royal Statistical Society to discuss the future of online data collection in social surveys. The event was organised by Dr Olga Maslovskaya and chaired by Professor Peter WF Smith, both from the University of Southampton.
The event had three presentations: one from an academic organisation (University of Southampton) by Dr Olga Maslovskaya, one from NatCen, an independent social research agency by Gerry Nicolaas, and one from the UK Office for National Statistics (ONS) by Colin Beavan-Seymour. The diversity of affiliations of the presenters suggest that the topic is very important for academics but also for non-academic organisations which design and implement surveys and collect social data in the UK.
Participants were also from diverse backgrounds including academics, PhD students, researchers from independent research agencies such as NatCen and data collection organisations such as Ipsos-MORI and Kantar Public among others.
Olga Maslovskaya presented the main findings from the ESRC-funded project 'Understanding survey response behaviour in a digital age: Mixed-device online surveys and mobile device use'. The presentation 'Data quality in mixed-device online surveys in the UK' concluded that we should not be concerned about allowing respondents to use smartphones for survey completion in social surveys as break-off rates for smartphones are very low and data quality is not different to other device such as desktop and laptops.
Gerry Nicolaas presented on 'Questionnaire design for online social surveys: Back to basics?'. Gerry concluded that current questionnaire design practices are not fit for web. She also stressed that while designing online questionnaires, it is important to get back to basic principles of good questionnaire design, to balance data user needs with respondent ability and willingness, to establish a collaborative relationship between survey customer and those designing and delivering the survey, and also to take multi-disciplinary approach to design. Gerry added that we need to develop guidance on transitioning offline-only surveys to include online data collection.
Colin Beavan-Seymour presented on 'User-centred questionnaire design and mixed mode testing of official surveys'. Colin presented Social Survey Transformation Division’s work on testing in the area of user-centred questionnaire design and mixed mode and specifically talked about four main tests which have been conducted within Labour Market Survey (LMS). The first test was focused on engagement strategy, the second on incentives, the third on mixed-mode design and the fourth one on online attrition. He reported on lessons learnt from the results of these tests.
The panel discussion was led by Peter WF Smith from the University of Southampton. He asked the panel on what they see are the main challenges in online data collection in social surveys. The panellists agreed that the main challenges are design of web surveys, representativeness of different strata of population and inclusion of off-line populations as well as understanding of respondents’ behaviour with technology (this list is not exhaustive). We need to consult other disciplines, for example, sociology and psychology, to gain more understanding of this behaviour.
The panel agreed that it is also very important to create a cross-sector network of partners to address challenges in online data collection in social surveys as a joint front.