Surveys are increasingly adopting mixed-mode methodologies, whereby data are collected by some combination of face-to-face, telephone, web and video. Due to differences in how items are presented (including the presence or absence of interviewers), responses can differ systematically between modes, a phenomenon referred to as a mode effect. Unaccounted for, mode effects can introduce bias in analyses of mixed-mode survey data.
This RSS event will present findings from recent research in the handling of mode effects, including: their placement within the simple and intuitive causal directed acyclic graphs (DAGs) framework; an overview of the methods available for handling mode effects; findings from a systematic review of the experimental literature on mode effects; and mode effects within the context of adaptive mixed-mode survey design. Brief abstracts for each presentation are provided below.
This event is suitable for anyone with an interest in survey mode effects and their handling, including survey methodologists, survey practitioners, and analysts of mixed-mode survey data across the many disciplines in which such data are used.
Joining instructions will be sent the day before the event.
Viewing survey mode effects through the lens of causal directed acyclic graphs (Richard Silverwood, University College London)
The increasing adoption of mixed-mode survey designs introduces two key challenges. First, individuals may respond differently to the same question depending on the mode (‘mode effects’). Second, different individuals may participate via different modes (‘mode selection’). In this presentation, mode effects and mode selection are placed within the simple and intuitive causal directed acyclic graph (DAG) framework. A range of possible data structures will be explored, with particular emphasis on the consequences of conditioning on mode (for example by including it in a regression model), a straightforward approach often employed in the suspected presence of mode effects.
Methods for handling survey mode effects (Liam Wright, University College London)
Using the previously introduced causal directed acyclic graph (DAG) framework, this presentation will describe the main methods for handling mode effects, including regression adjustment, instrumental variables, and multiple imputation. A further promising but underutilised approach is simulation-based sensitivity analysis, which does not assume no unmodelled selection into mode. Methods will be illustrated using real-world mixed-mode data from the Centre for Longitudinal Studies’ British birth cohort studies.
A systematic review of the experimental literature on mode effects (Georgia Tomova, University College London)
Performing sensitivity analyses such as those described in the previous presentation requires assumptions about the plausible magnitude of potential mode effects. A large number of experimental studies have been performed to estimate the size of mode effects across different variables and between-mode comparisons. However, results are spread across multiple papers and are not always straightforward to locate and utilise. This has been remedied by a recent systematic review, the findings of which have been incorporated into a searchable database so that relevant estimates can be more easily identified by researchers. This presentation will highlight the findings from the systematic review and demonstrate the use of the resultant database.
Adaptive mixed-mode survey design (Barry Schouten, Statistics Netherlands)
Adaptive survey design traditionally focusses on representation. But what if modes are one of the design features to adapt? Can mode effects be ignored? And how could one include such effects? The presentation will discuss adaptive survey design in the context of multiple modes.
Members - FREE
Non-members - £10.00