Experiences of video interviewing in two UK national cohort studies
Carole Sanchez (University College London)
Using video interviewing to conduct social surveys is relatively new but interest in this mode accelerated considerably during the COVID-19 pandemic. At the time of the outbreak in March 2020, the 1958 National Child Development Study (NCDS) was in-field with the Age 62 Survey and the 1970 British Cohort Study (BCS70), Age 51 Survey was due to launch. Both surveys were planned to be conducted face-to-face. Ongoing uncertainty with regard to future infection rates and associated restrictions made it difficult to gauge when resumption of face-to-face visits would be possible. The surveys are long, complex and involve cognitive assessments and health measurements which made switching to web or telephone data collection difficult. It was felt that video-interviewing had the potential to be the best way forward. We conducted a series of successful pilots which explored the feasibility of conducting the interviews via video prior to launching both surveys with a video-only approach during COVID, then continuing to offer this as an option when we resumed face to face fieldwork.
In this paper we will describe some of the key challenges arising including how to implement the video interview, interviewer training requirements and adapting protocols to enable the administration of cognitive assessments and collection of highly sensitive information via video. We will present findings on response rates achieved, initial findings on the impact of video interviewing on data quality and will consider the future potential of video interviewing as a mode of data collection for population surveys.
Collecting egocentric social network data in live video-based interviews
Henning Silber (GESIS), Theresia Ell (GESIS), Lydia Repke (GESIS), & Alexandru Cernat (University of Manchester)
For a project funded by the German Federal Ministery of Education and Research on the social consequences of the COVID-19 pandemic with a focus on loneliness and mental health, we collected egocentric social network data of 160 members from the probability-based GESIS Panel, which is a mixed-mode (online and mail) study of the German general population. Participants were selected by drawing a stratified random sample from the online panel members. In the live video-based interviews, respondents were asked to nominate their habitual contacts and provide various information on them (e.g., sociodemographics and relationship types) and the relationships between them (who knows whom). To facilitate the elicitation of such complex data, we used the open-source software Network Canvas. This interviewing tool is highly visual and based on consistent interaction (e.g., dragging and tapping), thereby reducing respondent burden. In our presentation, we will cover several methodological aspects of the data collection, such as response rates, nonparticipation bias, survey enjoyment, and participation motives. More generally, we will focus on those factors that impact individuals' inclination to engage in a live video-based interview.
Interviewer Effects in Video Interviewing
Brady T. West (University of Michigan), Ai Rene Ong (University of Michigan), Frederick G. Conrad (University of Michigan), Michael F. Schober (the New School for Social Research), Andrew Hupp (University of Michigan), Kallan Larsen (University of Michigan)
Live video communication tools (e.g., Zoom) have the potential to provide survey researchers with many of the benefits of in-person interviewing, while at the same time greatly reducing data collection costs, given that interviewers do not need to travel and make in-person visits to sampled households. The COVID-19 pandemic exposed the vulnerability of in-person data collection to public health crises, forcing survey researchers to explore remote data collection modes, such as live video interviewing, that seem likely to yield high-quality data without in-person interaction. Given the potential benefits of these technologies, the operational and methodological aspects of video interviewing have started to receive research attention from survey methodologists. Although it is remote, video interviewing still involves respondent-interviewer interaction that introduces the possibility of interviewer effects, and no research to date has evaluated this potential threat to the quality of the data collected in video interviews. We will present an evaluation of interviewer effects in a recent experimental study of alternative approaches to video interviewing, including both “live video” interviewing and the use of prerecorded videos of the same interviewers asking questions embedded in a web survey (“prerecorded video” interviewing). We find little evidence of significant interviewer effects when using these two approaches, which is a promising result. We also find that when interviewer effects were present, they tended to be slightly larger in the live video approach, as would be expected in light of its being an interactive mode. We conclude with a discussion of the implications of these findings for future research using video interviewing.
How to incoroprate AI interviewers in contemporary work surveys
Jan Karem Höhne (DZHW, Leibniz University Hannover), David Broneske (DZHW, University of Magdeburg), Cornelia Neuert (GESIS – Leibniz Institute for the Social Sciences), Joshua Claassen (DZHW, Leibniz University Hannover)
Web surveys are key for data-driven decision-making. Inexpensive and time-efficient web surveys successively replace other survey modes, especially in-person interviews. Even well-known social surveys, such as the European Social Survey (ESS), follow this trend. However, web surveys suffer from low response rates and frequently struggle with the assurance of high-quality data. The web survey expansion threatens all disciplines relying on survey data – ranging from sociology to health research – and puts decisions of officials and stakeholders at risk. Thus, there is a strong need for new data collection methods. New advances in communication technology and artificial intelligence (AI), coupled with an increasing electronic device ownership, introduce new ways of data collection. We build on these advances and study methodological and technological requirements for AI interviewer-based web surveys. Questions are asked through life-like AI interviewers and respondents answer through answer option selection or text. Fusing features of in-person interviews and web surveys has the great potential of mimicking human interaction and granting participants flexibility. We address the following overarching research question: How to create and implement life-like AI interviewers for surveying respondents? We will present data from a smartphone pilot (N ~ 1,200) conducted in November and December 2023 in which respondents were surveyed by one out of four AI interviewers. The interviewers were created through an AI video generation platform and varied with respect to gender and clothing. At the very end, respondents were asked to evaluate the interviewers with respect to warmth, rapport, naturalness, and closeness to a human interviewer. We present preliminary results on respondents’ evaluations of AI interviewers to shed light on how to design and conduct innovative AI interviewer-based web surveys. In addition, we provide researchers an introduction to the AI fundamentals associated with AI interviewers.