In this talk, Professor David Hand will speak about his book
Dark Data: Why What You Don’t Know Matters. The book explores how lack of awareness of what you are missing can lead to distorted understanding, incorrect conclusions, and mistaken actions.
Abstract:
Dark data are data you don’t have. It might be that you want today’s data, but all you have is yesterdays. It might be that certain types of cases are missing from your sample. It might be that the recorded values are inaccurate – no measuring instrument is perfect. It might be that the process of collecting the data changes those very data themselves. It might be that you have only summary values, like averages, which tell you nothing about extremes. Or it might be data that has been collected and stored but not analysed – perhaps they were collected for regulatory compliance reasons. I outline a taxonomy of fifteen types of dark data, showing just how serious the consequences can be. But then I go further, showing strategies for coping with dark data, and even how to take advantage of it in a strategic application of ignorance.
Professor David Hand is emeritus professor of mathematics and senior research investigator at Imperial College London, a former president of the Royal Statistical Society, and a Fellow of the British Academy. He has published 300 scientific papers and 30 books, including
Principles of Data Mining,
Information Generation,
Measurement Theory and Practice, The Improbability Principle,
The Wellbeing of Nations, and
Dark Data. In 2002 he was awarded the Guy Medal of the Royal Statistical Society, and in 2012 he and his research group won the Credit Collections and Risk Award for Contributions to the Credit Industry. He was awarded the George Box Medal in 2016, and the Research Medal of the International Federation of Classification Societies in 2019. In 2013 he was made OBE for services to research and innovation.