NOISE
A noise audit can reveal the differences that annoy your customers and frustrate your staff.
 
It is natural that your customers and/or your staff will resent being treated unfairly.  A recently published book suggests that much of the unfairness arises because of noise (ie random variation) in assessments or judgements or measurements.

The book, written by Daniel Kahneman and two co-authors, has the simple title “Noise”, and it suggests that the extent of this random variation, and the damage it causes, can be quantified by carrying out a noise audit within the organisation.  This would involve asking a number of assessors to judge a number of cases
The book does not simply rehash the knowledge and practices that are well-established in manufacturing and in business.  Indeed, the authors focus on situations rarely visited by scientists or statisticians.  These situations include:

• Custodial sentences imposed by judges,
• Predictions of cost and/or duration of projects,
• Reductions in insurance claims by loss adjusters,
• Diagnosis of medical conditions by doctors,

In all of these situations two judgements are likely to differ even if they were produced by the same decision maker and are likely to differ even more if made by two people. So, throughout the book the primary focus is on differences between measurements and the purpose of the noise audit is to quantify how much these differences are caused by the unintended random behaviour and the unintending bias of the assessors.  Kahneman asserts that the effect of the random variation often greatly exceeds the effect of any bias.

In this meeting we will carry out a noise audit and analyse the results.  As Kahneman’s book does not offer a procedure for such analysis we will adapt the two similar, but not identical, procedures used in measurement systems analysis and inter-laboratory trials.
 

RSS Quality Improvement Section

Free to RSS Fellows
£10 for non-Fellows