The RSS has issued a strongly worded statement regarding the latest Teaching Excellence Framework (TEF) consultation.
The RSS statement opens as follows: ‘The Royal Statistical Society (RSS) was alarmed by the serious and numerous flaws in the last TEF consultation process, conducted in 2016. Our concerns appeared not to be adequately addressed by the Department for Education. Indeed, the DfE’s latest TEF consultation exercise suggests that few statistical lessons have been learned from 2016’s experience.’
The subject level TEF is a proposed new measure used to judge the quality of teaching at subjects at university and the Department for Education is currently consulting on how to measure it. The RSS identified statistical and scientific shortcomings in the July 2016 consultation, Teaching Excellence Framework: Year Two and Beyond. Our response outlined concerns ranging from the paper’s assumptions around causality, lack of evidence about a link between teaching quality and employment outcomes, and the way in which uncertainty was being handled.
These concerns have not been allayed by the Department for Education’s latest consultation, Teaching Excellence and Student Outcomes Framework: subject-level, where two possible models, A and B were presented. The RSS, however, saw flaws in both models, and detailed its many concerns in a statement.
RSS vice president Guy Nason, who chairs the RSS Academic Affairs Advisory Group, is also concerned that around three quarters of those being consulted were from education providers and student unions. ‘While it’s right that they are consulted, it seems wrong that key statistical design issues could be decided by unscientific opinion polls,’ he says. ‘It is a bit like asking learner drivers to influence the contents of the driving test, or how examiners should assess them. Such an approach surely takes us in the wrong direction.’
Ultimately, the RSS believes that ‘there is a real risk that the latest consultation’s statistically inadequate approach will lead to distorted results, misleading rankings and a system which lacks validity and is unnecessarily vulnerable to being ‘gamed’. Even worse, continues Guy, ‘significant resources are clearly being, and will continue to be, devoted to an already discredited system.’