The RSS has set out why we believe one of the UK’s most important university ranking systems, the Teaching Excellence Framework (TEF), has statistical issues that are so major that it is 'likely to mislead students who use TEF to inform their university choices'.
We are concerned that the TEF does not meet the standards set by the written to the UKSA Director General for Regulation (PDF) about this issue.
The RSS has also submitted a review currently being carried out by Dame Shirley Pearce, outlining the following concerns:
• Uncertainty is not handled coherently and consistently, nor communicated clearly.
• TEF awards are not necessarily comparable from one institution to another.
• The RSS has several technical concerns over the benchmarking component of TEF, including concerns that differences might be due to factors other than teaching quality, concerns regarding institutional game-playing and an egregious example of the 'look elsewhere effect’.
• There is a lack of transparency in that it is not easy to discover the full specification and details on implementation of the TEF, which makes it hard to check the statistical assumptions underlying the process.
The letter concludes by asking the Office for Statistics Regulation to 'consider the validity of the TEF, and to rule on whether TEF does actually provide the public with information which is trustworthy, of high quality and value.'
The RSS has raised concerns about the TEF several times before, and we are not confident that all of our statistical points have been adequately dealt with. RSS president Deborah Ashby said: 'Many prospective university students rely on these rankings to help inform their choices about where to study. We are concerned that the TEF is not reliable enough to bear the weight of this and could be misleading young people making important life choices about where to study.'