Who’s best?

The results of REF2021, the latest iteration of the Research Excellence Framework assessing the quality of research at UK universities, were published last week. My institution, UCL, is boasting that it came second, above Cambridge and beaten only by Oxford. Cambridge is boasting that it came third, but behind Imperial and the Institute of Cancer Research; institutions that shouldn’t quite count, it implies, since neither covers the full range of academic endeavour. Imperial, however, is clear that it has been shown to be the UK’s top university. The same claim is made by Oxford.

The REF is divided into 34 ‘units of assessment’, each corresponding to an academic discipline. The UK’s 157 universities matched their staff to the available UOAs and submitted in as many UOAs as they saw fit. Each submission included an account of the ‘environment’ for relevant research at that university, a set of case studies demonstrating the ‘impact’ of the institution’s research, and a set of ‘outputs’, usually journal articles, produced between 2014 and 2020.

Universities were required to include an average of 2.5 outputs per ‘full-time equivalent’ (FTE) researcher, and at least one output for every member of staff. Thousands of UK academics then devoted hours of work to reading these submissions, sometimes in association with quantitative data about, for example, how many times an article was cited, and scored them as four-star (world leading), three-star (internationally excellent), two-star (internationally recognised), one-star (nationally recognised) or not good enough.

The governments in the four nations of the UK vary in how they use the results. In England, universities are allocated a share of the Quality Related (QR) income using a score that reflects both the proportion of four-star and three-star rated research in a submission and the number of FTE staff.

Oxford had the highest volume of world leading research, while Imperial had the highest proportion of research rated as four-star. Cambridge uses the metric published in the Times Higher, which converts the proportions of star-rated research to a single Grade Point Average. UCL have taken the GPA and weighted it, as the funding body Research England will, by FTE to give a measure of ‘research power’.

The REF is designed to be objective and provides successful universities with a useful source of claims to bolster their reputations. This matters, perhaps, as much as the money. For most Russell Group universities, QR represents between 5 and 8 per cent of their total income; for UCL, Birkbeck and Imperial it is very slightly more; for most others it is much less. Having a reputation as a successful research university, however, is considered very important for attracting staff and students.

The REF used to be a huge source of resentment. Staff hated having to justify themselves to administrators; the crudeness of the categorical judgments and subjective nature of the assessments were all sources of ill-feeling. The vast cost in time and effort, not just of the REF process but of the various preparatory operations launched years in advance at ambitious universities, seemed out of all proportion to the result, which inevitably rewarded the usual suspects.

The sector now seems, however, to have come to terms with the REF. Perhaps this is because the benefits, in rewarding pockets of excellence in less well-regarded universities, are more widely recognised (Oxford University is not the leading university in the UK when it comes to the percentage of research in English Language and Literature graded as four-star; it isn’t even the leading university in Oxford on that metric). Or perhaps it’s because the idea that people should be trusted to get on with their work without being subject to this kind of scrutiny seems, now, a rather remote possibility.

The crudity of the categorical judgments and their nonsensical labels still sting. The REF organisers are keen that their work should highlight how good UK research is and have made much of the fact that 41 per cent of submissions were four-star rated, i.e. judged to be world leading. It seems unlikely that the world would agree with that assessment but perhaps that doesn’t really matter.

The real impact of the REF is to cement the idea that the quality of research is what determines a university’s reputation. It is what makes a university good, it is what makes an individual academic good. This matters to students because they care not only about the quality of the teaching they receive but also about the status of the university that will award their degree. An awkward consequence is that, since research funding at most universities fails to cover its costs, students are in effect paying for staff to burnish their reputations by spending as little time teaching as possible.