The Stern Review
Brenna Bhandar
It is a rare moment when critics of exercises such as the Research Excellence Framework feel vindicated by a government-commissioned review. Nicholas Stern’s review of the REF, though broadly in favour of it, includes some important criticisms. It acknowledges that the REF has functioned to the disadvantage of women, Black and Minority Ethnic academics, and academics with disabilities; that it devalues interdisciplinary research; and that its narrow conception of ‘impact’ has been geared towards policy changes and the commercialisation of academic work.
Stern’s language presents a distinct contrast to the government’s recent White Paper, Success as a Knowledge Economy: Teaching Excellence, Social Mobility and Student Choice. There, the figure of the lecturer disappears; the idea of the university is replaced by ‘higher education providers’; and diversity and social mobility are given as reasons, without any basis whatsoever, for creating a fully fledged private market of higher education institutions. According to the Stern Review, meanwhile, ‘the HEFCE analysis of staff selection for the REF’ showed
a marked difference between the rate of selection for men and women. 67 per cent of men were selected, compared with 51 per cent of women. Black, Asian UK and non-EU nationals had lower selection rates, and the selection rate for staff with declared disabilities was lower than for those without.
As David Price observed in the THES last year, selection processes have ‘conspicuous potential for discrimination’ harming both individual researchers and research culture as a whole.
Stern criticises both the ‘gaming’ of the REF by institutions which appoint staff on costly fractional contracts, contributing little if anything to a department’s institutional research culture, and the ‘rent seeking behaviour’ of high-flyers who renegotiate their contracts just before the census date.
An even bigger problem that the Stern Review identifies is the way the REF encourages researchers to focus on the short term and avoid risk. Groundbreaking and discipline-bending research takes time. Academics interested in doing more than ‘shifting the deck chairs around’ (as an academic manager once described research to me) will welcome Stern’s emphasis on ‘finding ways to ensure that the REF can encourage researchers to explore big or fundamental problems in ways that may not deliver a steady stream of papers or a quick monograph’.
‘The disciplinary “silos” embedded in the Unit of Assessment panel’ have meant that interdisciplinary research is often ‘regarded less favourably than mono-disciplinary research’, Stern says. The fusing of ‘research quality’ with individual performance also threatens collaborative research. Some of the most important and discipline-shifting texts in the social sciences have been produced collaboratively. The Birmingham Cultural Studies scene of the 1970s and 1980s, for instance, placed collaboration at the nexus of intellectual-political projects, paid less heed to individual authorship and pushed disciplinary boundaries into new terrain.
Stern recommends that all ‘research active staff’ (though he doesn’t say who this would include or exclude) should be submitted to the REF, with an average of two pieces of work each, though some might have more than that and some might have none at all. The recommendation has been criticised for potentially introducing another damaging selection process (rather than who is submitted, the choice would be as to whose work goes forward). But it would be better to see it as an opportunity to emphasise the need to avoid the demoralisation and stress on individual academics that comes from such selection processes, which the Stern Review notes explicitly.
The recommendation that all staff be submitted to the REF could put an end to the often punitive internal review processes, in which a few ‘experts’ – who are often anonymous, their findings communicated through a line manager or REF co-ordinator – stand in judgment over their peers, deciding who is ‘in’ and who is ‘out’. The awarding of one, two, three or four stars to scholarship (which may fall far out of their own areas of research and interest, or what they consider valuable areas of inquiry), adds a level of infantilisation and indignity to the process. Stern recognises that peer review is not a ‘perfect measure’, an understatement given the broad range of expertise and time required to make a proper assessment, and the same goes for mock REF exercises. Ironically enough, these internal reviews or ‘mock REFs’ have developed in many departments and institutions at a time when open criticism and robust disagreement over ideas and research methods seem almost impossible to voice, for fear of offending someone, even in self-proclaimed ‘progressive’ spaces.
I wonder, when Stern acknowledges the ‘demoralising’ and ‘stressful’ effect of the REF selection process on academics, if he considered the extent to which this is compounded for Black, Asian and other minority ethnic academics, against whom the REF discriminates. The REF is often tied to probation and promotion, despite explicit assurances to the contrary, exacerbating what the THES has called the ‘horrifying figures on BME promotion’.
A more critical analysis of the Stern Review is certainly warranted – it praises the success and importance of the REF; it recommends that it be tightly aligned with the Teaching Excellence Framework, which has also come under increasing critical scrutiny; and concludes that higher education is better off with the REF than without – but the recommendations I’ve discussed here provide a solid basis for campaigning against some of the most pernicious effects of the REF.
Comments
As Bhandar notes, Stern criticises the way that the REF allows institutions to "game" the system by hiring staff with strong publication records shortly before a REF submission date. ("An institution might invest very significantly in the recruitment, start up and future career of a faculty member, only to see the transfer market prior to REF drastically reduce the returns to that investment," Stern writes.) Stern's proposed solution to this problem, which he outlines on pages 20-21 of the report, is to make research outputs "non-portable."
At the moment, the REF works on the basis that any research 'output' (a book, article, or whatever) is "portable", in the sense that the researcher who produced it can take it with them when they move jobs and submit it as part of their REF return at their new employer. This means that if you write 4 articles in 4 years, you can submit all of them for the REF submission in your new job at the University of Oxborne in Year 5, even though you completed 3 of the articles while you were a Research Fellow at the University of Camford. This is why the "transfer market" Stern identifies can function. Non-portability would mean that our researcher can only submit the last article she wrote at Oxborne, and Camford would retain the credit for the other 3.
Superficially this might be attractive (who would declare themselves in favour of 'gaming' the system?) but it masks negative impacts. For a start, there's nothing illegitimate about wanting to move jobs. It's normal in academia, as in life in general, to decide to move to a different part of the country, or to want to a new environment or a new challenge. People shouldn't be punished for such moves.
Non-portability will have severe consequences for academics in the early years of a career, as we all are. Short term contracts are a norm at this stage of a career (which is in itself a scandal), so moving institution regularly is not just a choice but an obligation. Often as an early career researcher we are employed on teaching-only contracts: our research outputs are written unpaid, or between the gaps in “portfolio working” (a tidy euphemism for precarity). As researchers in the Humanities, our publications are mostly single-authored, self-funded, and subject to a longer lag between research and publication (as our sector favours monographs and our journals have fewer issues and longer articles). This distinguishes our experience from colleagues in STEM subjects, and makes the impact of these reforms all the more acute.
Between the 4 of us we currently have 15 outputs eligible to go forward to the next REF. With non-portability that number would become zero, with strong pressure to produce more before 2021 to the exclusion of outreach work, professional development and pedagogical research. By a strange process of alchemy, our existing research will have disappeared and we will be given narrower paths along which to develop. Our previous employers, however, will happily be able to submit the work they did not pay us to produce while working there on teaching-only contracts. This is not a just exchange.
Stern's implication is that universities will now hire young academics not on their existing publications but on their 'potential', as evidenced in glowing references and job interviews. However, this risks further entrenching the advantages of certain kinds of academics: polished, white, upper middle class men. As Bhandar notes, the REF discriminates against minorities, but there's plenty of reason to believe that throwing things open to the gut feelings of hiring panels would do the same thing.
It seems clear that early career researchers were neither considered nor consulted in the creation of the review, since many of us - when we read the finished report - immediately pointed out this striking issue. We are now being asked to trust in the smooth implementation of these transformative recommendations, which seems both naively panglossian and oblivious to the lived experience of the last REF. These reforms will be channelled through the same administrative staff and institutional frameworks that created the “gaming” in the first place. Hiring panels would seem motivated to hedge bets and encourage potential applicants to shelve submissions, or perhaps find some other fudge that parlays the high language of reform into the easy vernacular of the status quo. What value then for publications and potential against the tempting gloss of confidence, contacts, and a chummy relationship with editors?
Non-portability addresses only the profession’s privileged elites and threatens a heavy cost for those seeking to build a career, while reducing our ability to engage with the positive aspects of the review. It is a solution to a problem which was itself created by the REF. Like the increasingly ridiculous diet of the old lady who swallowed the fly, it's no solution.
Erika Hanna, Charlotte Lydia Riley, Andrew WM Smith, Harry Stopes
As researchers in the Humanities, our publications are mostly single-authored, self-funded, and subject to a longer lag between research and publication (as our sector favours monographs and our journals have fewer issues and longer articles). This distinguishes our experience from colleagues in STEM subjects, and makes the impact of these reforms all the more acute.
In fact, much work in STEM and the humanities is `self-funded' (recte `not externally funded'). David Kernohan has given the numbers from the HESA data:
Arts and Humanities: of 3,674 academics in this area 3,234 (88%) did not have a research council grant.
Sciences: of 11,270 academics in this area 9,120 (81%) did not have a research council grant.
Social Sciences: of 7,226 academics in this area 6,583 (91%) did not have a research council grant.
http://followersoftheapocalyp.se/so-how-big-a-deal-is-unfunded-research-in-uk-he/
There are other sources of funding than research council grants, but the conclusions should be broadly similar with fuller data. Academic scientists, in other words, are almost as likely as social scientists or those in the humanities to be publishing without receiving a research grant.
I cannot comment on the `lag between research and publication', but again the time between submission and publication is not very different for mathematics (13 months), social sciences (14), and arts and humanities (14), though chemistry and engineering are quicker to publish (both about 9 months), and economics and business much slower (18 months). Data are given in Figure 1 of http://dx.doi.org/10.1016/j.joi.2013.09.001
As for the number of authors, multi-author papers are indeed more common in STEM subjects, but the difference is probably not as big as the commenters believe. Reliable data are hard to find, but variation within disciplines, and between sub-disciplines, is quite large:
In the Sciences, papers on average have 3 authors (i.e. 2 co-authors), whilst in the Social Sciences and Humanities papers on average only have 1.6 authors (i.e. 0.6 co-author). However, there are substantial differences even within these broader categories. The Computer Scientist on average only has 1.5 co-author, whilst the Cell Biologist on average has nearly three co-authors. The Cinema Studies academic publishes virtually only single-authored work, having only 0.1 co-author on average, whilst the Linguist and the Political Scientist on average have nearly 1 co-author.
http://www.harzing.com/publications/white-papers/citation-analysis-across-disciplines
There is a strong case against non-portability, but the problems it causes are no worse for the humanities than for STEM. Despite all the talk of `war on humanities', there is little reason to believe that one group of academic disciplines is being hit especially hard compared to others: universities and higher education are being attacked equally and the values of higher education and the working conditions of researchers should be defended on universal rather than sectional grounds.