Secrets are like sex

Like sturgeons​ and swans in medieval England, public information began as royal property. Today, we understand more vividly than ever before that information is also a commodity: I have it, you don’t; if you want it, you must pay me for it; if you don’t, I will use your lack of it to control you. Against this, and very reluctantly, a public ‘right to know’ has been imported into the Anglo-British state in the form of the Freedom of Information Act. But, as Richard Norton-Taylor’s pugnacious book shows, it’s a newborn right still struggling to survive against a centuries-old tradition of government.

The structure of the ‘British’ state is still essentially monarchical. Constitutionally, the rest of the democratic world has moved on, adopting variants of the Enlightenment notion of popular sovereignty. Power resides in theory with the people, whose communities lease upwards only those functions they cannot exercise themselves. But in Britain, its archaisms only lightly reformed, power still flows downwards. The absurd doctrine of parliamentary sovereignty – that weird English scrap of parchment – in effect means parliamentary absolutism, a hasty 1689 transfer from the divine right of kings. We don’t have ‘inalienable rights’, but are allowed to vote and speak freely only because the government, through Parliament, generously lends some of its power to its subjects.

Naturally, this monopoly includes information. Put crudely, the default position of Anglo-British governance remains that all official information is secret but that officialdom may choose to bestow access to parts of it on people who are not otherwise entitled to know it. For the last few centuries this has principally meant the press and the media, and the state has always been aware that such access can be sold in return for political support. The idea that all public information should be free and open unless otherwise specified remains a foreign notion, in spite of the Freedom of Information Act.

During the Napoleonic Wars, newspapers were allowed to read and reprint naval and military dispatches in return for carrying ‘paragraphs agreeable to the Ministry’. Today, as Norton-Taylor reminds us, Whitehall departments still run their own gentlemanly lobbies in which selected hacks are allowed to see confidential papers and hear civil servants’ unexpurgated thoughts on policy, but only so long as the journalists ‘play by the rules’ and write only what their host-official permits. The lobby system, with the Number Ten lobby at its apex, smoothly controls and shapes the outflow of official information to the public. Opposition leaders are frequently gagged by briefings given on Privy Council terms. What Norton-Taylor calls ‘the first building blocks of the wall of official secrecy’ were ‘erected around the year 1250 in the Privy Councillors’ Oath’. The oath, still taken in person to the queen, has what he calls ‘a profound, almost mystical effect, even on supposedly progressive individuals’. Prime ministers can safely confide secret information – often about intelligence – to senior peers, judges and members of opposition parties on the Privy Council without fear of its going any further.

But there are chinks in the wall. A City banker once said to me that ‘a tax breathes through its loopholes.’ In the same way, official secrecy breathes through its leaks. It was because the poll tax looked dodge-proof to all but non-dom millionaires that the people took to the streets, hit policemen and burned cars. If the Ministry of Defence were genuinely airtight, Britain might still be defending itself with patched-up Gloster Meteors while the generals or admirals who tried to warn the public would be locked up in Belmarsh with Julian Assange.

Norton-Taylor and a small band of his comrades are scarred but much decorated veterans of the ceaseless war between journalism and Whitehall secrecy – in Norton-Taylor’s case over a long career as the Guardian’s national security editor. Much of his achievement is a result of dogged, intelligent listening ‘between the lines’ to what officials are not saying. Much is his reward for persistent burrowing in the National Archives, again with a shrewd eye for which significant document is missing from a file (‘retained’). But a lifetime of experience in why, when and how officials prevaricate or lie has trained him to make proper use of an investigative journalist’s best source: the leak.

The list of known names of government employees who for moral and patriotic reasons have broken their duty of silence – and often their pledge under the Official Secrets Acts – is impressive. Some would call it a roll of honour. From recent years, it would include Clive Ponting, Cathy Massiter, Sarah Tisdall and Katharine Gun. Norton-Taylor seems to have known them all. Most of his best stories, however, came from the numerous civil servants, some of them very senior, who used a discreet journalist to reveal that government was up to something morally appalling, or disastrous for British interests, or both. Inevitably, several kinds of two-way relationship exist between the media and Whitehall – even between the media and spooks – and a journalist who takes at face value the warning that ‘this is just for you; must go no further’ will not get far. Norton-Taylor insists (though less sturdy colleagues hesitate to agree) that they need us more than we need them: ‘Too many journalists reporting on the activities of the agencies remain on the defensive, too ready to believe they have to rely on the goodwill of MI5, MI6 and GCHQ, and the official spokespeople of the agencies’ sponsoring departments, the Home Office and the FO.’ His own reputation for being relentless and immune to brush-offs led him to be seen, according to one irritated official, as ‘a long-term thorn in the side of the intelligence establishment’. But it also brought him respect where it mattered. Norton-Taylor was on good, if always guarded, terms with Stella Rimington and Eliza Manningham-Buller, two heads of MI5 who believed that the agency badly needed to be more open with the public. Manningham-Buller, an ‘extrovert’, he says, ‘was a prime example of how an individual could influence my perception of MI5, albeit temporarily’: her male successor closed the hatches again.

None of those named leakers, I suspect, would deny that any authority, even in a parliamentary democracy, must keep some secrets. (Norton-Taylor himself doesn’t discuss that question. Why should he, as a professional hunter, grant a right of life to rabbits?) But almost all of these critics of Whitehall, including Norton-Taylor, insist that Britain has for years abused official secrecy to conceal its mistakes, scandals and crimes.

Public postmortems and trials Norton-Taylor sat through include the Matrix-Churchill trial (‘arms to Iraq’) and the devastating Scott Inquiry which that trial provoked. He followed the Hutton Inquiry into the government scientist David Kelly’s suicide and the ‘sexed-up’ intelligence dossier on Saddam Hussein’s mythical weapons of mass destruction before the Iraq War. He dissected the grotesquely delayed Chilcot Report, which described how the Blair government blundered into that war. Best of all – because this kind of journalism can offer fun as well as lonely toil – he spent six happy weeks in Sydney reporting on the farcical Spycatcher trial, as Thatcher tried to block the publication of Peter Wright’s malevolent but fascinating account of his time in British intelligence. The judge even ‘waited for the British journalists to turn up late from their lunches of freshly caught fish washed down by the best Australian Chardonnay before starting the court’s afternoon session’.

Sometimes sheer moral outrage drives Norton-Taylor’s attacks. When a Chinook helicopter crashed into the Mull of Kintyre in 1994 – killing everyone on board, including most of Northern Ireland’s senior intelligence officers – the Ministry of Defence inquiry found the two highly experienced pilots guilty of gross negligence leading to manslaughter. It took 17 years for the MoD to apologise, finally admitting that there were grounds to suspect that the crash had been due to an equipment failure. Norton-Taylor had been violently abused for daring to doubt the inquiry, and for suggesting that the ministry was using the tragedy of the pilots’ death to smother any hint that other explanations should be looked at. In the aftermath of 9/11, he and other journalists refused to take seriously the assurances of Jack Straw, then foreign secretary, that Britain had taken no part in America’s lawless rendition of prisoners to secret sites and had had no idea that many of them were tortured there. The truth, published by the parliamentary Intelligence and Security Committee in 2018 after more than 15 years of cover-up and evasion, was that British agencies had assisted in 232 renditions, and in 198 cases had accepted information knowing that it was extracted by maltreatment or torture. Much of what Norton-Taylor came to know clearly emanated from civil servants and secret agents humiliated by what their masters were ordering them to do. His complaint is that the intelligence agencies, so often aware that governments are contemplating some train-wreck policy, keep this judgment to themselves and dutifully grease the rails for disaster.

Norton-Taylor’s main objection to Whitehall secrecy is not ethical but practical. Obsessional secrecy is self-defeating. If the British media are fixated on spookery – Douglas Hurd once observed that journalists would get excited by a blank sheet of paper if it was stamped ‘Secret’ – more openness would divert their energy towards hunting out the everyday deceits of ‘normal’ politics.

According to Sir Rodric Braithwaite, the wisest of our ex-ambassadors, ‘secrets are like sex. We all suspect that others get more than we do.’ Press sensationalism about the secret world has made discussion of its reform almost impossible. Lord Scott, in his inquiry, said that ‘government should, in my opinion, make it its business to do what it can to ensure that its critics are not ill-informed.’ Departmental secrecy, especially in the Ministry of Defence, conceals monstrous wastes of money: the useless and wildly expensive Zircon project – to develop Britain’s own defence satellite – was cancelled only after the journalist Duncan Campbell got hold of the story in 1987. More recently, open discussion about Britain’s real defence needs might have spared the country the uncontrollably soaring costs of Trident renewal and two new £3 billion aircraft carriers. ‘Secrecy and closing down debate were the root causes of Britain’s military interventions in Iraq, Afghanistan and Libya,’ Norton-Taylor writes. ‘The repercussions of these military adventures, which all fed into the fetish for secrecy in running the British state, are still being felt.’ As Manningham-Buller told the House of Lords in her maiden speech, ‘we compound the problem of terrorism if we use it to erode the freedom of us all.’

A fair few British reporters have been spooks of one kind or another, on salary or with some less formal connection. Norton-Taylor recalls that during the Bosnian war MI6 managed to infiltrate one of its officers into a job at the Spectator, where he wrote pro-Serbian articles under a pseudonym. Norton-Taylor himself – military family, jolly sound background – was looked over as a possible recruit when he left university, but ‘as far as I was concerned, secrets … were for sharing.’ All the same, when he started as a journalist his ‘right-stuff’ credentials won him the confidence of a lot of secret-bearers.

The search for the truth behind Whitehall cover-ups is always met with an exotic dialect of evasion, which Norton-Taylor has learned to translate all too well. ‘Robust’ means ‘frail and indefensible’, while ‘such decisions are taken on a case by case basis’ means ‘we can’t agree on a policy, let alone stick to one.’ ‘That’s mere speculation’ means ‘you’ve got it right.’ The Scott Inquiry produced some exquisite locutions. ‘Truth is a very difficult concept’ is the best remembered. Changes in the guidelines for exporting arms to Iraq arose from ‘a very vigorous implementation of a flexible interpretation’. Almost touching is a diplomat’s admission that he knew evidence in a trial was being suppressed: ‘I confess to innocent reluctance to connive at impeding the course of justice’ (he connived, all the same). Or this jewel: ‘I quite simply misled myself.’

The book ranges widely, and sometimes repetitiously. A long section about spies – the familiar tales of Philby, Blunt, Cairncross, Blake and all the others – is entertaining, but doesn’t reveal much that is new about their cases or about official secrecy in general. (Did Nicholas Elliott, sent to Beirut to confront Kim Philby with the evidence, urge his old MI6 friend to save everyone embarrassment by escaping to Moscow while he could? The reader gets no further on that one.) It was worth repeating, though again it’s not exactly new, that the battle to get access to important historical documents is very far from won. Why haven’t the complete papers on the 1914 Curragh mutiny – when a group of British officers threatened to resign rather than impose Home Rule on Ulster by force – been made public? Or the papers on the Suez crisis, or on Rudolf Hess’s 1941 flight to Britain? Or on the royal family’s foolish contacts with Hitler and his circle before the war?

The list of well-known intellectuals who were under MI5 surveillance before 1939 and during the Cold War is already notorious. But it leads Norton-Taylor into wise reflections about information overload – in particular, the preposterously expensive, usually irrelevant and basically unmanageable mass of detail about individuals that intelligence agencies acquire. Once this was gathered by armies of male spooks loitering in cars and pubs and armies of female spooks wasting their young lives transcribing what they heard through switchboard headphones. Now it is acquired digitally, saving human effort but adding to the Himalayas of data waiting to be tackled by witless algorithms. Whitehall’s ‘digital transformation’ allows officials to vaporise their own blunders before they reach an archive: handy for the officials, disastrous for historians.

The State of Secrecy​ seems to have been written – or proofread – in some hurry. Words are missing, the index is scanty and the author is made to say things he can’t have meant (for instance, that the Foreign Office feared that Argentinian – he must mean British – claims to the Falklands were far weaker than publicly admitted). Most readers under seventy will be baffled by a passing reference to ‘Commander Crabbe’ (who he?) or by the unexplained letters ‘PII’ (Public Interest Immunity order). And it’s puzzling that Norton-Taylor omits the most monstrous of all secret service intrusions into the media: the fact that for decades MI5 was able to control staff appointments at the BBC.

For detail and accuracy, Norton-Taylor’s book doesn’t reach the standard set by Tim Slessor’s devastating Lying in State (2004), which analysed many of the same cover- ups (its autopsy of the Mull of Kintyre crash remains a model for investigative reporting). But Norton-Taylor’s book, with its reminiscences, anecdotes and occasional (well-deserved) self-congratulation, wants to make some wider points about the context of Whitehall secrecy in the 21st century. One, inevitably, concerns the ‘special relationship’ with the United States, which now mostly depends on intelligence sharing and co-operation. But British military failure in Basra and Helmand has reduced the value of that intelligence to the Americans. Post-Brexit, Britain’s international standing is diminished, as it ceases to be a ‘global power’. As Norton-Taylor writes, it follows that ‘with weaker international links – save perhaps with the authoritarian regimes of the Gulf – Britain’s security and intelligence agencies will need even more support and understanding at home. For that, they will have to be much more transparent.’

His second point, desperately urgent in these early months of Boris Johnson’s administration, is that the law has often stood up for open government and rejected the establishment’s ingrained secrecy. Most people probably assume that when a government decision lands before a court, the judges move to the bench with a bucket of Cabinet Office whitewash ready beside their chairs. This has often been true in the past. Now it is not. Norton-Taylor includes many examples of the judiciary overturning official secrecy or excoriating activities by secret agencies. He recalls an intervention by Lord Igor Judge, who blocked police attempts to seize secret MI5 papers from the Guardian and the Observer and bring charges under the Official Secrets Act. Norton-Taylor quotes him: ‘Inconvenient or embarrassing revelations, whether for the security services or for public authorities, should not be suppressed … the safety valve of effective investigative journalism’ must not be stifled. ‘Such endorsements,’ Norton-Taylor adds, ‘have not prevented judges from imposing gag orders demanded by Downing Street and the security and intelligence agencies.’

This is all part of a momentous contest over constitutional liberty, a battle only now reaching full intensity. It’s a generation since judicial review began to pierce moth holes in government decisions. But the worn tweed blanket of parliamentary sovereignty – better described as Cabinet absolutism – developed a large rent last year when the Supreme Court struck down Johnson’s attempt to prorogue Parliament. Panic broke out on the authoritarian right. The law had forgotten its place, they cried, and was advancing uninvited into politics. It must at all costs be pushed back. Suella Braverman, the new Brexiteer attorney general previously known for her hatred of the Human Rights Act, now proclaims that ‘we must take back control not only from the EU but from the judiciary … the political has been captured by the legal.’

This is the language of a pre-Enlightenment government intolerant of opposition, refusing to acknowledge that power can reside and be legitimate outside the executive. It is, in short, monarchy-speak. And as the war begins between divine-right concepts of authority and the notion – now increasingly implied by English jurists – of the supremacy of the law to which prime ministers and parliaments must answer, the control of information will become a decisive battlefield. As Norton-Taylor warns, this government will fight hard to protect the secrecy of its members, spy agencies and special forces, and it will fight dirty.