Terrorist Databases

One effect of the failed Christmas Day underpants bombing on NW Flight 253 to Detroit was to reveal how hard it is for counterterrorism bureaucracies to know, or do, anything about anyone. A leaked US Transport Security Agency document briefly required not only that passengers had been to the loo 61 minutes before their flight landed, but also that – with the exception only of heads of state and a single nominated companion – pat-down searches should be conducted at boarding gates, paying particular attention to the ‘upper thighs and lower torso’, a clear euphemism for underpants. So for at least a week any thinking terrorist would have had to find some other body part to strap his explosives to, and blow up his plane a little earlier than he might have liked. But the biggest disclosure was of the workings of America’s post-9/11 terrorist database systems, the main component of which is the Terrorist Identities Datamart Environment, or TIDE.

For some time privacy advocates have campaigned against this centralised repository of information on those ‘reasonably’ suspected of involvement in international terrorism, particularly as it gradually grew in size to accommodate 550,000 people. The conspiracy theorist fears that data are being hoarded about the good guys as well as the bad, that once you’re on the list you can’t get off it, and that in some sense we’re all being watched as we go about our daily business – using credit cards, making phone calls, surfing the web. What has become clear is that, however much information may be being gathered by whatever agencies, it’s very hard reliably to connect it to individuals. For one thing, a name – which is all you need to give when buying a plane ticket – is a very bad way of identifying people, not just because it can be misspelled, as Umar Farouk Abdul Mutallab’s was when his US visa application was granted, or because it can exist in different forms (I am D N SOAR when I’m paying for something by card), or because it can be changed, but mostly because very few names are unique.

For a database to provide a dossier on anyone, or on anyone ‘of interest’, each person in it must be allocated what the jargon calls a ‘primary key’, a unique value identifying them in order to tie together all the bits of information about them, including their name in all its various forms: as it appears on their passport, on their driving licence, to their bank, to their co-conspirators, to the people they chat with in internet forums (Mutallab was farouk1986 to his pals online). It turns out that one of the main functions of TIDE is, as the name suggests (it’s an ‘identities’ datamart), to help establish whether person X is the same as person Y. Every weekday evening, and twice on Fridays, a ‘sensitive but unclassified’ subset of TIDE – including biographical and biometric and other identifying data but leaving out the juicy ‘derogatory’ information which explains why a person is considered to be bad – is sent to an agency called the Terrorist Screening Center.

The TSC basically operates as a call centre – available, as they say, ‘24/7’ – to help law-enforcement officers and gun-shop owners and airline workers all over the world decide whether the person standing in front of them, whose details appear to match an entry on a terrorist watchlist, is the bad guy he looks like. Last year the TSC processed 55,000 requests arising from what the agencies call ‘encounters’ with suspected terrorists: 150 a day, or one every ten minutes. It’s a busy place. But only, really, because a person’s identity is such an unexpectedly non-straightforward thing to establish definitively. The TSC says that 30 to 40 per cent of the requests it handles result in a positive match; but it’s not much to boast of when you consider that two-thirds of the time, even when all the evidence points to someone being person X, they aren’t.

But this turns out to be the least of problems for the counterintelligence agencies. After all, they aren’t going out actively looking for the people on their watchlists; they just wait for them to pop up occasionally, at which point they can sometimes find out more about them. The rest of the time they’re trying to keep their data tidy, and complete; often, it seems, they fail. It was this kind of failure that prevented Mutallab’s name being elevated from TIDE to the No Fly List, as it apparently would have been had the various bits of information about him been entered in the right boxes and searched for in the right way. In August 2008 a memo prepared for a Congressional subcommittee reported that TIDE was a mess: in database-speak, it contained ‘463 separate tables, 295 of which are undocumented’, meaning that information was being entered in all sorts of different places without anyone explaining what sort of information it was or how to find it. The memo gave as an example a category of information known as ‘pocket litter’, the odds and ends – phone numbers, addresses, credit card details – gathered by emptying a suspect’s pockets. These were being put in 23 separate tables, willy nilly.

Obama, of course, has promised to sort it all out. He began by asking his deputy national security adviser John Brennan to investigate why the system had failed to stop Mutallab. Brennan’s previous job was as CEO of the Analysis Corporation, a private firm which provides intelligence and IT services to government agencies. One of its past services was to build TIPOFF, the predecessor database to TIDE, so Brennan knows what he’s talking about. He also knows that the system his company was paid to help build, and which he has found to have failed because of ‘human error’, is a monster that hoovers up whatever scraps of unconnected data it can find, and feeds off itself, and has only a slippery grasp on actual people in the real world.