How to Snoop

On Saturday night the Home Office website went offline for seven hours. The hacker group Anonymous took it down, they said, as a protest against the government’s planned new surveillance legislation. The plan, we learned earlier in the week, was to introduce a bill that would allow the security services continuous access in real time to all UK phone calls, emails and web traffic. It sounded scary, but most people stopped worrying about it after it became clear that nothing concrete would be known about the proposed legislation until the Queen’s Speech in May. There were also vague promises that the law, which would now be published in draft form only and open to consultation, would include the ‘highest possible safeguards’. ‘All we’re doing,’ Nick Clegg said, ‘is updating the rules which... allow the police and security services to go after terrorists and serious criminals and updating that to apply to new technology.’ ‘Let’s be absolutely clear,’ David Cameron said. ‘This is not about what the last government proposed and we opposed.’ He was very nearly telling the truth.

What the last government proposed and in its dying days gave up on was the Intercept Modernisation Programme, a centralised database containing detailed information from ‘communication service providers’ – that’s companies like BT, Orange, Virgin and O2 – on all our phone calls and internet activity. What the new government seems to want to propose – but we have to wait to find out – are exactly the same new powers of surveillance, minus the centralised database. In the spirit of small government and private enterprise, phone and internet companies will be obliged to do the state’s work for it and install equipment to give the security services full access at all times to all the traffic on their networks.

The law is a red herring here. In the UK, it has been legal for the state to have access to all our communications since the Regulation of Investigatory Powers Act (RIPA) was introduced in 2000. Reading someone’s email or listening to their phone calls requires the signature of the home secretary, who has plenty of time to consider the reasonableness of each request: in 2010, 1682 warrants were signed, or one every five hours. But to gather details of any other aspect of a communication – who is communicating with whom, when and where the communication took place – needs only the approval of a member of whichever agency wants to know: MI5, MI6, GCHQ, HM Revenue and Customs, the Met etc. The safeguard is that the person approving the order must hold an 'office, rank or position' in their organisation, which presumably excludes the tea boy but covers just about everyone else.

The government’s problem isn’t with the law but with the technology. In the old days, any half-decent spy knew how to steam open a letter or tap a phone. But modern means of communication – Facebook updates, Gmail chats, Skype calls – risk being invisible to those who want to listen in: just a stream of indistinguishable bits travelling over the network. This is especially tormenting for the spooks, since new technology has created exciting new classes of information, from a target’s precise location at any given moment (thanks to mobile phones), through comprehensive details of their contacts and connections (thanks to email, instant messaging and social networks), to lists of what they’re reading and watching (thanks to the web).

Luckily for the security services, there’s a whole industry prepared to help. A few times a year the private companies whose business is ‘intelligence support’ gather at trade fairs to sell their systems to their customers, the phone companies and internet service providers throughout the world who are legally obliged – by laws like RIPA – to enable ‘lawful interception’ on their networks without letting the targets know that anything untoward has taken place. Since these companies have stuff to sell, their amazing capabilities are advertised in glossy brochures rather than shrouded in secrecy. Take a company like ipoque, based in Leipzig, which sells a system called the DPX Network Probe,

a passive probe for lawful interception and network analysis in real time. It has been developed in close co-operation with law enforcement agencies and state authorities and is tailored to their individual needs.

DPX stands for ‘deep packet inspection’, a technique for analysing the data travelling over a network to find out through recognisable patterns what sort of information is being conveyed. The ipoque system can distinguish between several hundred different data types, from media delivery systems like iTunes or iPlayer to messaging tools like Gmail or BlackBerry Messenger to storage mechanisms like Dropbox. Companies that provide internet access, particularly mobile internet access, where data costs are high and bandwidth limited, have good reason to perform this kind of analysis, to manage the load on their networks and prioritise traffic so that half the world can still get to its email while the other half is watching Netflix or downloading movies through BitTorrent. But once the basic work of classification has been done there’s so much more to learn. Deep packet inspection, as ipoque’s marketing material explains, can identify usernames, email addresses, VoIP phone numbers and IP addresses: the where and between whom of all communication on the network.

The systems that companies like ipoque sell don't stop there, however. The DPX-1G server allows for the simultaneous search in real time for up to 25,000 keywords in all the data it is monitoring. Since the software supports ‘single character wildcards’ and ‘multi-word expressions with full Boolean support’, that means you could input a search phrase like ‘David Cameron is a f***er OR a c**t’ and be told instantly when anyone anywhere on the network was using the offending words, whether on a website or in an unencrypted email. It's pretty cool. Unfortunately for the security services, they can't (yet) officially use technology like this themselves. Under current legislation it’s illegal for the content of communications – the words in an email, the audio in a phone call or the video in a Skype transmission – to be intercepted (‘in whole or in part’) without Theresa May signing it off first. And since an intercept warrant requires an individual target’s name to be given or a 'single set of premises', it's impossible legally to intercept all the content travelling over the network at once, which is effectively what ipoque's system does. It may not be the hardest thing in the world to get the secretary of state to agree to the secret and ongoing interception of all of Abu X’s communications, but this isn't much use to the security services if they don’t have a clue who the next monster mullah may be. How they must wish they had access to a system like ipoque's, which could set a red light flashing at GCHQ the moment anyone mentions 'jihad' and 'Stratford' in the same sentence in a text message or IM.

Since this technology seems so dodgy you wonder who it's designed for. A company like ipoque doesn't disclose who its customers are, though it says that its deep packet inspection system is installed in more than two hundred networks in more than fifty countries. Those countries can't all be Syria and Iran. Why exhibit your gizmos at conferences in Washington and Prague, as ipoque does, and advertise your usefulness in fulfilling European and American law enforcement requirements, if much of your technology is illegal in countries like the UK? The curious truth is that it isn't illegal, even under the current legislation. There are two exceptions in RIPA to the rule that interception requires a warrant signed by the home secretary. One is if all parties to the communication have consented to the interception taking place – I don't suppose this happens very often. The other is if the interception is carried out ‘by or on behalf of a person who provides a postal service or a telecommunications service’ – that’s our favourite mobile phone companies or internet service providers – so long as it takes place for purposes relating to the operation of the service, or to comply with other laws. In other words, the telecoms companies can read all the email they like, using whatever advanced computing tools they fancy buying. What we don't know is whether they tell anyone else what they've read.