Dark Markets

‘Dark pools’​ are private, electronic share-trading venues in which a participant can bid to buy shares or offer to sell them without those bids or offers being visible to the market at large. For most of their history – they’ve been around for nearly thirty years – they have attracted little attention, but that has changed fast in the last couple of years. This is doubtless partly because of the name, which resonates with a widespread sense that financial markets are opaque, at least to outsiders, and a place where murky deals are done; the Wall Street Journal reporter Scott Patterson called his recent book on automated trading Dark Pools, even though it isn’t really about dark pools. But a more specific reason for their higher profile was the announcement on 25 June 2014 by Eric Schneiderman, attorney general of New York, that he was filing a securities fraud complaint against Barclays Bank. Barclays, Schneiderman said, claimed that its dark pool protected ‘institutional investors such as mutual funds and pension funds holding the savings of millions of New Yorkers’ from ‘the predatory high-frequency trading tactics that are seen on public exchanges’. High-frequency trading, or HFT, is the fast, entirely automated trading of large numbers of shares or other financial instruments.1 Schneiderman is an outspoken critic of it. In reality, he alleged, there was ‘no protection for any ordinary investor’ in Barclays’ dark pool, which was ‘full of predators who were there at Barclays’ invitation’. Schneiderman is also said to have launched investigations of the dark pools run by Credit Suisse, Deutsche Bank, Goldman Sachs, Morgan Stanley and UBS.

Schneiderman isn’t the first New York attorney general to show his unwillingness to leave the policing of financial markets to federal bodies such as the Securities and Exchange Commission. Eliot Spitzer, who was elected attorney general in 1998 and went on to become governor of New York, took on Wall Street using an old, largely forgotten legal weapon that Schneiderman is now using against Barclays: New York’s 1921 securities fraud law, the Martin Act. The act grants the attorney general wide-ranging authority to subpoena documents, and action under it does not require proof of scienter (knowingly fraudulent intent); it seems too that the attorney general may not even have to show that losses were caused by the bank’s behaviour. Nicholas Thompson, writing in Legal Affairs in 2004, reported that those questioned under the Martin Act have no automatic right to have a lawyer present and that the Fifth Amendment right not to incriminate themselves did not apply. In the years before Spitzer turned to the Martin Act, these draconian powers had mainly been used against small-time fraudsters. There was, Thompson said, ‘an unspoken gentleman’s agreement’ that it shouldn’t be used against major financial institutions.

Dark pools were originally set up in the US in the late 1980s as an alternative to the main trading venue at the time, the New York Stock Exchange. Suppose you were a portfolio manager at an institutional investment firm in the 1980s who wanted to buy a block of shares in a corporation traded on the NYSE. You, or (if the firm was a large one) a trader working on your behalf, would typically phone up an investment bank or another ‘broker-dealer’ firm that was a member of the NYSE. There were essentially three things the broker-dealer could do with your order. First, so long as the order wasn’t too big, they could submit it electronically to the NYSE. It would then arrive at a trading post in one of the five large rooms that made up the NYSE trading floor. There, the order would no longer be handled entirely automatically. Each stock traded on the NYSE had a ‘specialist’. He (they were nearly all men) was both an auctioneer and a trader on his firm’s own account. If his ‘book’ (his list of orders that had not yet been executed) contained offers to sell shares at a price that matched that of the incoming buy order, he or his clerk could execute the trade, charging a commission for doing so. If that wasn’t the case, the specialist could trade on his own account, selling the shares himself; or he could put the incoming order into the book to be executed at some later point. Only he and his clerks could see the full book, which was a valuable, private source of information.

The second thing that a broker-dealer firm could do with an institutional investor’s order was to telephone its booth on the NYSE trading floor, whose staff would in turn page a ‘floor broker’, usually also employed by the firm. After receiving the order at one of the many yellow telephones in the five trading rooms, the broker would walk over to the specialist responsible for trading the shares in question. If the specialist was already surrounded by a crowd of other brokers, all of them bidding to buy shares and/or offering to sell them, the broker could join the action. If there was no crowd, he could simply leave the order for the specialist to execute, but instead would often have a brief chat with him. Two sociologists of finance, Daniel Beunza and Yuval Millo, witnessed some of these conversations in 2003, at which point the NYSE trading rooms hadn’t yet been fully automated. They describe following a floor broker around, noticing ‘how he addressed, backslapped and saluted with nicknames the people he met on his way. Everyone on the floor was Johnny, Jimmy or Bobby; there were no Johns, James or Roberts.’ It wasn’t just masculine clubbiness: good personal relations were important for business. Beunza and Millo watched another broker, who had an order to buy a large quantity of shares, ask the relevant specialist about the book. ‘I think it’s a little heavy,’ the specialist said: there were lots of existing bids to buy, so the broker might do better by his firm’s customer if he held on to his order until a little later.

The third thing that a broker-dealer firm could have done with the order was to pass it not to a floor broker but to one of its ‘upstairs’ brokers, whose job was to keep in regular touch with big market participants who might wish to buy or sell the shares for which he was responsible. When he received a customer buy order, he would then phone his contacts or use a private computer network, AutEx, to seek out in a discreet manner an institutional investor or broker-dealer firm that might be prepared to sell the shares in question. If he were successful in executing the order this way, it would never circulate on the trading floor.

The NYSE’s interpersonal way of trading shares had its virtues. If a customer needed to sell a block of shares in difficult, volatile market conditions, a skilled broker could reduce what’s called ‘market impact’: the tendency to drive prices down while executing the sale. An experienced specialist could keep trading going in an orderly fashion through temporary panics or frenzies. One specialist told Beunza and Millo how he practised ‘crowd control’ when surrounded by floor brokers all frantically trying to sell shares on behalf of their customers, telling them: ‘OK, let’s calm down, let’s see if we can find some buyers, let’s see what happens at various prices, let’s talk this thing out, let’s do business.’

Crucially, too, brokers on the NYSE trading floor could exercise human judgment in the face of bizarre price movements. On 6 May 2010, the new world of automated share trading suffered its first systemic crisis, the so-called ‘flash crash’.2 As prices fluctuated wildly, many electronic traders simply stopped trading, switching off their automated systems. The NYSE reverted to manual trading. Three of the five NYSE trading rooms had already shut, but enough of the old way of life was left, Beunza and Millo report, to stop shares trading on the NYSE – as they did at exclusively electronic venues – at plainly absurd computer generated prices of a single cent or $99,999.99.

Set against these advantages were a number of disadvantages, especially from the viewpoint of an institutional investor that wanted to trade a large number of shares without making its intentions known. All this interpersonal interaction required skilled, experienced personnel whose services were expensive, some of whom had incentives to do things that hurt the institutional investor. A specialist might be tempted to ‘front run’ an incoming customer order (that is, to profit from his knowledge of the order by trading ahead of it on his own account), or to ‘interposition’ himself: buy shares from one customer and sell them at a higher price to another, when he could and should simply have matched the two orders. Observing the NYSE trading floor in the early 1990s, the sociologist Mitchel Abolafia was initially sceptical that rules such as those against front running and interpositioning would be properly observed, but eventually he was persuaded that, in general, they were. However, as commissions fell and the shift to pricing US shares in dollars and cents (rather than the traditional eighths of a dollar) reduced specialists’ trading revenues, they became increasingly opportunistic in their pursuit of profits. By 2003, there were only seven firms of specialists left on the NYSE, five of which agreed in March 2004 to pay a total of $242 million to settle charges of front running and interpositioning.

Institutional investors worried too about the upstairs brokers. They handled very large orders, and the investment banks and other broker-dealer firms that employed them traded on their account too. Steve Wunsch, a stock market veteran, reports that some were worried ‘the market was effectively rigged.’ Upstairs brokers ‘had more information’ than their institutional-investor customers, says Wunsch, ‘and were constantly accused of abuses of that privileged position’. They might, for example, curry favour with customers, hoping to be rewarded with future business, by revealing other customers’ trading intentions. Even putting deliberate abuse to one side, it is part of an upstairs broker’s job to canvas other firms, so buying or selling through them was ‘prone to information leakage’, as one of my informants, who had helped set up a dark pool, put it to me.

It isn’t surprising, therefore, that there was some interest in finding a way for ‘naturals’ to trade directly with each other, rather than having to rely on expensive and possibly self-interested intermediaries. (A ‘natural’ is an institutional investor that wants to add a large block of shares to its portfolio or to sell a portion of it, in contrast to a professional trader who is just looking to turn a short-term profit.) Some naturals were doing this already: they would simply phone their counterparts in other institutional-investment firms to carry out a trade. But you could do that only if you trusted those you spoke to not to take advantage of having learned that you were, for example, trying to buy a big block of IBM shares.

What is now​ regarded as the first dark pool was set up in the autumn of 1986 by a firm called Instinet, which created computer-trading links among institutional investors. Users of the service, called the Crossing Network, could submit anonymous bids to buy or offers to sell shares after the NYSE and other public trading venues had closed. The user simply entered the number of shares he or she wished to buy or sell; the price of the shares was always that day’s price at the end of public trading. At 6.30 p.m. Eastern Standard Time, Instinet’s computer systems would ‘cross’ those orders, matching as many bids and offers as possible.

A similar anonymous crossing service for ‘naturals’ called Posit (the ‘Portfolio System for Institutional Trading’) was launched in 1987 by the Investment Technology Group and a financial analysis company called Barra. Posit’s four daily crosses also matched buy and sell orders at the price at which shares were trading on the public markets, in its case at a randomly chosen moment in the seven-minute interval during which one could enter orders into its system. A later ‘first generation’ dark pool, Liquidnet, was set up in 2001 by Seth Merrin, who was well known to US institutional investors because his original firm, Merrin Financial, had pioneered order management systems. These cut out the need to use the phone to transmit orders, which could lead to misunderstandings and mistakes; order management systems made it possible for an investment manager to type in the details of the shares they wanted to buy or sell, and the system would pass the order electronically to the firm’s traders, who could then also use it to send orders on to the broker-dealer firms that executed them.

While the Instinet and Posit ‘crosses’ required an institutional investor to take an active decision to use them, Liquidnet has continuous electronic access to the digital ‘blotters’ of institutional investors’ order management systems: these contain lists of the orders for shares that haven’t yet been executed. Whenever Liquidnet’s system discovers that one institutional investor’s blotter contains an order to buy, say, IBM shares and another’s blotter an order to sell them, it invites the two traders to begin an anonymous, computerised negotiation over the price. (Each order has to meet the minimum size of trade that the other trader will countenance, but providing that criterion is met, neither trader is told anything more about the size of deal the other is seeking.) A window opens on each of the two traders’ computer screens, with a range of possible prices between the highest bid and the lowest offer on the public markets. If the two traders agree on a price, Liquidnet’s system tells them how many shares they have bought and sold (that number is simply the smaller of the sizes of transaction each trader was seeking). ‘Most veterans that have been on the system for a while don’t even negotiate,’ I was told, ‘they just offer the mid’ (the midpoint of the range of prices in the window), and typically that is accepted straight away.

Dark pools such as Posit and Liquidnet made it possible for institutional investors to trade more cheaply than by using an investment bank as a broker, and sometimes very large deals were and are done using them: a bugle sounds in Liquidnet’s New York office when a deal is made for a million shares or more. However, the take-up of these first-generation dark pools was limited. This is partly because professional traders weren’t allowed to use them, and sometimes you just couldn’t find another natural who wanted to buy when you wanted to sell, or vice versa. But, more significant, these dark pool interactions didn’t cater to every aspect of the relationship between institutional investors and investment banks. The banks weren’t merely the investors’ brokers: the relationship between them was also influenced by what are known as ‘soft dollars’. Investors may have paid a higher premium to do their trading via the bank, but in return the bank showed them the results of its stock market research, and sometimes offered other benefits too, such as the use of very expensive Bloomberg terminals, which gave them easy access to market data.3 (The excess cost of trading was ‘soft’ from the perspective of a firm managing pension or other savings funds because trading costs are charged to those funds. Research or Bloomberg terminals, though, had to be paid for with the management firm’s own money.) The bank might also reward an institution for trading with it by being more active in marketing the institution’s savings products to its retail customers, or it might allow the institution to buy shares in the profitable IPOs (Initial Public Offerings) of the booming dotcom companies of the late 1990s.

Eliot Spitzer wasn’t happy about these cosy relationships between institutional investors and investment banks. After the dotcom and telecom bubble burst in 2000-1, he deployed the Martin Act against investment bank analysts who had issued research reports boosting the shares of companies from which their firms were earning investment banking revenues, shares that privately they may have believed to be dubious or worthless. The settlement that Spitzer and the Securities and Exchange Commission negotiated with the banks forced greater separation between the different kinds of business they did with institutional investors. This, and the development of ‘transaction cost analysis’, which enabled institutional investors to measure more accurately how much buying and selling shares was really costing them, focused their attention on the cost of trading.

At the same time​ , cheaper, automated forms of trading were becoming available to institutional investors. As stock markets became more completely electronic from the mid-1990s onwards, ‘algorithmic’ trading became possible. This involves taking a big order from an institutional investor, splitting it up into small parts and computerising the execution of the small orders. Alongside the ‘high-touch’ execution of orders by brokers, investment banks started to offer their institutional-investor customers less expensive, ‘low-touch’, algorithmic execution – for example, via Credit Suisse’s Advanced Execution Services department, set up in 2001. Credit Suisse’s algorithms didn’t demand any technical knowledge on the part of institutional investors, who were provided with a simple computer interface. You would select the shares you wanted to trade, I was told, ‘just type in “buy 100,000” … and it all gets worked on by the computer behind the scenes. You don’t need to be a programmer.’

Another way of reducing the cost of trading was ‘internalisation’, in which an investment bank such as Credit Suisse matched buy and sell orders from its customers internally without ever sending them out to the wider markets. This process was the antecedent of a second wave of dark pools set up by the big investment banks, many of which are in Schneiderman’s sights. As someone involved in creating one of these pools told me, internalisation ‘had a nasty connotation in the US, because there was always a lot of gamesmanship … “Did I get a fair price?”, “Did I not get a fair price?”’ But the process could be made more legitimate by turning the internal matching of orders into a dark pool governed by the SEC’s regulations, even if initially little else changed: ‘We could create an execution venue, a proper execution venue with the blessing of the SEC, and all of a sudden it wasn’t internalisation anymore, it was crossing, right? It’s an identical process, identical flows; everything was the same except we had a machine do it instead of having people do it.’

Credit Suisse again took the lead. It launched the Crossfinder dark pool in 2004, followed by Goldman Sachs’s Sigma X and by similar offerings from other investment banks. Where the first generation of dark pools didn’t usually require users to enter a price but simply matched buy and sell orders in the middle of the price range that prevailed on the public markets, the new wave of dark pools mimicked the electronic exchanges: you can enter into their order books bids to buy shares – or offers to sell them – at specific prices, and the orders are matched only if another order is there to match it. There is no direct human involvement in the consummation of trades: bids are matched by a program running on the dark pool’s computer systems. The pool’s managers most likely work at the investment bank’s Manhattan headquarters, but its computers will be in one of the big data centres in northern New Jersey in which US shares are traded (or, in the case of European dark pools, in one of the data centres in and around London). Usually, the investment bank’s algorithms executing its customers’ orders run on servers in the same data centre as its pool, so that the algorithms have fast access: ‘My smart order router can ping my own pool in sixty microseconds,’ one pool manager told me.

However, the order books of these dark pools, unlike those on electronic exchanges (or ‘lit’ markets, as they are increasingly being called, to distinguish them from dark markets), are not visible to the traders and computerised trading systems that buy and sell shares on them. The new dark pools also differ from their predecessors in that they typically handle large numbers of small, algorithmically generated orders, each for as few as a couple of hundred shares, whereas the original pools aimed to match much smaller numbers of far bigger, human-generated orders for tens of thousands of shares or more. And while the original pools typically excluded professional traders, the new dark pools allow them to participate – including those who employ computerised high-frequency trading – and a bank’s trading desks will also often trade in its own dark pool.

Volumes of trading in the new dark pools quickly exceeded those in their predecessors. By the autumn of 2009, Crossfinder was trading an average of more than 140 million shares a day, and Goldman Sachs’s Sigma X nearly 120 million, compared to around 30 million traded on Liquidnet. In 2008, only just over 4 per cent of share trading in the US was in dark pools; by January 2014, that had climbed to nearly 15 per cent.

There is​ a persistent fear about dark pools, which flickers in the background of Schneiderman’s allegations: that they aren’t entirely dark. ‘A lot of dark pools are different shades of grey,’ one trader told me: information leaks out of them. It’s not a new issue. Even the early ‘crossing’ systems could leak information. If, for example, you had submitted a buy order for IBM, and only a quarter of it had been ‘crossed’ with sell orders, you could guess that there was probably further unsatisfied interest in buying IBM. If you knew the total volume of IBM shares that had been crossed, you could easily work out just how big that unsatisfied interest was.

In the case of Liquidnet, the worry was that a participant could appear to be a ‘natural’, but might instead be lurking, doing nothing in response to invitations from the system to negotiate, or beginning to negotiate but then ‘fading’, never agreeing a price. What you could learn by doing that wouldn’t be precise (the system blocks you from trying to find out the exact size of counterparties’ orders), but again it could be exploitable information. Liquidnet monitors the behaviour of firms that use it, and kicks members off the network, a source told me, for ‘using improper behaviour or not adhering to member community protocol’.

Fishing for information in this way is perfectly legal. But there is an illegal form of market manipulation that is made possible by the fact that the price at which many dark pools cross bids to buy and offers to sell is tied to the prices prevailing in a ‘lit’ market such as the NYSE or the London Stock Exchange. What you can do is enter bids or offers into the lit market that will temporarily change prices on it, and immediately send an order to a dark pool that exploits that change in price. The reason Posit and other dark pools often randomise the exact time of the cross is to make this form of manipulation harder.

There’s currently something of a moral panic concerning high-frequency trading, fuelled not least by Michael Lewis’s Flash Boys, which leads critics to equate opportunistic behaviour in relation to dark pools with HFT. But I would be surprised if a mainstream HFT firm practised market manipulation of the kind I’ve just described. Legally, it’s far safer for a human being to manipulate a market (intention is very hard to prove, unless someone has been stupid enough, say, to describe their strategy in an email) than to have a machine do so. The problem with the latter is that you need to write a program to perform the manipulation, and that is dangerous evidence to have around.

Much of what the big HFTs do is ‘market making’: they continually post bids to buy shares and offers to sell them at a slightly higher price, hoping to earn the difference between the two prices. Those who run markets have no problem with that. Dark pools and lit markets are involved in a fierce struggle with each other for market share (there’s a ‘market for markets’), and in recent years the pie has shrunk: the overall volume of stock trading in both the US and Europe has been falling. So you badly want HFT market makers in your dark pool to help keep prices keen.

But dark pool managers are more ambivalent about ‘aggressive’ HFT. This isn’t market making: it involves ‘hitting’ bids or ‘lifting’ offers that are already in the order book. An HFT system may, for example, infer from conditions in the futures market that share prices are about to rise or fall. Or it may receive a fast datafeed direct from a lit market (perhaps one whose computers are in the same data centre), and be able to profit if the feed is more up to date than the prices at which bids and offers are being matched in a dark pool.

Managers of dark pools particularly dislike it when an automated system ceaselessly sends in and then cancels tiny bids and/or offers – it’s called ‘pinging the book’ – in the hope of being able to gain potentially profitable information about the contents of the hidden order book. If, for instance, your tiny offers of a particular stock suddenly start being accepted straight away, it would be reasonable to infer that there is a biggish buy order for that stock in the pool or an algorithm in the process of executing such an order. You can then expect there to be a rise in that stock’s prices. This is how one dark pool manager described the process:

I ping it and send [a sell] order; I get a fill. There’s something there. I do it a couple more times; there must be some size [a big buy order]. I take a guess: I don’t know if it’s ten thousand or a million shares … With the knowledge there’s something there, I go into the public markets and I start lifting offers [i.e. buying shares]. Boom, boom, boom.

The dark pool managers are in a dilemma. On the one hand, they can’t realistically ban HFT completely (as the older pools are able to do), because they need it to ensure there are always orders in the book for institutional investors’ algorithms to execute against. On the other hand, they also know that their institutional-investor clients are concerned about information leakage, and even before the publication of Flash Boys some (though not all) of them worried that high-frequency traders were predatory.

The main solution found so far by investment banks is to monitor the behaviour of participants in their pools, and to assure their institutional-investor customers that they are doing so. The monitoring is done by computers, of course. They check the frequency of ‘pinging’, as well as the extent to which a participant’s activity consists in adding bids and offers to the order book as distinct from taking up bids and offers that are already there. But the most important variable is a participant’s short-term profitability (its ‘alpha’). Short-term, here, is very short-term indeed: what’s monitored is typically what happens to the price of a stock in the single second following every transaction by every participant. If a program has bought shares and the price goes up, or if the program sells shares and the price goes down, that is positive alpha.

Egregious behaviour (hugely excessive pinging, or blatant efforts at manipulation) can get you expelled from an investment bank’s dark pool, but that’s rare, and generally won’t happen unless you have first been warned. If your alpha is persistently high, you risk being electronically stigmatised: you may be labelled ‘opportunistic’, ‘aggressive’ or even ‘predatory’. Participants in the pool can then elect not to let their algorithms trade with you. Not everyone goes for this option – ‘We have a lot of traders who say, “I’m smarter than anybody else, I’ll take whoever wants to trade with me,”’ one dark pool manager told me – but some participants do choose to steer clear of opportunists, and sometimes a manager will advise them to do so if their trading seems consistently to lose them money.

Schneiderman charges that in order to boost the volume of trading in its dark pool, Barclays overrode the results of its computerised monitoring: certain participants (including, allegedly, some of its own trading teams) should have been labelled opportunists, but weren’t. He also alleges that Barclays claimed not to ‘favour its own dark pool when routing client orders to trading venues, while in fact doing just that’. Barclays denies any wrongdoing, and is challenging both the factual accuracy of the allegations and the applicability of the Martin Act in this case.

Note that Schneiderman isn’t claiming institutional investors lost money because their orders were executed in Barclays’ dark pool, which would matter to anyone, say, whose pension or savings funds were being managed by an institutional-investment firm. Is the way the firm is doing its trading imposing unnecessary costs on those funds? You can’t really tell; neither, it would appear, can Schneiderman’s team. Indeed, I’m told that even a firm’s managers can’t always fully work it out. They get transaction cost reports from their broker-dealers, but there’s no adequate public database to which they can turn for verification or more complete analysis.

Just​ such a database has been proposed by a man called Stéphane Tyč, whose firm, McKay Brothers, supplies high-frequency traders with fast communications links. His suggestion is that a unique identification number be assigned to the ‘matching engine’ of each stock exchange and dark pool (the matching engine is the program that consummates trades). Each matching engine would have to report, in a publicly accessible file, anonymised details of every trade it consummated, along with the time of the trade, measured to within ten millionths of a second of the global time standard known as UTC (to assess whether information leakages are being exploited by ultra-fast trading, you need to measure time very exactly).

A public database of this kind would make it a lot easier to conduct a detailed independent analysis of the new world of electronic exchanges, dark pools and automated trading. Institutional investors could also demand from each broker-dealer the equivalent data file for the trades it has done on their behalf. By comparing that private file with the public database, one could find out how and where trading costs are being incurred by the institutional investor. The costs could even be reported to those whose savings are being managed. That would be an important step. It might, for one thing, spur them to ask just why the people managing their savings are doing quite so much trading.