The Market's Hidden Cards: Algorithmic Trading Strategies

Articles

Almgren R, Thum C, Hauptmann E, Li H. 2005. Direct estimation of equity market impact. (PDF, 251 KB)

Almgren R, Chriss N. 2000. Optimal execution of portfolio transactions. (PDF, 324 KB)

Coppejans M, Domowitz I, Madhavan A. 2000. Liquidity in an automated auction. (PDF, 108 KB)

Domowitz I, 2001. Liquidity, transaction costs, and reintermediation in electronic markets. (working paper; PDF, 173 KB)

Domowitz I, Finkelshteyn I, Yegerman H. 2008. Cul de sacs and highways: an optical tour of dark pool trading performance. Investment Technology Group. (PDF, 608 KB)

Engle R, Ferstenberg R. 2006. Execution Risk. NBER Working Paper No. 12165.

Grinold RC. 2007. Dynamic portfolio analysis. Journal of Portfolio Management 30: 12-26.

Lillo F, Farmer JD, Mantegna RN. 2002. Single curve collapse of the price impact function for the New York Stock Exchange. ArXiv.org. (PDF, 170 KB)

Mittal H. 2008. Are you playing in a toxic dark pool? A guide to preventing performance leakage. Journal of Trading, Vol.3, number 3, Summer.

Sofianos G, Jeria D. 2008. Quantifying the SIGMA X crossing benefit. In Street Smart, Goldman Sachs, (March 31).

Almgren R Chriss N. 1999. Value under liquidation. Risk (December; PDF, 149 KB).

Sponsorship

This meeting and eBriefing were made possible with support from:

The chips are down

The argot of Wall Street is thick with technical jargon. Behind the talk of options, alpha, and basis points, though, lies an enterprise strikingly similar to gambling. Traders constantly calculate odds and place bets, trying to beat the market the way a Vegas high-roller wagers against the house.

In this high-stakes game, the developers of quantitative trading algorithms are the card counters, closely watching and analyzing the market's behavior to pick the best bets. At first glance, the problem looks straightforward: trades on the open market are a matter of public record, and powerful computers capable of analyzing those data are relatively cheap. However, the market's players can hide some trades, making the game as subtle and risky as a hand of Texas hold 'em. On April 16, 2009, members of the Academy's Quantitative Finance Discussion group met to talk about the science of uncovering the market's hidden cards.

Probing the dark pools

Ian Domowitz of the Investment Technology Group explained, "Texas hold 'em ... is a game where most cards are actually displayed, and a few cards are hidden, and so everybody gets to figure out what's going on with the other people's hands." The key to winning in this popular poker variant is to understand the odds on other players' cards, and to place bets without revealing information about your own hand.

A similar challenge confronts traders who deal with dark pools. Despite their sinister sounding name, dark pools are a simple financial tool for solving a basic economic problem. If a trader needs to sell a large number of shares of a stock, the act of selling them will increase the market's supply of that stock, driving the price down. The trader could try to solve this by selling the stock off very slowly, or in very small increments, but holding a volatile stock too long can also be costly.

Dark pools minimize the market impact of large trades.

Dark pools, first developed in the 1980s, offer a better solution. "A dark pool is a venue for executing trades off the exchange in a completely anonymous and confidential manner," said Domowitz. Because the supply and demand in a dark pool are completely hidden from the broader market, selling a large quantity of stock in such an exchange should not cause its price to drop. Automated systems match sellers' offer prices with buyers' bids, without revealing the outcome of the trades to anyone else. Once shares have been traded in a dark pool, the buyer can hold onto them, sell them on the open market, or trade them in another dark pool transaction.

Until very recently, there were only a few dark pools in existence. The concept was profitable, and not patented or particularly difficult to understand, but starting a successful dark pool required an enormous amount of liquidity. Smaller pools were worthless, because traders could not trade large orders in them.

That all changed about two years ago, when a new crop of dark pools suddenly appeared. "The reason for the explosion was algorithmic trading," said Domowitz. Using new algorithms, traders can now take a large block of stock and move it through different dark pools, selling off chunks of the order in each pool. The algorithms can find a pool of liquidity very quickly, then sell as much of the order as possible there before moving on.

To determine whether this new landscape of dark pool trading is actually productive, Domowitz and his colleagues analyzed 12.6 million orders that were placed over the course of 2007. For some parts of the analysis, they also supplemented the data with another 8 million recorded orders, providing a robust statistical base.

The data reveal that dark pool trading decreases transaction costs compared to other strategies, even in highly volatile markets. That result was both timely and surprising. "The news is full of the fact that we are high volatility these days, and actually it is conventional wisdom that you don't use these algorithms to access dark pools in high volatility environments," said Domowitz, but he added that "the evidence suggests otherwise."

There are some pitfalls to using algorithms to trade in dark pools, though. Looking at the distribution of outcomes across many order executions, the team found that the algorithmic approach sometimes risked large losses. "That gain you get from using an algorithm to access these pools, although overall it's beneficial, has introduced risk into the equation that you didn't have before," said Domowitz.

The risk appears to come from slippage, or accidentally revealing one's hidden cards. Because the algorithms shop through multiple dark pools to complete an order, the strategy leaves tracks in the market that sharp-witted competitors could detect. Knowing that someone else is trying to push a large quantity of a stock onto the market, another trader could exploit that information and make money at the seller's expense.

Moving between dark pools reveals some information about a trade.

Traders themselves may exacerbate the problem by failing to understand how the algorithms work. For example, Domowitz says that traders may send part of a big order into a dark pool algorithm, and another part into the public market, without realizing that this compromises the algorithm. "It's almost like they took one of their hole cards and put it up with the displayed cards in Texas hold 'em, and then they wonder why there's information leakage," said Domowitz.

The team also compared the individual dark pools to each other, and found that different pools vary widely in execution quality. Large, deep pools can move orders of magnitude more shares in a given period of time than small, shallow pools, so by constantly moving through different pools, modern algorithms may actually do worse than traditional dark pool strategies. "No matter what time period I look at, no matter what order duration, I get very distinct differences in execution quality across pools, all of which are being touched by the same algorithm," said Domowitz, adding that "aggregation of crossing system liquidity through algorithms is not a panacea."

The portfolio that never sleeps

Lee Maclin of the Courant Institute of Mathematical Sciences broadened the discussion to cover the use of algorithms beyond dark pool trading. In a persuasive introduction, he outlined how algorithmic strategies have come to dominate everything from portfolio theory to market making. "Virtually everything that we know about modern finance is in fact related to algorithmic trading and optimal execution," said Maclin.

In an idealized model, a marketable buy (sell) order will disrupt the equilibrium of the market, causing the value of the traded stock to rise (fall). Once the trade is done, the market slowly recovers to a new equilibrium, which will inevitably be higher (lower) than the pre-trade price. Splitting the trade into two separate orders, and waiting for the market to stabilize between them, will instead cause two smaller price changes. The final impact on the market is still the same—the stock equilibrates at the same price after all of the trades are done—but the trader's average loss due to market impact are lower.

The theoretical market impact of splitting a large trade into two smaller trades.

Modern trading algorithms elaborate on that concept, allowing traders to parcel their orders in sophisticated ways to minimize their market impact. Meanwhile, portfolio managers can optimize their strategies with similar algorithms. Unfortunately, no algorithm can account for the conflicts that arise between the portfolio manager and trader.

"Traders don't see risk and reward the same way as portfolio managers do, meaning they're separate desks in many firms," said Maclin. Typically, a trader gets an order from the portfolio manager, but doesn't know anything about the portfolio it comes from. The trader's priority is simply to execute the order. "They just feel risk in having that order on their desk, and they want to reduce that risk over time, whereas the portfolio manager of course sees the risk to the entire portfolio, which is different," said Maclin.

Worse, firms generally provide incentives for traders to reduce their risk and for portfolio managers to maximize their returns; for either one, accommodating the other's needs could mean taking a pay cut. That raises a major barrier to adopting the optimum set of algorithms. "If you force people into this new framework, you're not going to be able to hire good traders or portfolio managers. Portfolio managers will not be willing to share risk, traders will not be willing to be benchmarked on someone else's performance on that trade," said Maclin.

The collapse of hedge funds has encouraged more passive strategies.

Overcoming that problem will be hard, but firms can still learn important lessons about portfolio management from the underlying theory. That will be especially important in the aftermath of a major shakedown in the money management field. "Recently we've witnessed a collapse of the hedge fund industry," said Maclin, adding that "in fact what we saw was that there weren't a thousand different strategies, there were only half a dozen different strategies, and when they lost money, they all lost money together."

One result has been a new appreciation for "passive," or low-return portfolios. In these strategies, managers allocate their assets in a particular way, then wait for some period of time, usually measured in days or weeks, then reallocate. According to Maclin's analysis, though, that's a bad approach: "You should be trading all the time—what you're doing by holding your positions constant is taking on more risk." Furthermore, the expected deltas of the portfolio—the changes required to move the portfolio from its current holdings to its new optimal risk/reward state—grow over time, eventually necessitating large, compressed, impact-heavy executions. Therefore, it can be said that these lazy portfolios ultimately increase both risk and costs.

A more insidious problem is that alpha, or expectation of a return, decays throughout the holding period. By holding stocks as alpha decays, passive portfolio managers are losing money. Instead, Maclin argues, managers should trade constantly. "The biggest gain to come out of the new framework is going to be when people take a realistic assessment of investment management returns and start applying the principles of ... continuous trading."

Open Questions

Can algorithms be structured to reduce slippage, or information leakage, when trading in multiple dark pools?

Are there any benefits to the proliferation of smaller dark pools?

Will passive portfolio managers be willing to adopt a more continuous trading strategy?

Can theoretical models for periodic markets be adapted to find optimal strategies in continuous markets?

How should companies recruit traders and portfolio managers willing to collaborate on optimal trading strategies?