Everything You Need to Know About High-Frequency Trading

Hello, fellow traders!

Surely, many of you have heard of such a concept as “high-frequency trading.” High-frequency trading has become a very popular topic over the past decade and has brought significant improvements to the markets. These improvements include reduced volatility, greater market stability, improved transparency, and lower costs for traders and investors.

Today I have prepared a lot of information for you about what high-frequency trading (HFT) actually is, how HFT systems are used in modern financial markets, the various HFT strategies, and the history and development prospects of this area of trading. Let's begin!

Let's first look at the history of the development of modern markets to understand the prerequisites for the emergence of HFT. Over the past couple of decades, consumer demand for computer technology has led to a significant drop in the price of trading equipment. As a result of advanced technologies and subsequent investment in software, trading platforms have become much more accessible and powerful. In addition, increased terminal resilience, greater reliability in the execution of orders, and the provision of platforms for connectivity and custom software development have led to an ever greater complexity in the trading process.

The figure above illustrates the main segments of the financial services market in the 1970s. Here is what the market looks like today:

In the 1970s, the main participants in the financial markets were institutions and large individual players, many of which still hold leading positions today. These were mainly various funds: pension funds, mutual funds, and hedge funds. Private traders, market makers, and various intermediaries also joined them.

Transaction costs were very high, while securities turnover was quite low. There was also a high probability of errors in order processing, since all orders were handled manually. Most traders in those days relied primarily on their own experience and intuition rather than on technical or fundamental analysis because the calculations were too complex.

Now let's take a look at the markets today. New participants successfully compete with financial tycoons, because these days high technologies, complex mathematical calculations and the construction of accurate models of market processes no longer seem like something fantastic.

The various funds use the latest economic and financial theories, as well as the latest mathematical tools, to make increasingly accurate predictions of price behavior in financial markets, which lead to increasingly efficient trading.Market makers,brokers and hedge funds are exploring the microstructure of markets and emerging technologies in developing automated HFT strategies to ensure low transaction costs while taking significant market share from traditional dealers. Funds dealing statistical arbitrage, also use quantitative algorithms, including high-frequency ones.

Nowadays markets are very democratic. Due to the spread of cheap technology, anyone can trade on real markets, place orders and thereby participate in the formation of the price of an asset, which was previously strictly the privilege of dealers. With all this automation of the trading process virtually eliminates the possibility of errors when executing trading operations. Strong competition between new entrants and old players has also led to a decline marginal requirements from brokers.

Here's how the trading process happened in the 70s:

Nowadays, clients are often better informed about market analysis and equipped with more modern equipment than the brokers themselves. The scope of brokers' competence has also narrowed significantly. Here is a modern algorithm for broker-client interactions:

Steve Swanson was a typical 21-year-old computer geek. It was the summer of 1989, and he had just received a mathematics degree from the College of Charleston. When it came to clothing, he was drawn to T-shirts and flip-flops, and to television, the Star Trek series. He spent most of his time in the garage of Jim Hawkes, a statistics professor at Steve's college. There he programmed algorithms for what would become the world's first high-frequency trading company, called the Automated Trading Desk. Hawkes was obsessed with the idea that he could make a profit in the stock markets by using formulas for predicting price behavior developed by his friend, David Whitcomb, who taught economics at Rutgers University. Swenson's task was to turn Whitcomb's formulas into machine code.

A satellite dish mounted on the roof of Hawkes' garage picked up signals carrying updates.quotes, receiving which, the system could predict the behavior of prices in the markets within the next 30-60 seconds and automatically buy or sell shares. The system was called BORG - short for Brokered Order Routing Gateway, Brokerage Team Routing Gateway. The name also referred to the Star Trek series - or more precisely to an evil alien race capable of absorbing entire species, turning them into parts of a single cybernetic mind.

One of the first victims of BORG were market makers on the floors of the stock exchange, who manually filled out cards with information about buying and selling shares. ATD not only knew better who gave the more attractive price. The new system carried out the process of buying and selling shares in a second. By today's standards, this is a snail's pace, but then no one could surpass it. As soon as the stock price changed, ATD computers began trading on conditions that other market participants had not yet had time to adjust, and a few seconds later ATD sold or re-bought shares at the “correct” price.

ADT averaged less than a penny per share, but the company handled hundreds of millions of shares a day. As a result, the firm was able to move from Hawks' garage to a modern $36 million business center in a swampy suburb of Charleston, South Carolina, about 650 miles from Wall Street.

By 2006, the company was trading approximately 700-800 million shares per day, representing more than 9 percent of the entire U.S. stock market. And it had competitors. A dozen other major electronic trading firms entered the scene: Getco, Knight Capital Group, and Citadel grew out of the trading floors of the Chicago commodities and futures exchanges and the New York stock exchanges. High-frequency trading began to gain momentum.

The largest stock exchanges in the world are in a state of fierce competition and are too dependent on the interests of investors who expect constant growth in profits. As a result, exchanges are forced to look for non-standard marketing solutions and ways to stand out among competitors. Let's look at what allows the world's leading exchanges to develop.

Australian Securities Exchange (Australian Securities Exchange—ASX)

The main goal of the Australian Securities Exchange (ASX) is to maintain a dominant position in the Australian securities market. In addition, the ASX is committed to listing securities of companies from Southeast Asia. Low costs and consistently high sales performance make the Australian Securities Exchange competitive in the global financial system.

In 2005, the ASX gave brokers the ability to trade anonymously. The initiative helped to significantly increase liquidity securities - in particular, shares included in the S index&P and ASE, which account for more than three-quarters of the total market value.

Other ASX initiatives include opening a secondary market similar to the London Stock Exchange's Alternative Investment Market (AIM) for companies with a market capitalization below A$100 million (which is two-thirds of the AIM).

German Stock Exchange (Deutsche Börse)

The German Stock Exchange seeks to differentiate itself by creating a unique portfolio of services that covers the entire chain of exchange processes, such as securities and derivatives trading, settlement and closure of transactions, provision of up-to-date market information, development and operation of electronic trading systems. Thanks to its exchange-oriented business model, Deutsche Börse creates an efficient capital market: issuers benefit from low capital costs and investors benefit from high liquidity and low transaction costs.

European stock exchange Euronext

The European stock exchange Euronext (now part of the world's largest exchange NYSE Euronext) was formed as a result of a large-scale merger of the stock exchanges of Amsterdam, Brussels and Paris and subsequently expanded to include the Lisbon stock exchange, LIFFE and the London Financial Derivatives Exchange.

Euronext was created with the aim of dividing spheres of influence in Europe and jointly controlling the three original securities markets. Contrary to the agreements, Paris took the lead in most areas of Euronext's activities. The exchange now uses the original French electronic trading system. In addition, most major French privatizations take place on Euronext.

Euronext is pursuing a strategy of diversification and expansion, adding new products and services and seeking to increase its influence internationally. Euronext analysts have developed a “Technological Improvement Program” similar to the system operating on the London Stock Exchange. The new electronic platform will help Euronext significantly increase the speed and number of simultaneous transactions.

The Hong Kong Stock Exchange (HKEX)

The Hong Kong Stock Exchange (HKEX) is inextricably linked to China's rapidly developing economy. HKEX's main advantages are its geographical proximity to mainland China, relatively lenient corporate governance and the favor of the Chinese government, which is conducting privatizations of state-owned enterprises here.

Chinese companies trust the Hong Kong Stock Exchange and prefer it to Western and American competitors. It is more convenient, cheaper and culturally easier to list securities on the Hong Kong Stock Exchange. Listing standards on HKEX are high, but the requirements for companies are not nearly as stringent as in the United States, as the exchange's management has repeatedly stated.

Currently, only securities of companies registered in Hong Kong, China, Bermuda or the Cayman Islands can be listed on the exchange. However, the Hong Kong Stock Exchange's marketing strategy involves changing the rules to include shares of companies from other countries in the Asia-Pacific region (for example, Australia) and reduce dependence on China.

London Stock Exchange (LSE)

The London Stock Exchange is spending a lot of money on its Technology Road Map, a massive program to modernize its trading mechanisms. One of the latest successful steps in this direction was the introduction of a new system for storing and transmitting market data Infolect, which made it possible to reduce the average speed of a transaction to two milliseconds (which is approximately 15 times less than previously required).

Like Euronext and NASDAQ, the London Stock Exchange is looking to expand its influence around the world. LSE's focus is on China, India and Russia. The strategy of attracting foreign companies to the listing procedure really works - in 2006, several large Russian private enterprises listed their shares on the London Stock Exchange. Management also decided to open an additional office in Hong Kong in October 2004 to compete with US exchanges for Chinese business.

US stock exchange NASDAQ

NASDAQ is the largest electronic stock exchange in the United States based on the number of securities trades closed and the presence of companies that are leaders in their industries - for example, shares of Microsoft, Intel, Google, Oracle, Nokia, K-Swiss, Carlsberg, Starbucks and Staples are traded on this exchange. Despite the fact that NASDAQ initially positioned itself as an “exchange for growing companies,” today it faces some of the most stringent requirements for applicants to be listed on the exchange.

The world's first electronic stock market, NASDAQ strived to become a leader in trading technology. The speed of transactions was reduced to a record low of one millisecond.

New York Stock Exchange (NYSE Euronext)

NYSE Euronext positions itself as the world's leading stock exchange. The most liquid stocks, the highest standards of listing and blue chips (securities of the largest companies with stable income) allow the New York Stock Exchange to maintain its secret gold status.

Like all major stock exchanges around the world, NYSE Euronext is looking to expand its reach beyond the US and overcome the competition of local exchanges that have grown over the past few years in large metropolitan areas (for example, Milan or Mumbai).

In order to gain access to shares of companies located outside the United States, in June 2005 the New York Stock Exchange proposed merging with Euronext, one of the largest securities exchanges in Europe. The merger, approved by Euronext shareholders, took effect in the first quarter of 2007 and for the first time created an “intercontinental” securities market, with the total value of listed companies amounting to approximately 26 trillion dollars.

Singapore Stock Exchange (SGX)

The Singapore Stock Exchange has carved out a niche in the Asian securities market. The largest companies from the countries of the Asia-Pacific region are represented here (except for Japanese, Korean and state-owned Chinese enterprises, which prefer to list their shares for sale on domestic exchanges). SGX is an extremely attractive trading platform for countries that do not have a globally recognized exchange. In addition, the Singapore Stock Exchange has succeeded in attracting private Chinese capital.

In order to maintain its competitive advantage, the Singapore Stock Exchange seeks to cooperate with smaller regional exchanges and thereby expand its global network of trading platforms. In mid-January 2007, SGX became the only Asian exchange to officially announce its desire to acquire a 26% stake in the Bombay Stock Exchange. The other three exchanges competing for Bombay shares, NASDAQ, London Stock Exchange and Deutsche Börse, are based in North America and Europe.

Japan Exchange Group, Inc

The Japan Stock Exchange strives for regional leadership and global competition and positions itself as a "prestigious listing destination." The significant advantages of Japan Exchange are the sale of securities with high liquidity and the introduction of cutting-edge information technologies into the trading process.

The exchange is a member of the Federation of Stock Exchanges of Asia and Oceania. Japan Exchange Group, Inc resulted from the merger of the Tokyo Stock Exchange and the Osaka Stock Exchange in 2012. Before this, the main player in the Japanese stock market was the Tokyo Stock Exchange (it absorbed the Osaka Stock Exchange).

Moscow Exchange

The Moscow Exchange was formed in December 2011 as a result of the merger of two main Russian exchange groups - MICEX and RTS. The exchange structure that emerged as a result of the merger gained the ability to trade in all major categories of assets.

At the moment, the Moscow Stock Exchange is the largest stock exchange in Russia and Eastern Europe. In addition, the share of post-trading services on the Moscow Exchange is increasing, which, according to management, can attract new issuers and investors. Armed with the experience of American competitors, MB began to provide management services risks and provide investors with business information. Trading mechanisms are being modernized, and the speed of transactions on the derivatives market is increasing.

The term "hft" includes a wide range of operations from algorithmic trading. High-frequency trading is a fairly closed area. It is very difficult to find information about how HFT firms work. However, some information can still be obtained from lists of open vacancies, advertisements and individual Internet articles. HFT is also very different from other forms algorithmic trading. It is based solely on technical solutions and a huge number of calculations. Once trading is launched using a specific algorithm, virtually no adjustments are made to its operation (as long as it remains profitable), which is very different from low-frequency systemic trading, in the process of which people often make their own adjustments.

Working in such an environment is highly competitive and can often break people down. Many months of research become irrelevant overnight if the exchange's operating structure changes, a new legislative framework appears, or if competitors are able to start processing data at higher speeds. Therefore, this type of work is suitable for well-disciplined people with several higher technical educations, able to work under pressure, who value independence and a highly professional team.

Despite the fact that the activities of HFT traders are often criticized, only certain types of HFT trading create chaos in the modern financial market. The line between algorithmic trading, electronic market making and harmful HFT trading is quite blurred, and high-frequency trading often refers to electronic trading. In fact, the phenomenon of HFT trading in itself is neither good nor bad, but the devil is in the details.

To clearly understand the possibilities of HFT trading, it is worth taking a closer look at some types of market activities.

Algorithmic/systemic trading is the general name for the process of using programmable systems that use a specific mathematical model to automatically execute trades. A person creates a program on a computer for a specific financial strategy based on a given criterion and controls the developed system from this computer. HFT trading is a type of algorithmic trading, but not all algorithmic trading can be considered high-frequency.

In 2011, the Commodity Futures Trading Commission (CFTC) admitted that it was not trying to come up with a precise definition of high-frequency trading. Instead, she proposed seven main signs of HFT trading:

Many people now complain that high frequency traders who use mathematical algorithms have an unfair advantage over those whose algorithms are not as good, or that their (hft traders) trading systems are faster than other players.

This dissatisfaction underscores a larger historical fact: any technology that increased the speed of information flow was immediately adopted by the trading community in both Europe and the United States. Traders have used every known vehicle to execute trades faster and with less effort. They were among the first to master high-speed boats, faster crews and private couriers.

In the late 1830s, Philadelphia broker William Bridges operated a personal signal station between New York and Philadelphia that relayed stock market news to him and his patrons (and no one else). Signals were transmitted using an "optical telegraph", which consisted of a series of shields on a pole mounted on a hill, which could be seen through a telescope. Reports indicate that they could transmit stock market information anywhere from New York to Philadelphia in 10 to 30 minutes. In the 1830s this was high speed trading.

It is not surprising that complaints began to come from speculators in New York, who were not involved in this system and who had until then enjoyed a significant advantage. When the system was shut down after the advent of the telegraph in 1846, a local newspaper report wrote that “many of the ingenious moves in the Philadelphia stock and commodity markets were responsible for the speculators who contributed to the creation of the telegraph. No doubt the speculators paid its creators well."

Unfortunately, the organized trading community was not very keen on openness. In its early days, the NYSE (New York Stock Exchange, then known as the New York Stock and Exchange Board) did not allow the public to listen in on trading sessions (sessions were not available to the public until 1869). Competing traders (over-the-counter traders working literally from the outside) who intended to sell trading information on the NYSE were furious that they could not be near the exchange. In 1837, the NYSE discovered that over-the-counter traders had drilled a hole in the brick wall of the exchange building in order to eavesdrop on trading.

While the public was wondering how to get ahead of the fast horses, a new technology appeared on the scene that turned trading into a truly high-speed area: the telegraph, which came into use after 1844. He was the greatest invention of his time. Newspapers took time to produce and were mostly released at regular intervals. But the telegraph worked constantly, and it could be used for personal communication.

As expected, the use of the telegraph to transmit “secret knowledge” caused outrage. Several inventors of the early telegraphs were forced to stop their experiments by warnings that they might be persecuted for disseminating information faster than mail. The telegraph's leading inventor, Samuel Morse, supported the introduction of the telegraph into mass production for personal and public purposes, particularly to protect it from being used for profiteering purposes.

Forty years later, the telegraph was still the main tool of stock market speculators. In 1887, the president of Western Union stated that 87% of the company's income came from speculators in the stock and commodity markets and those who made money from horse racing.

Introduced in 1867, the stock ticker became the next great electronic device and was immediately adopted by the trading community. Before its appearance, stock exchange transactions, as a rule, were carried out with the help of “runners” - boys who ran from the stock exchange pit to brokerage houses. It was hugely superior to the telegraph for several reasons: traders no longer needed to be physically present in the trading pit, its introduction reduced transaction costs, it helped disseminate information continuously in real time, and its invention eliminated the number of pesky middlemen like telegraph companies and newspaper editors. Not surprisingly, journalists and editors became concerned that the introduction of the ticker would force them out of the lucrative trading of financial news.

The first stock pit was patented by Reuben S. Jennings in 1878. He designed the pit in such a way that traders could see and hear other traders in the best possible way. Therefore, there were several steps in the pit. The trader at the very top had the best visibility and the advantage of being able to easily see and hear his colleagues, all of which allowed him to execute trades faster.

To gain a speed advantage, one had to be physically taller than other traders, so height began to play an important role in reducing delays. That’s why former basketball players often became traders: it was easier to notice them. Already at the end of the twentieth century, some pit traders wore high heels to stand taller and execute trades faster.

This has led, for example, to falls due to lack of balance when walking in high heels. As a result, the Chicago Exchange was even forced in November 2000 to decide to set a maximum heel and/or platform height of two inches (just over 5 centimeters), and, for example, the London Metal Exchange still has a rule according to which transactions can only be concluded while sitting.

Introducing such standards was one way to level the playing field in terms of speed on the trading floor, but in the end there was always someone who would beat the competition. Those traders who were most successful in reducing trading delays profited from the inefficiencies of the then existing system, in which growth could provide a big advantage, for example.

As a result of their work, these inefficiencies were gradually leveled out - somewhere by the introduction of regulatory rules, somewhere by the very course of history - for example, the computerization of exchanges itself made the desire to be physically superior to everyone else simply irrelevant.

This continued with the invention of the telephone (first tested by Bell in 1876, by 1878 the NYSE already had its own telephone), with the creation of pneumatic mail, the computer in the 1950s, punch cards in the 1960s, the first advent of electronic trading when the Nasdaq exchange began operating in 1971, and algorithmic trading in the 1990s. The same disputes arose: someone received information before others, and some could trade because they had a faster ship, horse, carriage, telegraph line, computer communication, algorithm.

In 1967 Edward Thorpe, a professor of mathematics, published the book “Beating the World.” The author described a method by which one could make money in the stock markets. The system he invented was so good that some trading houses had to change their trading rules.

Later in Britain, the developments of mathematicians brought new methods of analysis and the belief that in the future computer systems could make a real revolution in predicting market fluctuations. Then a completely new branch of science was born - quantitative analysis.

In 1989, with the advent of newer technologies and computer systems, the idea of high-frequency trading was born as a method of using high-performance systems to make money on trading exchanges. The author of this idea is Steve Swanson.

He worked on analyzing the movement of quotes on stock exchanges 30 seconds before a transaction. At the same time, he and his partners David Whitcomb and Jim Hawkes founded the first and only automated trading company at that time - AutomatedTradingDesk. While all financial market participants worked via telephone, the order processing speed through AutomatedTradingDesk was one second. This is how the history of HFT began. As a result, 70% of trades on Wall Street these days are carried out by high-frequency algorithms.

Today, trading is typically carried out using electronic servers in data centers, where computers exchange offers to buy and sell by transmitting messages over a network. This shift from back-office trading to electronic platforms has been particularly beneficial for HFT companies, which have invested heavily in the infrastructure required for trading.

Even though the place and participants in trading have changed a lot in appearance, the goal of traders, both electronic and traditional, has remained the same - to purchase an asset from one business or trader and sell it to another business or trader at a higher price. The main difference between a traditional trader and an HFT trader is that the latter can trade faster and more often, and the portfolio holding time of such a trader is very low. One operation of the standard HFT algorithm takes a millisecond, which traditional traders cannot match, since just blinking in humans takes approximately 300 milliseconds.

Brokers who were vocal against HFT tended to rely on technical analysis when deciding when to enter or exit a position. Technical analysis was one of the earliest methods to become popular with many traders and in many ways it is a direct predecessor to modern econometrics and other HFT methods.

Technical analysts, who came into vogue in the early 1910s, sought to identify recurring patterns in prices. Many techniques used in technical analysis measure current price levels relative to moving average prices or combinations of moving averages and standard deviation of prices (Bollinger Bands).

For example, a technical analysis indicator such as MACD, uses three exponential moving averages to generate trading signals. Advanced technical analysts look at prices in conjunction with current market events or market conditions to get a better idea of where prices might be headed next.

Technical analysis flourished in the first half of the 20th century, when trading technology was in its infancy and the complexity of trading strategies was much lower than it is today. The speed of dissemination of information and quotes, among other things, was amazingly low. The previous day's trades did not appear in the newspaper until the next morning. In the post-war years, technical analysis became a self-fulfilling prophecy.

If, for example, enough people believed that the figure "head and shoulders", a huge number of traders began to place sell orders, realizing the prediction in this way. Currently, classical technical analysis works well only on timeframes from D1 and above. And yet, many technical analysis methods and indicators are used by quants to build high-frequency trading strategies.

Scientifically proven that investors tend to trust more strategies that have worked in the past. It also follows common sense that what worked before will probably continue to work. As a result, vehicles operating last month are also likely to operate next month, forming a trade trend, which can be detected using simple technical indicators based on a moving average, as well as more complex quantitative tools. Quite often, quants use the Bollinger Band indicator in their strategies to track the current market conditions.

Another type of analysis, Fundamental Analysis, originated in the stock market in the 1930s. Traders have noticed that future cash flows, such as dividends, influence market price levels. Graham and Dodd (1934) were the earliest traders to use this approach, which remains popular to this day. Fundamental analysis developed throughout most of the 20th century. In equity markets, fair prices are still often determined based on forecasts of companies' future earnings.

On the market forex The most common are macroeconomic models, which calculate fair prices based on information about inflation, trade balances of various countries and other economic indicators. Derivatives are traded primarily through advanced econometric models that incorporate statistical properties of the price movements of the underlying instruments. Various aspects of the application of fundamental analysis are also used in the construction of HFT systems. The date and time of news release are usually known in advance, and the information necessary for making a decision is disclosed during the announcement of the news.

It is absolutely clear that in such a situation, the systems that respond most quickly to changes receive the maximum profit. In fact, speed has become the most obvious aspect of competition. To speed up the process of executing transactions, traders began to use more and more powerful computers and use more and more advanced technologies.

By 2012, there was a trend towards a decrease in the efficiency of HFT and its market share. Since 2009, in just three years, the volume of profits from high-frequency trading has decreased 5 times from $5 billion to $1.25 billion. In 2014, the book “Flash Boys: The High-Frequency Revolution on Wall Street” was published, detailing the history and mechanisms of HFT as financial fraud and market development. The product became a bestseller, its author is Michael Lewis. In 2016, due to low volatility, most of the smaller HFT companies began to leave the market. Their profits have become incomparable to what they were in 2009-2010.

For the successful implementation of the HFT system, signal-generating algorithms, algorithms that optimize order execution, risk management algorithms, and optimization are required.portfolios and so on. The figure below illustrates a survey of traders conducted by Automated Trader in 2012. Here's how traders responded about the purposes for which they use automated trading systems:

HFT systems cover almost the entire range of decisions a trader makes, from selecting trading instruments to achieving the best possible execution of orders.

And yet, even today, not all markets are suitable for high-frequency trading. According to researchb conducted by Aite Group, equity markets have the largest percentage of algorithmic participants, who account for more than 50% of trading volumes. Futures are in second place (over 40%). The share of algorithmic traders in the Forex market, options markets and fixed income markets is noticeably lower.

Algorithmic trading has been shown to outperform human trading in several key metrics. Aldridge (2009), for example, shows that algorithmic funds consistently outperform traditional funds. Aldridge (2009) also shows that algorithmic tools outperform classical ones in returns during periods of crisis.

Interesting study was carried out by the Central Bank for the Russian stock market and currency pair USDRUB. According to it, 50 HFT algorithms work on this currency pair, providing more than half of the order volume.

This may be due to the lack emotions inherent in algorithmic trading systems compared to human beings driven by emotions. In addition, computers are superior to humans in such basic tasks as collecting information and quickly analyzing multiple data and news. Physiologically, the human eye cannot capture more than 50 data points per second. In modern films, the human eye is exposed to only 24 frames per second. Even then, most static images displayed over successive frames appear to us to be continuously moving objects.

For comparison, modern price flow includes rapidly changing quotes, the number of which can easily exceed 1000 per second per financial instrument. You need to be able to quickly process all this information, make various calculations and make trading decisions based on them.

In the spring of 2017, Credit Suisse bank analysts published a report on “the real role of HFT trading in the modern financial market ecosystem.” The document talks about how high-frequency trading has changed the landscape of world exchanges. Here are the study's four main findings.

Trading volumes have increased

The development of high-frequency trading technologies has had the largest, most noticeable and long-lasting impact on trading volumes. Credit Suisse estimates that the trading volume of fiduciaries and investors, both active and passive, in the US stock market has remained virtually unchanged over the past ten years (3-4 billion shares per day).

At the same time, the total trading volume on US exchanges more than doubled in the period after the 2008 crisis; it was during these years that HFT trading developed especially actively.

This fact also has negative consequences. For example, the topic of “fake” trading activity is widely discussed robots, which may place multiple bids and then cancel them outright in hopes of influencing the price. However, overall, Credit Suisse analysts believe that “much of the HFT activity helps connect people in the financial market, reducing the time it takes for a counterparty to wait.”

The difference in prices for buying and selling shares has changed

In theory, the less spread, the better for the market. The development of HFT has had an impact here too. The size of spreads for shares of large companies has decreased, while for smaller companies, on the contrary, they have increased. This suggests that high-frequency traders are more often interested in more liquid shares of well-known companies.

According to a report by Credit Suisse, equity spreads move in line with volatility, and the dispersion of spreads between the most and least liquid stocks has increased significantly since 2009. That is, spreads between large and small company stocks no longer move in the same direction.

Stocks of large and small companies are volatile at different times of the day

Increased volatility in shares of large and small companies in recent years has been observed at various periods of the trading day. For example, at the beginning of trading, the price of shares of not the largest companies changes more actively. This happens because it takes more time to determine a fair (at the moment) price for such shares. However, by the end of the trading session, on the contrary, such shares behave calmer than securities of large organizations.

In contrast, shares of large companies that are actively traded in the market sometimes experience price fluctuations where they move rapidly within the spread multiple times at the end of the trading day. Analysts also attribute both of these phenomena to hft.

The number of noticeable spikes in the prices of shares of large companies has decreased

Typically, HFT trading strategies are aimed at profiting from market inefficiencies rather than participating in large price movements. This results, among other things, in a reduction in large price fluctuations of well-known companies, with which high-frequency traders more often transact.

Often, an extra millisecond can lead to a trader receiving a loss instead of a profit, because his trading robot was ahead of someone else. The pursuit of speed and the financial bottom line at stake has led to the rapid development of various technologies to reduce trade delays. Here are some of the approaches used to improve performance.

Direct access to the exchange

To trade on the stock exchange, an investor must enter into an agreement with a broker who provides access to trading. Typically, such companies also develop their own trading systems, which process customer orders before sending them to the core of the exchange system. However, in a situation where everything can be decided in a few milliseconds, the “user - brokerage system - exchange” scheme is not suitable for everyone.

To remove the unnecessary link in the form of a brokerage system, the technology of direct access to the exchange (DMA, direct market access) was created. Its essence lies in the fact that the application is submitted directly to the exchange’s trading system, bypassing the broker’s infrastructure.

Direct access is a technology for high-speed access to exchange platforms, in which an application is submitted to the exchange's trading system directly, bypassing the broker's trading system. All this allows you to significantly reduce the time it takes to deliver an application to the exchange and obtain information about its status.

With such an organization of the trading process, a trader can count on a significant gain in time. For example, when directly connecting to the stock and foreign exchange markets of the Moscow Exchange, the application processing time is reduced to 0.5 ms, and on the FORTS and Standard markets it does not exceed 3 ms. When using a brokerage system, orders are processed in a time from 5-10 ms to 150-500 ms, depending on the brokerage system, market type and connection method. Through brokerage systems, orders are processed 10-100 times slower than with a direct connection (although this speed is quite satisfactory for many traders).

We are still talking about very short periods of time from a human point of view, but for some trading strategies such a difference can be critical and affect their overall performance. Naturally, using direct access technology costs more, often significantly, and is only suitable for those who perform a large number of transactions per day and are willing to pay for their speed.

Despite the fact that technically, thanks to direct access to the exchange, traders can carry out trading operations without going through a broker, documented access is still provided through broker companies. That is, in order to get the opportunity to directly trade on, say, the stock market of the Moscow Exchange, an investor needs to enter into an agreement with a broker and purchase the service of direct access to the exchange from him.

Placing equipment near the exchange

If we continue to move along the chain of reducing the time for performing transactions, it becomes obvious that it is necessary to place the trading robot not only logically, but also physically as close as possible to the servers with the core of the exchange trading system.

Direct access to the exchange allows you to logically bring the trading system closer to the core of the exchange, but it is obvious that you can get even greater speed gains by placing it physically closer to this end point. As a rule, exchanges provide equipment colocation services in their data centers. In this case, the trading system can be launched on a server that is actually located in the same rack with the exchange core servers.

The robot can be located either on a separate server, which can be placed in a rack in a data center (this service is called Colocation), or on a virtual machine (Hosting), which in turn is launched together with the virtual machines of other clients on a server also installed in the data center, next to the exchange servers. Hosting services are, as a rule, provided only by large brokers with their own racks in data centers.

Placement in the exchange colocation zone allows you to connect trading robots directly to the exchange core. Also in this zone it is possible to obtain market data (Market Data) using the FAST protocol, which we will talk about a little later. The advantage of using a free colocation zone is the fact that this option is much cheaper. But when it comes to the pursuit of speed, high-frequency traders choose the fastest option, even though it can sometimes be the most expensive.

Placing in the exchange colocation zone has obvious advantages: virtual machines and servers connect directly to the core of the exchange, while from the free zone the connection goes through intermediate servers. In addition, receiving data via the FAST protocol and a dedicated channel for connecting to the market are available only from the exchange colocation area.

One way to significantly reduce infrastructure costs is to place the robot in a free colocation zone. The services provided in it are almost similar to the services of the exchange colocation zone. However, there is only free cheese in a mousetrap - you will have to pay for the relative cheapness of deploying a robot with a few milliseconds of increased transaction processing speed.

In addition, since the interfaces for creating direct connection software do not initially imply any graphical capabilities for displaying information about trading, the ability to synchronize orders and positions formed on a direct connection with the broker’s trading system in real time is practically a necessary thing for monitoring trading operations. Therefore, many brokers try to provide their clients with such opportunities.

All this, in comparison with regular access to the exchange through brokerage systems, costs money, and quite a lot. However, for those investors who have reached a certain level of income, such spending makes sense. According to representatives of the exchange, the owners of robots that won the “Best Private Investor” competition in 2011 spent from 100 to 500 thousand rubles per month on services related to direct access. However, taking into account the fact that some traders (although there were not so many of them) managed to reach a profitability of over 8000% and earn millions of rubles per month and, taking into account all the commissions of the broker and the exchange, these expenses paid off quite quickly.

Hardware acceleration (FPGA)

In the last few years, the use of FPGAs has become widespread among algorithmic traders to reduce latency in trading applications. Modern FPGAs can implement various aspects of high-speed trading systems. For example, market data processing can be done entirely on the FPGA without transferring it to the machine's processor.

Using programmable hardware allows you to get serious gains in processing speed and reduce latency, but there are also some difficulties. First of all, these include the complexity of developing and supporting trading applications using FPGAs. To interact with hardware, traders need to master not only high-level programming languages, but also the so-called hardware description languages (HDL, hardware description languages). Also, do not forget about the need for additional expenses on the equipment itself.

New data transmission technologies

The most important component of success in high-frequency trading is data transfer speed. Market players are actively looking for various ways to optimize it, which leads to the development of technologies such as data transmission using microwave radiation. Despite certain disadvantages (unresistant to rain and fog, limited bandwidth), it makes it possible to send data directly. In other words, you don’t need to lay a fiber optic cable through the mountains, you can simply install antennas on towers and find the shortest distance between point A and B. Thus, requests can be transmitted over the air faster than through fiber optics.

Such technologies are quite expensive, but the possible financial return from their use is so great that many HFT companies are investing millions in building their own microwave networks.

However, the use of microwaves to transmit data is not the only innovation. Like the Wall Street Journal wrote back in 2014 - the next technological breakthrough in this area could be the creation of data transmission networks using lasers. According to journalists, HFT companies have already agreed to create a similar network to work with the Nasdaq exchange.

In general, the direct access scheme is as follows: the server with the trading robot is connected to an intermediate server, which is located as close as possible to the core of the exchange trading system. This server has special software installed - the so-called gateways, which are used to transmit orders and market information directly to the trading system. At the same time, various protocols and connection methods are used to perform operations and receive data.

Currently, exchanges offer the following protocols for developers to directly access exchange markets:

ASTS Bridge

The exchange gateway is Bridge. Gateway, which I also really want to translate as a gateway, is another story, it’s more of an access server. Gateway is a native protocol of the ASTS trading and clearing system, existing since 1998 (previously the solution was called TEAP (TCP/IP version) or TEServer (RS-232 version, no longer supported)). Many developers know the protocol under the name MTESRL, after the name of the corresponding DLL. Due to the native nature of this protocol, its main feature is the support of all transactions and all market data from all markets operating on the ASTS trading and clearing system.

The use of this protocol is recommended primarily for those who need access to clearing data and operations (viewing their positions, obligations, risk parameters, setting various kinds of limits, transferring securities and money between accounts, and so on), as well as participating in trading in negotiated transaction modes (that is, not quick anonymous trading in the order book, but direct conclusion of transactions with a specific counterparty). The API is provided as a dynamic library - in 32- and 64-bit versions for Windows and Linux.

The connection architecture is as follows: the dynamic library is included in the package of the software you developed, this package is installed on a server that has network access to the so-called server part of the gateway. The server part is a kind of proxy server, which is located at the broker and is connected to the exchange infrastructure via dedicated network channels.

In the case of HFT trading, when your software installed in the exchange data center under colocation conditions, an intermediate link in the form of a gateway server is no longer required - you connect directly to the exchange Gateways.

An interesting feature of the gateway protocol is its support for “interfaces”. An interface is a versioned set of tables and transactions available to the user, with the corresponding structure and data types. With almost every update of the trading and clearing system, new opportunities appear for users that require modifying the table structure or changing the transaction format. The presence of versioned interfaces allows users who are not ready for changes to remain on the old version of the interface and not modify their software.

Protocol Plaza II

For direct connection, native protocols are used. These protocols arose even before the merger of the MICEX and RTS exchanges into the Moscow Exchange. Thus, in the markets belonging to the RTS exchange (FORTS - futures and options, Standard), the Plaza II protocol is used to directly carry out transactions and receive data in connection mode.

To connect using this protocol, the exchange provides the CGate API. On the one hand, this allows trading participants to implement full-fledged functionality for accessing trading, including clearing functionality for limiting sections, setting restrictions on instruments and viewing market maker obligations. On the other hand, this allows clients of trading participants to implement their own high-speed robots with a minimum set of functions (place an order or withdraw an order). The API is provided as a set of dynamic libraries - in 32- and 64-bit versions for Windows and Linux.

With almost every release of the derivatives market, the exchange makes changes and improvements to its own program code, which is transmitted to clients in the form of an API. To the user, it looks like a new distribution with new versions of the libraries inside. In addition to the code itself, the structure of the data provided to users changes periodically along with the release.

FIX protocol

The FIX protocol (Financial Information eXchange) is a financial information exchange protocol that is a global standard for the exchange of data between exchange trading participants in real time. Supported by the world's largest exchange platforms, including the Moscow Exchange and all Forex market brokers.

The creation of the FIX protocol was initiated by a number of US financial organizations in 1992 - brokers and investment funds wanted to speed up the process of carrying out trading operations on the exchange. At that time, a significant part of trading transactions were carried out using the telephone, and the FIX protocol made it possible to transfer interactions to electronic form.

As a result, an open standard for transmitting information electronically was born, which is not controlled by any of the large organizations. Today, FIX has become an industry standard that is used by financial market participants in different countries to link their products.

Currently, the protocol is defined at two levels - session (working on data delivery) and application (describing the content of the data). There are two protocol syntax options - traditional, like Tag=Value, and in XML format (FIXML).

Work on creating the XML syntax began in 1998, and the first version of FIXML appeared in January 1999. At the beginning of the XML version of FIX, only the DTD syntax definition mechanism was used. Subsequently, the W3C organization developed a new mechanism - XML Schema, which forced the FIX developers to adapt the standard to use this syntax option.

This step allowed us to improve the XML version of the FIX protocol, in particular, users were able to add attributes and contextual abbreviations to messages. The basic organization of XML schema involves the data types used in the fields, which are contained in a separate file. FIX fields are defined in a special shared file, and components and FIXML syntax elements are defined in special component files. FIXML messages are defined using special files that indicate the category.

FAST protocol

In November 2004, the then CEO of the financial holding Acrhipelago Holding, Mike Cormack, at a FIX community conference called FPL (FIX Protocol Limited) in New York, stated that the current version of the protocol could not cope with the increased volume of financial information generated in the stock market. When transferring large volumes of data using FIX, there were significant delays in their processing, which brought losses to traders and deprived them of the opportunity to develop effective trading strategies.

The classic Tag=Value message passing format used in FIX turned out to be too cumbersome to process quickly. Soon after this speech, the first steps were taken to correct the situation.

When creating the FAST protocol, the developers pursued the goal of achieving the ability to transfer large amounts of data, avoiding delays in receiving information. The development of the protocol was carried out by a working group of the FIX community called the Market Data optimization working group (mdowg), which was formed in 2004.

In 2005, the group’s experts presented a pilot project (proof of concept) of the protocol, and a year later the first version of FAST 1.0 was released. Subsequently, several updates were released, and currently most financial market players use protocol version 1.2.

According to the FIX protocol standard, each message has the format Tag = Value SOH, where Tag is the number of the transmitted field, Value is its value, and SOH is the separator character. The FAST protocol eliminates redundancy by using a template that describes the structure of the entire message. This method is called "implicit tagging" because the FIX tags in the transmitted data are only implied.

ISS

This protocol stands out somewhat from the general series, since it covers a segment of tasks related not to carrying out transactions, but to working with exchange data. Essentially, this is an API for exchange web services, implemented according to the Restful concept. It makes it possible to obtain general market information, such as quotes, transactions, indices, volumes, trading results, and so on, via the http/https protocol. The service is only available via the Internet, so minimizing delays in receiving data does not apply to it.

This protocol is used to display stock quotes on websites (including all data on the moex.com website is broadcast from there), download trading results for analytics, and render graphs on various demo panels and displays and in any other applications running via the Internet.

For those traders who do not use robots for trading, there is the opportunity to trade on a direct connection, using a trading terminal that is more familiar to them. However, the software that works with the brokerage trading system does not work with direct exchange protocols, so separate programs are created for it.

Additionally, because direct-to-connect technologies are open, investors can develop the software for themselves. However, since these programs ultimately have almost direct access to the core of the trading system, the exchange has implemented a procedure for certifying trading solutions from third-party developers to eliminate the possibility of a “crazy robot” completely destroying the entire system. This procedure goes through both the development of individual investors and software created by special companies to order.

The NYSE Euronext data center is located in Mahwah, New Jersey. The area of the halls used for traders' server colocation is about 18 thousand square meters, while the total building area exceeds 120 thousand square meters. Data Center Knowledge published some photos of this data center.

Facility Management Center - it combines the interfaces of building management systems (BMS) and data center infrastructure management (DCIM). This is where specialists sit who control temperature and humidity conditions, the condition of power supplies and other elements in each server room.

And this is what the “hot corridor” looks like, into which the air emitted by the servers enters:

The long main corridor of the data center allows you to feel the enormous scale of the facility.

Equinix is one of the world's largest players in the data center and colocation market. One of its facilities is a former optical manufacturing plant in Secaucus, New York, with an area of more than 100,000 square meters, converted into a state-of-the-art data center. Exchanges such as NASDAQ, BATS, and CBOE use its services, and this is what it looks like.

There are bars on either side of the data center's long main corridor; These grilles and posts are connected by a cable that runs through yellow overhead cable ducts.

The 12-meter high ceiling provides ample space for multiple levels of cable ducts that fill the top of the Equinix data center, containing separate ducts for interconnect, carrier and power cables.

The uninterruptible power supply (UPS) room in the Equinix NY4 data center, powered by a 26-megavolt-ampere substation. The equipment consists of a 30 megawatt uninterruptible power supply system that supports the operation of the computing installation in the event of a power failure.

In the event of a power outage, the NY4 data center equipment is equipped with these 18 backup Caterpillar diesel generators with a capacity of 2.5 megawatts each, which in total provide 46 megawatts of emergency power, enough to fully power the equipment, as well as refrigeration units and UPS systems. During Hurricane Sandy, these generators kept equipment running for an entire week.

The NY4's refrigeration system features huge pipes that carry chilled water to the equipment, as well as a plate heat exchanger (right) that acts as a refrigerator in the winter, saving energy that would normally be wasted running the refrigeration units.

Currently, the largest Russian exchange platform, Moscow Exchange, offers traders the opportunity to place their equipment in the Moscow M1 data center. The data center was put into operation in 2006, its total area is 3850 square meters, of which 2400 square meters are allocated for server rooms (load capacity 5-8 kW per rack). There are about 950 such racks in total.

To maintain the required temperature (22 ± 4 degrees) and humidity (45 ± 10%), supply and exhaust ventilation systems and precision industrial-type air conditioners are installed in the server rooms.

Server racks are arranged according to the principle of hot and cold aisles; closing the latter prevents the mixing of hot air emitted by equipment with cold air coming from air conditioners.

Uninterruptible power supplies are designed for a longer operating time of all equipment than is necessary to start diesel generator sets and switch to autonomous power supply to the data center.

In 2014, the management of the Moscow Exchange decided to move to the DataSpace1 data center. The data center was commissioned in July 2012. Its total capacity is 1,062 rack spaces across 12 machine rooms, each with an area of up to 255 square meters.

The data center has 6 independent power supply circuits, each computer room is supplied with electricity from 2 independent circuits.

Chillers and dry air coolers are installed according to a N+1 distributed redundancy scheme, cold air is supplied under the floor.

The building's perimeter and interior spaces are equipped with an 8-level security system that includes various access control and surveillance components.

Exchange platforms and HFT traders pay great attention to building their own trading infrastructure. Today, in the stock market, success and failure are often separated by a fraction of a second, so both the software and hardware that powers financial applications must work extremely reliably.

The load level is so high that it is difficult to cope with and requires significant investment on the part of data center service providers. Otherwise, situations such as the one may arise happened in August 2015, in the CenturyLink data center - during a serious movement in the market, the infrastructure for HFT trading was working in an increased mode, which the ventilation system (HVAC) could not cope with. As a result, many servers not only overheated, but physically burned out.

Currently, most traders and brokers build their HFT systems using popular software and hardware technologies. This allows algorithms to be described in high-level programming languages familiar to many developers, and changes can be made to them quite quickly if necessary. However, the pursuit of speed means that the unpredictable response times of software systems become an obstacle to successful trading. Let's look at the existing software and hardware approaches to building HFT systems.

Software high-frequency platforms

There are quite a large number of companies offering software for high-frequency trading (for Western exchanges these are, for example, Mantara, Ulink and QuantHouse). When using them, most of the delays occur in the operation of the operating system on which the software is running, as well as the network stack. To combat this, users can use high-performance network cards (such as those from Solarflare or Myricom) that speed up certain parts of the network stack.

Custom hardware HFT platforms

The relatively high latency of software trading platforms has forced the industry to look for alternative approaches to reducing latency using custom hardware. Generally, things like ASICs are not considered in HFT trading because they lack the flexibility to be reconfigured or handle new protocols. GPUs also cannot offer significant performance. FPGA (Field-Programmable Gate Arrays) technology has become a suitable tool for obtaining flexibility and achieving the required performance.

FPGAs can be used to accelerate financial applications in a variety of ways. One of them is called Hybrid Computing and is used, for example, in models risk management, option pricing and portfolio modeling. When using it, the speed of the system can increase by three orders of magnitude.

FPGAs are often used to create modern online trading systems. The tasks of processing Ethernet, IP, UDP connections and decoding the FAST protocol are transferred to this hardware. FPGA parallelism allows for significant speed gains compared to software-only tools. The architecture of the system described in this work looks like this:

This approach complements conventional multi-core processors with FPGA coprocessors. Typically, communication with the CPU is carried out using high-speed connectors such as FrontSide Bus (FSB), PCI Express or QPI. In this case, the trading modules themselves are written in high-level programming languages.

Another way to use programmable logic for acceleration is through the use of so-called Smart NICs. Typically this refers to a combination of high-speed network interfaces, host PCI interfaces, memory and FPGAs. Here, the FPGA acts as a NIC controller, acting as a bridge between the host computer and the network and allowing software logic to be integrated directly along the data path. Thus, Smart NIC can operate as a trading platform controlled by the host machine's CPU.

Modern FPGAs can implement any aspect of HFT applications. Incoming market data can be processed entirely on the FPGA without having to be sent to the processor. Incoming network data is fed directly to a customized, highly optimized system via the MAC and PHY hardware. Moreover, in fact, the necessary information can be extracted even before the packet is fully received. Thus, the use of FPGA allows for a significant reduction in overall latency.

The use of FPGAs also has its disadvantages compared to traditional approaches to developing trading systems. The root of the problem is the higher complexity of the FPGA development flow. A significant portion of financial system developers and traders are not familiar with this technology and lack the knowledge and expertise to implement hardware-oriented development.

Due to the lower level of abstraction, developing and testing new hardware solutions is a more complex and time-consuming process compared to the usual writing of a trading robot. All this is illustrated in the figure below:

To avoid this, some FPGA-based HFT platforms have special high-level environments that allow you to create trading systems without the need to use hardware description languages (HDL).

The architecture of modern graphics cards is based on a scalable array of streaming multiprocessors. One such multiprocessor contains eight scalar processor cores, a multi-threaded instruction module, and shared memory located on the chip (on-chip).

When a C program using CUDA extensions calls a GPU kernel, copies of that kernel, or threads, are numbered and distributed to available multiprocessors, where their execution begins. For this numbering and distribution, the core network is divided into blocks, each of which is divided into different threads. Threads in such blocks execute simultaneously on available multiprocessors. To manage a large number of threads, the SIMT (single-instruction multiple-thread) module is used. This module groups them into “packs” of 32 threads. Such groups are executed on the same multiprocessor.

Financial analysis uses many measures and indicators, the calculation of which requires serious computing power. A measure called the Hurst exponent is used in time series analysis. This value decreases if the delay between two identical pairs of values in the time series increases. The concept was originally used in hydrology to determine the size of a dam on the Nile River in conditions of unpredictable rainfall and drought.

Subsequently, the Hurst exponent began to be used in economics, in particular, in technical analysis to predict trends in the movement of price series. Below is a comparison of the speed of calculating the Hurst exponent on the CPU and GPU (acceleration indicator β = total calculation time on the CPU / total calculation time on the GeForce 8800 GT GPU):

Experiments show that the use of GPUs can lead to significant improvements in financial analysis performance. At the same time, the speed gain compared to using a CPU architecture can reach several tens of times. At the same time, you can achieve an even greater increase in performance by creating GPU clusters - in this case, it grows almost linearly. That is why GPU and FPGA technologies are widely used when building HFT systems.

The order book (DOM, Depth of Market) is a list with a numerical display of current buy and sell orders for a given asset on the stock market at the prices set by participants. This indicator reflects the sentiment of market participants and is one of the trader's most important tools. It also has other names: the book of orders, Order Book, market depth (Depth of Market), Level 2, and the trading terminal's order book (Open Book). In simple terms, it is a table that displays information about the orders currently submitted by sellers and buyers.

During the trading sessions The exchange platform collects thousands of applications from all participants every second and brings them together. At the same time, the order book is just a form of visualization of the limit orders closest to the current price.

Analysis of the stock market makes it possible to objectively evaluate supply and demand levels at the current moment of trading for the instrument of interest. Practitioners of technical analysis use an order book to identify the line of least resistance for an asset's price movement. Also, using the glass, you can make short-term forecasts, which are effectively used in scalping.

It is generally accepted that with the rapid disappearance of submitted orders in one direction, the price will soon move in the same direction. The main difference between the order book and price charts is that it does not provide a visual display of market data. It only displays received orders that are close to the market and the execution of which will in some way influence further pricing.

An increase or decrease in the price of an exchange instrument is determined by the distribution of supply and demand for it. In turn, supply and demand depend on the steps taken by active market participants. Any of the financial markets is a two-sided auction. Let's say someone decides to sell 10 lots some asset. To do this, a buyer must appear who is ready to buy the proposed volume of the asset at the stated price. In this way, a transaction takes place between a seller and a buyer, of which a large number are made on the exchange every second.

There are three types of exchange orders. Market orders to buy/sell are executed at the best market value in the desired volume.  Limit orders are regular orders that include the required asset, its price, and the desired volume. Conditional orders are all orders that require compliance with the conditions set by the market participant, excluding limit orders.

The order book shows only limit orders. Trades on the market are not visible, as they are executed instantly at the best prices. Conditional orders are not displayed due to the fact that they are awaiting the occurrence of the required conditions under which they will become limit or market.

Applications, which reflect the depth of the market, are also divided into small, medium and large. This division is conditional and is made relative to the average daily trading volumes for an instrument on a specific exchange platform. For example, if the average volume of transactions on RTS index futures is 1 million contracts per session, then an order for 2–5 thousand contracts can be classified as large orders, and it must be closely monitored.

In addition to dividing into types and volumes, applications are also divided according to their strategic purpose into aggressive and passive. Buy/sell orders that are statically placed at prices that are close to each other in value and do not move, but seem to defend some price mark and do not show aggression, are called passive. Market depth demonstrates passive orders when the market approaches powerful graphical support or resistance levels. As a result of the confrontation between bulls and bears, the level will be broken or quotes will rebound from it. Orders submitted from the market (market orders) are called aggressive.

As a rule, such orders are the driver of prices. There are also aggressive orders of a different kind: they are limited, but at the same time demonstrate a steady movement behind changing prices. Such orders tend to suddenly appear in the order book and, when the market jumps in any direction, constantly strive for the current value, pushing up prices. Such order movements are capable of maintaining the market direction for a long time.

Often passive and aggressive orders interact with each other, and the order book reflects this interaction. At breaking through the level support it becomes a line of resistance. In the order book, you can see the following situation: the condensation of passive orders is overcome, then an aggressive seller comes into play, placing orders at the nearest Ask levels.

Thus, now in order for the market to turn upward, you will need to break through resistance, which many traders began to actively protect. Passive density is also used for exposure stop loss. If after opening a position a large seller appears, and the price moves down, then the stop order can be safely set at 1–2 point higher than the seller's price. If a large order is “eaten up” by the market, it is better to close the deal, because inertia will certainly drag it down with it quotes up.

The more a trader knows about the activities of other market participants, the easier it is for him to make a decision and make money on it. For all this, technical analysis is used, which includes data on prices, terms of transactions and trading volumes that can be found in the order book. A separate robot does this, and this data is used to set up trading algorithms.

Market making

The trader makes a profit due to the spread - the difference between supply and demand. The larger the spread, the greater the profit in the end. The essence of this strategy is to increase competition between traders and investors by narrowing the spread across different assets. This strategy is common among large investment firms. It allows you to improve the quality and attractiveness of the trading platform. This type of strategy provides increased market liquidity and new territories for trading.

Liquidity is the ability to sell securities quickly and without significant losses. Popular stocks already have good liquidity. Investors who want to buy or sell a low-liquid stock often have difficulty finding a counterparty willing to offer an acceptable price. The essence of the passive market making strategy is to place a colossal number of Limit orders on both sides of the price (slightly below the market when buying, above when selling).

The result is market liquidity, making it easier for private traders to execute trades. Profit from HFT trading in this case is formed due to the difference in supply and demand prices. It is on this difference that the tweeter makes money. In addition, market makers often receive additional fees from trading platforms for increasing liquidity. Moreover, the algorithm itself may not make money and even lose a little, while the trader will earn on payments from trading platforms and ultimately end up in the black.

This strategy consists of increasing competition between investors and traders and narrowing spreads in different assets by placing orders on one side or the other of the price spread. Consequently, the most profitable for such strategies are new “territories”. Moreover, the larger the price spread of the asset, the more profit the strategy will bring as a result. Thus, the liquidity of the instrument on the platform increases, spreads narrow, which attracts new investors to the trading platform.

Frontrunning

The algorithm is based on the speed of concluding a deal when favorable conditions are discovered. The work of the algorithm can be divided into two periods - monitoring of all conditions for submitting an application and action when the application is already in progress.

First, an analysis is carried out for all large bids (ask prices) above a given condition, and if the system finds such a volume, then the robot places an order one step above this order. If the order is removed, then the order placed by the robot is canceled and monitoring continues. If the volume moves, then the robot also moves, while maneuvering to be one step ahead.

The calculation here is that before a large order is fully satisfied, the price will bounce off this volume several times.

Ignition pulse

The momentum ignition strategy is used by traders to encourage bidders to quickly execute trades. At a time when there is rapid market movement, the difference between the prices of bids for sale and purchase on the market quickly expands. This creates favorable conditions for making a profit.

For example, the buy price for a share is $200 and the sell price is $200.01, and then the buy price changes to $199 and the sell price becomes $200 per share. In such conditions, it turns out that the sale price becomes the previous purchase price, and the execution of the last remaining orders in the queue for purchase at $200 will allow the trader to ultimately resell the share at $200. The goal of directional strategies is to make a profit by predicting the directional movement of securities prices. This is where they differ from other types of strategies by taking on unhedged risks.

In some cases, HFT traders themselves try to provoke market participants to quickly carry out trading operations, leading to sharp price fluctuations. The essence of the principle, called “spoofing,” is the manipulation of algorithms and manual traders who are forced to trade aggressively. In addition, when using this strategy, the player can cause further price movement due to the exposed stop losses. Spoofing is considered an illegal strategy but is difficult to detect and prove.

Statistical arbitrage

A neutral market strategy that makes a profit in any situation of inequality on the stock exchange. The strategy is based on searching for discrepancies between prices by receiving various news that affect the financial market.

The HFT algorithm monitors prices and trading volumes on various exchanges in the run-up to significant events, looking for anomalous behavior. According to it, the trader, even before the official news appears, reacts to deviations and concludes a deal. The essence of the strategy for making money for HFT traders by performing arbitrage operations is to search for discrepancies between the prices of the same financial instruments in different markets.

Statistical arbitrage aims to profit from price discrepancies that arise between related trading venues. HFT traders try to find correlations between related financial instruments (for example, between a stock and a futures contract on it) and earn income from the imbalance between them.

This is a market-neutral strategy that brings profit in any current situation on the stock exchange - the market is going down, up or standing still, based on the effect of high correlation of asset prices. The instruments used in this strategy are options contracts, bonds, forward and futures contracts, and derivative financial instruments. This type of arbitration uses mathematical modeling methods and can be used on any time period.

Arbitration of delays

It aims to generate income by obtaining data on financial instruments earlier. To have a time advantage, traders place machines with algorithms as close as possible to the exchange servers, ideally in the same machine room.

Financial instruments used on different trading platforms are interconnected, and price fluctuations on one exchange affect all others. During trading, all information cannot move instantly, for example, between the Chicago and New York exchanges 1200 km. In terms of time, this is about 5 milliseconds. Trading robots on the New York platform receive information with a delay.

Latency arbitrage is aimed at generating income for the HFT trader due to earlier receipt of data. For this purpose, servers with trading software are located in exchange data centers (colocation) near the equipment used to host the cores of exchange systems. As a result, HFT traders receive important information a moment earlier than other market participants.

Liquidity detection

With this strategy, high-frequency robots try to detect large or hidden orders from regular platforms and from automated systems even before trading begins. For this purpose, robots send small orders to the market, time their execution, thus tracking when a large transaction should occur.

Trade By tape

This strategy monitors all events in the stock markets, such as sales volume and price quotes. This helps to collect a lot of important information. Monitoring all information (certain stocks) and all significant events (company news, financial statements and macroeconomic data releases) allows you to calculate abnormal behavior in sales volume and stock prices. As a result, based on all the collected and analyzed information, the high-frequency robot is able to determine in advance “patterns"even before the official news.

The incident occurred on May 6, 2010. That day the Dow Jones index fell by 990 points in 5 minutes for no reason. This caused panic in the market and a decline in quotes. As it became known later, at that time the market share of HFT traders was 70%, and they only had to close positions for the market to collapse.

Since their operating patterns are largely similar, this is exactly what happened from 14:42 to 14:47 on May 6th. The above events caused a strong resonance in society. The media spread criticism, and there were many protests from financiers and politicians who convened commissions and hearings to ban or tax high-frequency trading. However, all this turned out to be in vain, and HFT trading only became more firmly entrenched in the market.

Another incident occurred on October 15, 2014, when Treasury yields fell by a quarter of a percent within a few hours. The fall occurred completely unexpectedly and, judging by indirect evidence, was a consequence of the HFT strategy.

In addition, despite the fact that high-frequency traders generate a large number of orders to buy and sell financial instruments, in reality, not all of them lead to an increase in liquidity. Trading robots generate a large number of orders that do not actually result in transactions, or transactions are performed not with real stocks, but with exchange-traded funds (ETFs).

Only developed investment structures and funds can afford to maintain high-frequency trading programs. Private investors and traders are far from such an industry; it is inaccessible to them. With rare exceptions, there are also prop trading companies that work with HFT using their own funds. In general, all users of high-frequency trading can be divided into four categories:

This state of affairs is due to several factors:

All this taken together is inaccessible to the average trader, regardless of his financial capabilities or desires. Many describe this infrastructure as a monopoly on the stock market, which also requires corporate connections and special position. This is not surprising, since HFT companies receive information about all transactions on the market much earlier, and, as a result, have a huge advantage over other market participants.

High-frequency trading has been around for a long time and has faced a number of challenges in recent years: physical limits, rising infrastructure costs, competition, declining profits, and increased regulation. Because of all this, HFT's share of total trading has been steadily declining in Western markets. However, high-frequency trading still accounts for more than half of all trades in the U.S. and Europe.

The impact of high-frequency trading has been widespread and likely long-lasting. HFT infrastructure and practices could be a must-have for the financial industry of the future, as high-frequency trading improves the quality of markets not only for institutional investors, but also for retail ones.

High-frequency trading algorithms are focused on speed, mainly because most of them use arbitrage or passive investment strategies to generate their returns. Higher trading volumes and liquidity are the most significant, lasting and noticeable impact on equity markets today. Total stock trading volume in the United States has doubled since the advent of high-frequency trading.

Ultra-high trading volume leaves no options for players looking for tiny profits from millions of trades, so the increase in volumes has significantly changed the balance of power in the market. This means that HFT algorithms increase liquidity through passive market making.

This feature of speed traders plays an important role when there is a significant outflow of liquidity from conventional providers during market turmoil caused by major macroeconomic news, political events or natural disasters.

Profits for most HFT players require extremely high levels of volatility in preferred assets. On the other hand, stock price volatility is the main building block of the market. Therefore, high volatility is undesirable for large institutional investors and companies. It can lead to an increase in the perceived risk of the company's shares, and therefore to an increase in the weighted average cost of capital - the costs of providing each source of financing for the company.

The ultra-high trading volumes that result from high-frequency trading influence price movements throughout the trading day and can reduce stock volatility as HFT players provide liquidity to the market and allow large traders to execute their trades without significantly affecting stock prices, indirectly reducing price variances.

Arguments that this liquidity is some kind of fake can be countered by the fact that high-frequency trading strategies do not benefit from stock price movements. They generate income from the difference in purchase and sale prices and discounts provided by the electronic system for carrying out purchase and sale transactions (ECN).

High frequency traders uncover the largest and most liquid assets, continuously provide liquidity and essentially shape the market, so HFT can be said to reduce the bid/ask differential for large cap companies and facilitate faster price discovery in the market.

The effect of increased liquidity may allow traditional institutional investors to more easily adjust their portfolios to reflect their fundamental views of a company's performance. Thus - HFT can reduce the transaction costs faced by institutional investors and helps bring market prices closer to their fundamental value.

High-frequency trading isn't everyone's cup of tea, but it is clearing the market of irrational investors. Because of HFT, they are unable to withstand lightning price changes.

In relation to hft, the financial world is divided into two camps: those who think markets have benefited from the technology, and those who argue that high-frequency trading benefits only a handful of VC-backed academics and trading platforms that have gained an advantage over retail investors.

But after almost 10 years of rapid technological development, dozens of market crashes and increased trading speeds, high-frequency trading can be seen as a natural outcome of attempts to make financial markets more efficient, bring them closer to theoretical purity and enable them to immediately reflect any new information.

There will always be two points of view on hft, but it must be taken into account that in 5-7 years high-frequency trading will be closely related to artificial intelligence and machine learning. Recent improvements in high-frequency trading have significantly changed the way markets trade, but it is unclear whether the impact will be positive or negative.

Most investors have probably never seen the equity curve of a high-frequency strategy. There are objective reasons for this: due to the typical profitability of such strategies, firms using them have little need to attract outside capital. In addition, HFT algorithms have capacity limitations, which is very important for institutional investors. Therefore, it is interesting to observe an investor's reaction to the profitability of an HFT strategy that they are seeing for the first time. Accustomed to a Sharpe ratio in the range of 0.5-1.5 or, in a successful combination of circumstances, up to 1.8, they are amazed that such strategies can show coefficient values expressed in double digits.

To illustrate, the figure above shows the performance graph for one such HFT strategy, which trades about 100 times a day in the E-mini S futures contract.&P500 (including night session). Note that the statistical advantage of the algorithm is not very high - on average, 55% of profitable trades and a profit per contract of about half a tick are the usual characteristics of most high-frequency strategies. But due to the large number of trades, they lead to significant profits. At this frequency, exchange commissions are small, the tariff is about $0.1 per contract.

However, hidden from us are additional costs associated with using the strategy: fees for market data, a software platform and an Internet connection that provides large volumes of data and speed to track microstructure signals and manage the placement of orders with the best priority. Without this infrastructure, strategies are unlikely to be profitable.

Let's lower our requirements a little and consider an intraday strategy with the number of transactions approximately equal to 10 per day, on 15-minute bars. While this is not ultra-high frequency, it is still high enough frequency to be latency sensitive. In other words, you will not implement such a strategy without market high quality data and a trading platform with minimal execution delays of 1 millisecond.

With the same probability of profitable trades as in the first strategy, the lower frequency of trades makes the profit per contract more than 1 tick, while the equity line is much less smooth, reflecting a Sharpe ratio of only about 2.7.

The most important characteristic of HFT algorithms is the probability of execution of limit orders. Strategies most often use limit or IOC (if not executed immediately, cancel) orders, only a certain percentage of which will be executed. When receiving correct signals, profit increases in direct proportion to the number of transactions, which in turn depend on the probability of execution. A probability of 10% to 20% is usually enough to guarantee profitability (although this also depends on the quality of the signal). The low probability of execution that typically occurs when trading through widely offered trading terminals will destroy the profitability of any high frequency strategy.

Developing algorithms for HFT is a science-intensive activity. At a minimum, you will need mathematics and economics, and only then specific programming languages and technologies. The need for fast operation of algorithms leads to the fact that the main programming languages in the financial market are C, C++ and Java. Also valued is experience in optimizing package processing, working with databases and using the scripting languages Python and MATLAB, which are used for initial testing and development of trading strategies.

Most high-frequency trading companies are small. Their modest staff usually numbers about 20-25 people. This is explained by the fact that employees of such companies follow a very specific entrepreneurial culture and a meritocratic outlook on life. During the interview, you, as a candidate, will certainly be asked what innovations you can bring to the organization.

Given the fact that the bonus fund is common to everyone (albeit distributed taking into account the different weight of each employee), you will need to demonstrate the ability to generate income that (explicitly or implicitly) exceeds your salary and bonus. Otherwise, there is simply no point in hiring you. That is, in order to even be considered as a candidate for a position, you need to show skills that no one in the organization yet possesses.

On the other hand, here is a chance to create a place for yourself. People in the company may not be looking for new employees at all, but if they realize that you have a lot of experience in a certain area, they may open a vacancy for you. The meritocratic approach in HFT firms usually allows for sufficient autonomy in projects. Therefore, if you want to work in a proactive environment alongside extremely smart and talented people, then HFT is probably for you.

Working hours in this area are above average. 60-70 hours a week is not uncommon, especially when the project deadline is close. However, the intense intellectual work and monetary compensation usually outweigh the workload. This lifestyle is not for everyone.

Keep in mind that HFT is a highly technical field. It attracts the most outstanding candidates in mathematics, physics, computer science and electrical engineering, most often during their graduate studies or after several years of work in a highly specialized area of industry. Despite its high pay, working in HFT companies will require significant costs in terms of training and invested effort.

Usually people come to HFT firms after:

Postgraduate studies. Most HFT firms hire candidates after graduate school in a specific area relevant to the company’s tasks. This approach is the simplest, since it is easier to determine the candidate’s abilities based on the doctoral dissertation, publications or university status. So if you really want to make a career in hft, researching low latency systems could be a good way to go. There are also frequent cases of selecting distinguished students from their final years at the most popular technical universities (MIT, Stanford, Cambridge, Imperial) and then training them for a specific position.

Gaining experience in the high-tech industry. Experts in specific domains where low latency is required (such as telecommunications) are typically hired for their domain knowledge. However, it is worth noting that, as a rule, for normal work in their field, they in any case need an extensive technical base. Scientists working on projects related to high-performance computing (for example, at the CERN data center or other national laboratories with supercomputing) are in high demand due to their expertise in working with big data.

Experience working on the stock exchange. Those who know how the work of the exchange is organized “from the inside” are also considered in demand. This is explained by the fact that such people can most likely help create new algorithms that take advantage of the information architecture of a particular exchange.

One of the most common misconceptions is that getting a job in HFT requires extensive knowledge of finance. Most HFT firms do not pay attention to the availability of knowledge in the field of finance, given the sufficient level of technical competence of the candidate in other areas needed by the organization.

Job responsibilities in HFT firms are quite varied. Almost every employee is forced to have a higher technical education and the ability to conduct independent research in this area (must have a good knowledge of theoretical material). Since HFT is essentially a technology sport, most will also have a background in computer engineering, electrical engineering, or experience with low latency data transfers in other fields such as telecommunications.

There is also now a demand for knowledge and experience with certain types of software, such as graphics processing units (GPUs) and field programmable arrays (FPGAs).

By and large, any subject area that can somehow reduce trading latency or increase the speed of algorithmic calculations will find its place in HFT. Examples of such areas are:

Schemes of stock exchanges. Among those involved in high-frequency trading, the basis of skills is extensive knowledge of the organization of stock exchanges. Knowing how the order book works, as well as all the intricacies of technology on a particular exchange, can work to your advantage.

Processor architecture. High-frequency trading involves a significant number of transactions in a relatively short period of time. Keep in mind that increasing the speed of these operations in any way will benefit you. It is useful to have knowledge of processor and hardware architectures, especially systems other than x86 architecture (such as GPUs and FPGAs).

Networks with minimal delays in information transmission. One of the main sources of data transfer delays in trading is the network stack. HFT firms value experience in optimizing packet processing, writing special network modules, and using the high-speed switched serial bus Infiniband.

Understanding the laws. Knowledge of such legal acts as the “Regulation of the National Bank System” (Regulation NMS) of the United States of America and the European Union Directive “On Markets in Financial Instruments” is necessary in HFT operations.

Kernel optimization. The main goal of optimization is to reduce delays and increase the speed of operations. Therefore, today it is not uncommon to rewrite the core software to speed up the process. Many HFT firms value experience in modifying the Linux kernel.

Online algorithms. Speaking about delays during operations, I did not go into details of the operation of HFT algorithms. Often these algorithms involve “repeated” operations with arithmetic mean, error term, and linear regression. Therefore, analysis of the results of previous calculations is very important.

Programming languages. Although most UHFT firms have moved to dedicated hardware for direct processing and network communication, in less latency-sensitive cases trading firms can use multithreaded C, C++, and Java with specialized garbage collection. Some trading firms value experience with such languages and parallel computing.

As you can see, these skills are often technical in nature and require an academic degree or several years of working with certain technologies on an industrial scale. If you have experience with the above things, then you have a chance to prove yourself in an interview with an HFT company.

As with most finance jobs, it's best to look for work through recruitment agencies. The largest HFT firms are located primarily in New York and London. Chicago is also home to a large number of high-frequency trading organizations. Recruiters are usually knowledgeable in the subject area and can answer whether your experience matches the requirements of the position. Please note that the bar is very high! Most likely, you will have to put all your efforts into searching for a vacancy, which can take up a lot of your time.

Although it is possible to apply directly to a firm, the most difficult process is to find companies involved in high-frequency trading. Typically, if you are popular in your niche field, the organization will try to hire you themselves. This can work to your advantage if you really want to work for that firm, publish your research, attend conferences, and just seriously improve your track record.

So, today we have lifted the veil on the rather secretive but very interesting world of high-frequency trading. Not all types of HFT trading are beneficial; there are also toxic trading systems, but for the most part HFT is a good thing. It is quite difficult, almost impossible, to enter the HFT business on your own, so traders band together and organize private companies. Even so, it is already quite difficult for them to compete with large companies. Competition in this field is so intense that, if you want to work in HFT, you should look for a quant position at a large fund. But to pass such an interview successfully, you need many useful skills and an excellent university education, or better yet several, so this path is not easy either.

You also should not trust expert advisor sellers if they claim to have developed an HFT algorithm for one of the Forex trading terminals: the probability that they are not scammers is practically zero. The same also applies to arbitrage as a special case of HFT. No matter how modern the terminal is, if your connection scheme reaches the trading venue through a broker, your execution speed will be uncompetitive and, even if you do manage to earn something, it will happen only over a very short period of time. The Forex market already contains a large number of professional HFT systems that use expensive equipment and systems developed by an entire staff of employees.

However, at the moment there are markets in which there is almost no competition. For example, cryptocurrency exchanges, just like securities exchanges and many Forex venues, provide access to trading through their APIs. This market is still very young and still uses very slow technologies. For example, the average time required for a trade on a typical crypto exchange is still about one second, as at the very beginning of the emergence of HFT. So if you want to try your hand at HFT trading, this market can be an excellent starting point. It will definitely develop, exchanges will upgrade their equipment, and speeds will rise. But for now, you still have a chance to enter this business before large professional players come to the cryptocurrency markets. Some types of HFT systems are already used in cryptocurrency markets, and their owners earn super-profits, but competition is still very low.

I hope this information was useful to you and that you will draw the right conclusions. Even if you are not going to work with HFT systems, this article will allow you to understand the overall picture. If you decide to try writing or creating your own HFT system, I recommend paying attention to emerging markets, for example the cryptocurrency market.

Best regards, Dmitry aka Silentspec TradeLikeaPro.ru