How can herd instincts be used in trading?

We have already discussed many types of strategies in forex: high-frequency trading, arbitrage, moving averages, cycle theory, price return to the mean, trend systems and breakout ones. But there are several more types of trading systems that we have not yet had time to discuss. And one of these types is systems built on the psychology of traders and investors.

It is hard not to think of the market as a person: its mood can shift from irritated to causelessly cheerful, then it can react hastily to something and change everything the next day. But can psychology really help us understand financial markets? Behavioral finance theorists believe that it can. This field of research argues that people are far from as rational as traditional financial theory describes them. So, let us sort it out.

The idea that psychology drives the market runs counter to the established theory according to which markets are efficient. Supporters of the efficient market hypothesis say that any new information quickly gets into market prices through the process of arbitrage.

Behavioral specialists explain that irrational behavior is not an anomaly but an ordinary thing. Indeed, researchers were able to reproduce market behavior using very simple experiments.

Here is one of the experiments: offer someone a choice between receiving $50 and getting a chance to toss a coin and win $100 or win nothing. Most likely, the person will choose $50. Conversely, if offered a choice between losing $50 and a chance to lose $100 or lose nothing, the person will probably want to toss the coin. The chance to toss the coin is present in both scenarios, but people will go for it in order to save themselves from losses, even though they may lose even more. People tend to consider the chance to recover losses more important than the chance to get a bigger gain.

The priority of loss avoidance is also true for traders. Just think about how popular various grids and martingales are. No matter how low the price falls, traders believe that, in the end, it will rise again, and they still hold losing positions right up to a margin call.

The herd instinct explains why people tend to imitate others. When the market moves up or down, traders think that others know more or possess more detailed information. The consequence is that traders feel a strong urge to do what others are doing.

Behavioral finance also found that traders tend to attach too much importance to judgments obtained from a small sample of data or from individual sources. For example, it is known that if an analyst picks winning stocks, then traders attribute it to skill rather than luck.

On the other hand, traders' beliefs are not so easy to shake. In the late 1990s, long-term traders were gripped by the belief that any sudden market drop was a good time to buy. Moreover, this point of view still reigns. Traders are often too confident in their judgments and tend to lunge at a single "speaking" detail instead of listening to the more obvious average.

In the process of decision-making people use facts as factors, but they act under the influence of emotions. Even if it seems to you that you are working with a cool head, you are mistaken. There is no magic button that could completely turn off the emotional component even for a short time.

In the world of statistics there are classifications for different types of variables. These are rules that guide research and trading decisions. They relate to how we analyze models and conduct stress tests in order to achieve statistically significant results, and how we ultimately make decisions. These variables are procedural, and they are highly controllable.

Imagine that you were taken to a police station to give witness testimony. The police will conduct the questioning according to their procedure. Will you look at real suspects? Will they show you photographs? If so, will the photos be shown one at a time? Or six at once? Will the officer be peering over your shoulder at that moment? What gender, race, and age will the officer be relative to the witness? Or relative to the suspects? These are choices that need to be made when carrying out a specific procedure.

Now imagine your mental state when you are sitting in a police station. How can your mental and emotional status change depending on the nature of the crime? What if a weapon was used in the crime, will you focus on the weapon or on the attacker? How confidently will you answer questions if the crime happened an hour ago, or a day ago, or a week ago? If it happened next door or far from your home? Will you be more or less likely to point to a photograph matching your race or gender? Are people with tattoos criminals or creative personalities?

All these are "contextual variables," because they relate to you and to the context surrounding your individual decision-making process, in our case to your ability to provide reliable witness testimony.

There is a connection between systemic and contextual variables, they are not completely independent of one another. For example, optimizing systemic variables by introducing procedures that reduce witness anxiety and the time period between the crime and the clarification of circumstances can help stabilize other volatile contextual variables, which will lead to more accurate eyewitness testimony.

Trading works in much the same way. We constantly seek to explore new methods, integrating ideas where appropriate and introducing into the working system variables that show strong statistical significance. And we know that if we succeed, then we will most likely get a suppression effect on other volatile contextual variables. In simple words, if we develop a system that ensures stability and growth, and at the same time try to suppress the influence of contextual variables as much as possible, then such a system will be easier to follow, and the probability of wrong decisions due to emotions and the money losses connected with this will be minimized.

Unfortunately, in most cases the influence of contextual variables is ignored when developing trading systems. The average trader gets significantly worse results than an investor who sticks to a simple "buy and hold" strategy. Most of this performance gap is explained precisely by behavioral shortcomings (that is, contextual variables). Very many investors, seized by fear and greed, buy high and sell low.

Therefore, despite nice and tidy theories, instruments are often traded at unjustified prices, traders make irrational decisions, and you struggle to find a person who has that so desirable and so heavily advertised 60% annual return every year, as if on schedule.

So what does all this mean for us? It means that when traders make decisions, emotions and psychology play a big role and sometimes force them to behave unpredictably or irrationally. This does not mean that theories do not matter, their principles work. But not always. And since people more often behave like idiots, it is simply necessary to make money from this. All that remains is to understand how.

So, will these ideas help generate profit? After all, a lack of rationality should provide many profitable opportunities for smart people. However, in practice few people use behavioral finance in their trading strategies. The influence of behavioral finance is still more often studied in academic circles than in practical money management.

Despite the fact that behavioral finance points to numerous deviations from rationality, it offers few solutions that make money from market passions. Robert Shiller, author of the book "Irrational Exuberance" (2000), identified that in the late 1990s the American stock market was at the center of a bubble. But he could not say when it would burst. Likewise, during a market decline, specialists in human behavior will not be able to tell us when the market reaches bottom. However, they can describe what it will probably look like.

Specialists in human behavior have still not come up with an intelligible model that actually predicts the future rather than simply explaining after the fact what the market did in the past. The main lesson is that the theory does not tell people how to make a profit. Instead, it tells us that psychology affects the way market prices deviate from the norm over long periods of time.

Behavioral finance does not offer investment miracles, but perhaps it will help you learn to watch your own actions, which in turn will help you avoid mistakes that reduce personal wealth.

The differences between theoretical and behavioral finance are better viewed this way: theory is the foundation from which one can push off in improving understanding of the subject under study, while behavioral aspects are a reminder that theory does not always work the way it was expected to. Accordingly, a good knowledge of both points of view can help you make better decisions.

The idea that financial markets are efficient is one of the basic principles of modern portfolio theory. This principle, defended in efficient market theory, assumes that at any given moment prices fully reflect all available information about a specific market. Since all market participants know the same information, no one will have an advantage in forecasting returns, because no one has access to information unavailable to the rest. In efficient markets, prices become unpredictable, therefore patterns cannot be traced, which completely denies any planned approach to trading. On the other hand, research in behavioral finance, which studies the influence of investor psychology on rates, reveals some predictable patterns in markets.

In theory, all information spreads equally. In reality, if that were true, insider trading would not exist, and unexpected bankruptcies would never happen. There would be no need for the Sarbanes-Oxley Act of 2002, which was designed to move markets toward higher levels of efficiency. And let us not forget that personal preferences and personal abilities also play some role. Obviously, there is a gap between theory and reality.

Theoretically, everyone makes rational decisions. Of course, if everything were rational, there would be no speculation, bubbles, or irrational exuberance. No one would buy when the price is high and sell in panic when the price falls. Without taking theory into account, we all know that speculation exists, and that bubbles grow and burst.

In 1995, the consulting company Gartner introduced the so-called "hype cycle," or the technology maturity curve.

Since then Gartner has regularly published a chart of technology maturity on which it marks what phase this or that innovation is currently in. Investors follow these publications closely in order to understand when and what to invest in.

This is a typical development of hype not only for technologies, but also for any events in the markets. In fact, the reaction to any news item can be placed into such a pattern. Let us rename the stages from the first image as applied to the market.

It is clear that the model is idealized. In reality, the configuration may have differences (as, in fact, it does for the parent cycle for technologies), but the main thing remains the same: the sequence of stages expressed in the price change.

Let us look at it in reality. The hype cycle for annual U.S. GDP data released on January 30, 2019:

In this case the chart fits the model well, because the news is strong and pulls the blanket over itself. In general, many events happen on the markets at the same time, and hype cycles can overlap and influence one another, ultimately expressing themselves in the price.

An example of a long-term hype cycle is the peaks on bitcoin. But there they can literally be called hypes. The bitcoin chart is a direct connecting link between financial market charts and the parent hype cycle for technology:

In a market where we can enter sell positions, it seems like a good idea to sell on a pullback with the lower point of disappointment in mind. By the way, for news that causes the market to fall, the same applies in reverse, when buys on pullbacks will be of interest.

However, the pattern is intuitive, and surely many of you came to it on your own. I personally have met several traders who trade this model, though they did not know that this was exactly what it was.

As you have probably already understood, traders and investors, even professional ones, often make mistakes and go along with their emotions when trading the markets. Therefore, another completely logical idea would be to use their mistakes for our own purposes. The Supremacy strategy discussed earlier on the blog is built on this principle. You can also look at the Caiman indicator, which allows you to track the sentiment of traders right in the terminal.

Some people believe that if you enter against the crowd, it will always produce a result, since the crowd is constantly wrong. The crowd does indeed lose money over the long run. But not at all because it constantly predicts the trend incorrectly. Most determine it correctly, because that is not difficult at all.

Watching the open positions of retail traders when price approaches significant support/resistance levels, trend lines, before important news, all this will allow us to peek behind the scenes of the retail market and understand what emotions are hiding in the head of the average currency speculator.

There are quite a lot of tools for analyzing traders' open positions. Many large brokers have their own table. You can find a fairly good tool on the pages of the native blog. This indicator is good because it collects positions from eight sources at once, which is quite enough for analysis. By the way, in the near future I will write another mql lesson for this tool, we will parse and analyze all these data in an expert advisor, gather statistics and, in the end, perhaps build a profitable robot. There is also already an expert advisor on the forum that works by this principle, Supremacy.

Of course, we have all heard that more than 90% of people lose money in the currency market. But, as we have already said, this does not mean at all that in any separately taken period of time more than 90% of traders predict the market movement incorrectly. According to the simple study I cited at the beginning of the article, the problem here is different. Human psychology is such that he is more likely to take a guaranteed profit and will delay closing a losing trade until the very end. According to statistics, the average profit of traders on the EURUSD pair is 48 pips, while the average loss is 83 pips. That is a difference of almost 70%, and it is critical.

Thus, forex traders lose more money on unprofitable trades than they make on profitable ones. The ordinary retail trader is extremely jittery and poorly disciplined, he does not know how to work with risks and money. Hence the typical paradox: most people predict the price movement quite well (drawing a trend line is not that difficult), but they do not know how to make money from it.

Also do not forget that these are data for the retail Forex market. An ordinary retail trader, as a rule, jumps around a lot and rarely keeps a position open for more than 2 days, which should be taken into account when analyzing his trades. Also bear in mind the fact that all of retail forex is less than 15% of the total volume of the currency market, a tiny niche. Therefore, use such data as yet another market indicator, which will allow you to get another hint regarding market movement.

Pay special attention to strong imbalances. If you see overbought or oversold conditions on a certain currency pair at 80% or more, this may indicate a reversal. Look for boundary, polar values, mismatches, market "vulnerabilities," which are confirmed by various correlations.

In fact, there are rather few data for analyzing this approach, and it is not easy to find the history of changes in the ratio of positions even for one of the currency pairs. But I am absolutely sure that a profitable trading system can be built from this idea, it is only necessary to collect these data and carefully test the strategy on a sufficient amount of them. To make sure that further research is worthwhile, it is enough to look at the first chart at hand with marked zones where the ratio of open positions exceeds the 30/70 levels:

So, with a quick assessment we got 8 losing trades and 5 profitable ones. The total loss amounted to 220 pips, the total profit to 486 pips, and the net profit to 266 pips.

As you can see, levels above 80% work best. If we had taken only the trades where the imbalance was above this level, we would not have had a single losing trade, the profit would have been 486 pips, and the maximum drawdown in one of the open positions would have been only 25 pips. Then it would have been possible to work calmly with a stop of 50 pips, which with an initial deposit of $1000 and trading with a risk of 3% per trade would have brought us $324 of profit or 32.4% on one currency pair with a drawdown of 1.5% in literally a month and a half! Of course, in reality the picture may be, and will be, not so rosy, but such a result indicates a certain potential of this approach.

To develop this idea and turn it into a potentially profitable strategy, one should accumulate statistics for at least a year for all available currency pairs and all brokers. Then, with the help of testing, determine the best data sources. Perhaps it is worth using the averaged values of the sources that show the best results in tests, some pool of the best brokers.

So, as we have seen, most traders behave inadequately in the markets. Based on a study of trader behavior, the following conclusions can be drawn:

1. Most trades are often opened against the trend. This is especially clearly visible when the trend is prolonged.

Approximately from the middle of the trend, most traders open positions in the opposite direction in the hope of predicting its reversal and jumping in at the very beginning of the price movement. Naturally, this almost never works out:

Therefore, when more than 80% of traders believe that the trend is over, the most ideal time comes to enter in the direction of its continuation. Such a signal in our test data produced a 100% hit rate.

2. Profitable trades are closed much faster than losing ones. And this is also confirmed by our test data.

Note that even at the 70% level, when we got 8 losing trades and only 5 profitable ones, the profit exceeded the losses by about two times. This is clearly visible even visually:

The first buy trade, which turned out to be profitable, was closed very quickly by the majority of traders. The second and third ones, which immediately moved into a negative zone for the majority, dragged on for a very long time and the drawdown was much deeper than the profit on the first trade. The fourth trade, a sell, immediately moved into profit and was likewise closed by the majority at once, although the trend continued for a long time and, potentially, could have brought at least four times more profit.

All this once again confirms the expediency of the "against the crowd" trading approach and gives us an important lesson: if you want to make money in the financial markets, you have to go against your own psychology and against comfort, do the opposite rather than follow the main herd to slaughter.

3. One more conclusion: traders most often do not forget to set take profits and quite often completely forget about stop losses.

Thinking about profit is always much more pleasant than thinking about possible losses. Besides, it is quite possible that traders hope for their iron will and for that fantasy where they close a losing position at a level they thought out in advance. I allow for the possibility that the majority do not know what mathematics is, and that is why their profit-to-loss ratio is 1 to 2, but still this theory is unlikely.

4. The favorite level for the majority of traders' stop is an extremum. The following picture illustrates this idea perfectly:

As you can see, in losing zones the imbalance between open positions quickly dissolves when the previous extremum is broken. This suggests that the majority's stops are located precisely at these levels. As soon as the price drops a little lower or rises a little higher, many positions are closed by stops, and the imbalance fades away. You should not do that. Place stops after thinking a little.

So, once again we have convinced ourselves that 90-95% of idiots trade in the retail forex market, who, one way or another, lose their money. We have also once again become convinced of the already banal and worn-out cliche that "psychology in trading is the most important thing," and of the fact that despite all its worn-out and banal nature, nobody listens to it.

Very often in human psychology there is such an amusing contradiction: advice that is too banal gets ignored. Take ordinary human health for example. Doing morning exercises and not eating all kinds of dubious things are very banal pieces of advice that help prolong life by years, even decades. But look around, and you will be surprised to find that almost no one follows this advice. It is the same in trading.

We live in a far from ideal world, and the main source of this imperfection is precisely human nature. Therefore, all that remains for us, strange as it may sound, is to use it in our trading to the fullest. As long as there are people in the market with their hand-made grail strategies, there is always an opportunity to earn a little money from their holy faith in themselves and their own strength. And one excellent tool for this is working against the crowd.

Today we have examined two excellent tools: the hype cycle, which gives good opportunities for trading against traders who get carried away by the news, and the ratio of traders' open positions, which gives excellent opportunities for making trades with a high probability of a favorable outcome.

And finally, I urge you to think with your head more often and not give in to emotions. This is unacceptable both in life and in trading, with only one difference: in life you pay for idiotic behavior merely with relationships with people just like yourself, possibly work, and only rarely with something valuable, while in trading you lose your real money.

Respectfully, Dmitry aka Silentspec TradeLikeaPro.ru