Market regime detection: 3 regimes, how to measure and test them

Contents

1. What is a market regime, and which three regimes matter for futures traders?
2. How do you measure the current market regime with simple indicators?
3. Regime filter inside the strategy or regime switching outside it: which should you use?
4. How do you test whether a regime filter really adds value?
5. Why do regime models overfit, and how do you keep yours honest?
6. What does a regime model look like today for index futures?
7. Frequently asked questions
8. References

What is a market regime, and which three regimes matter for futures traders?

A market regime is a stretch of time in which price behaves in one recognizable way. For index futures traders, three regimes cover most situations: trending, ranging and high volatility. You measure each with one or two indicators read at the bar close, turn the readings into a label, and let the label decide which rules run and at what size.

The three regimes are defined by two things you can measure, not by opinion. The first is direction persistence: does price keep going the same way for many bars? The second is bar range: are the bars small, normal or unusually large? A trending regime has high direction persistence and normal bar range. A ranging regime has low direction persistence and normal bar range. A high-volatility regime has unusually large bars in both directions, whatever the direction persistence says.

The reason to care is that most futures strategies belong to one of two families, and each family does well in one regime and poorly in another. A trend-following strategy (buy after price breaks up, sell after it breaks down) collects its gains in trending regimes and pays small repeated losses in ranging ones. A mean-reversion strategy (fade a move back to an average) does the opposite. High volatility hurts both, because stops are hit by noise and slippage grows. Regime analysis, in practice, is just asking once a day which of the three you are in before you decide what to run.

How do you measure the current market regime with simple indicators?

Three inputs are enough to label the regime at each bar close: volatility, trend strength and breadth. Each one is turned into a yes/no reading with a single threshold, and the three readings together give the label. These are the market regime indicators most traders already have on their charts; the work is in fixing how you read them.

Volatility. ATR (Average True Range) is the average size of a bar's full range over a chosen number of bars, including any gap from the previous close. ATR on its own tells you little, because a normal ATR in one year is a large one in another. Compare it with its own longer history instead. The one formula in this article is the volatility ratio:

volatility ratio = ATR(14) ÷ average of ATR(14) over the last 100 bars

Here ATR(14) is the ATR over the last 14 bars, and the denominator is the plain average of that same ATR value over the last 100 bars. A ratio near 1.0 means today's bars are normal size. A ratio well above 1.0 means bars are unusually large. One threshold, for example 1.5, turns this into "high volatility: yes or no".

Trend strength. ADX (Average Directional Index) is an indicator that measures how strongly price moves in one direction, on a scale from 0 to 100, without saying which direction. Low values mean price is going nowhere; high values mean a sustained move. A simpler alternative that answers the same question is the distance of price from a long moving average, measured in ATR units, so that the distance is comparable across years. Either way, one threshold gives "trending: yes or no".

Breadth. Breadth is the share of an index's member stocks that are above their own moving average, for example the percentage of S&P 500 members above their 50-day average. It answers a question price alone cannot: is the move carried by most stocks or by a few large ones? Breadth comes from the cash market and updates once a day, so it is a daily input even if your strategy trades intraday bars. Two thresholds (a low one and a high one) give "broad up", "broad down" or "mixed".

Input	Question it answers	Common indicator	Trend (illustrative)	Range (illustrative)	High volatility (illustrative)
Volatility	Is movement large?	ATR(14) vs its 100-bar average	Ratio near 1.0	Ratio near 1.0 or below	Ratio above 1.5
Trend strength	Is movement directional?	ADX(14), or price distance from 200-bar average in ATR units	ADX above 25	ADX below 20	Any value
Breadth	Is the move broad or narrow?	Percent of index members above 50-day average	Above 70% or below 30%	Between 30% and 70%	Often falling fast

The order of the tests matters. Check volatility first: if the ratio is above its threshold, the label is "high volatility" and the other two readings do not change it. Only when volatility is normal do trend strength and breadth decide between "trend" and "range".

Regime filter inside the strategy or regime switching outside it: which should you use?

Start with a regime filter inside the strategy, because it is one extra condition with one threshold and is easy to test honestly. Move to regime switching outside the strategy only when the filter has already proved itself and you have more than one strategy to choose between.

A regime filter is an extra entry condition that must be true, together with the strategy's own signal, before a trade is taken. "Enter long on the breakout, but only if ADX is above 25" is a regime filter. The strategy is still one strategy, with one more parameter. When the filter is false the strategy simply does not trade.

Regime switching is a separate model that sits above several strategies and decides which one runs, or at what size, based on the regime label. "Run the breakout rules in trend, the mean-reversion rules in range, and halve the size or stay flat in high volatility" is regime switching. It is more flexible, and it gives you something to do in every regime instead of sitting out. The cost is a second thing that can be overfitted: the switching model has its own thresholds, and its history of decisions is short because regimes change rarely.

	Filter inside the strategy	Switching outside the strategy
What it is	One extra entry condition	A separate model that routes between strategies or sizes
New parameters	One or two	Several, plus the parameters of each strategy
Easy to test against a baseline	Yes: with and without the condition	Harder: the baseline is a portfolio of the unfiltered strategies
Behaviour when the regime is wrong	Misses some trades	Runs the wrong strategy
Best first step	Yes	After the filter has proved itself

How do you test whether a regime filter really adds value?

A filter earns its place only if it passes three comparisons against the unfiltered version of the same strategy: a better out-of-sample result after costs, removed trades that were mostly losers, and a result that stays similar when the threshold is moved a little in either direction. One good backtest of the filtered version proves nothing on its own, because the filter was chosen while looking at the data. The backtesting in AlphaPrime (Valynox) applies the same fees, slippage and next-bar execution to the filtered and unfiltered versions, so the first comparison, after costs, starts from equal settings.

Run the test in this order:

1. Run the strategy without the filter. This is the baseline. Record its net result after commission and slippage, the number of trades and the maximum drawdown, separately for the in-sample and out-of-sample periods. If you have not split the data yet, the article on walk-forward analysis covers how to divide the history into in-sample and out-of-sample windows.
2. Add the filter with a threshold fixed in advance, for example the common ADX level of 25 or an ATR ratio of 1.5. Write the threshold down before you run anything.
3. Compare the two versions on both periods. The in-sample comparison tells you what the filter did to the data it was built on. The out-of-sample comparison is the one that counts.
4. Look at the trades the filter removed. List them and check their average result. A filter that removes mostly losing trades is doing what you intended. A filter that removes a random mix of winners and losers has only reduced the trade count, and a smaller trade count can make any curve look smoother by chance.
5. Move the threshold up and down by a few steps and rerun. The result should change gradually. A threshold that works at one exact value and fails at its neighbours was fitted to noise.

Why do regime models overfit, and how do you keep yours honest?

Regime models overfit because regimes change rarely, so a few thresholds tuned on a handful of regime changes can fit the past perfectly and still mean nothing. Ten years of daily data may contain only a dozen clear switches between trend, range and high volatility. A threshold chosen to catch those dozen events has learned the dates, not the market.

Compare it with a weather rule. If it rained on six days last year and you build a rule from six readings, you can always find a rule that predicts all six. The rule is fitted to those six days. The next year it fails, because six examples are not enough to separate a real pattern from coincidence. Bailey and colleagues (2014) show the same effect in backtesting: when many variants are tried on the same history, a strong in-sample result is easy to find and tends to shrink out of sample. A regime filter with two or three thresholds is exactly such a set of variants.

Four habits keep the model honest:

1. Fix the number of regimes and the thresholds before testing. Three regimes, standard thresholds, written down. Changing them after seeing a result is tuning by hand, and the overfitting in strategy generation article explains why that inflates every number that follows.
2. Compute the label only from completed bars and apply it to the next bar. Look-ahead bias is the use of information that was not available at the moment of the decision. If the indicator values at the close of bar N produce the label, that label can only affect decisions on bar N+1. Applying it to bar N's own entry uses the close to decide a trade that happened before the close.
3. Count the filter's parameters as part of the strategy. A strategy with four parameters plus a filter with two is a six-parameter strategy. Its in-sample result should be judged as such, and its walk-forward windows must include the filter in the optimization, not added afterwards.
4. Prefer thresholds with a story. An ATR ratio of 1.5 means bars are half again as large as usual. An ADX of 25 is the level most textbooks name. A threshold like 23.7 that came out of an optimizer has no story and will not travel to another market.

What does a regime model look like today for index futures?

A working regime model is a small table read once per day after the daily bar closes: three readings, three thresholds, one label, and a written rule for what each label changes. It takes a minute to fill in and it replaces the daily debate about what the market is doing with a repeatable check. When people search for the current US market regime, this table is the answer they are really looking for, computed on ES or NQ with the breadth of the underlying index.

The table has four parts:

1. Readings. The ATR ratio, the ADX value and the breadth percentage, all from completed bars. Breadth is a daily cash-market number; ATR and ADX can be daily or taken from the bar size your strategy trades.
2. Thresholds. The same fixed values you tested in the previous section. They do not move from day to day.
3. Label. Volatility is checked first. If the ratio is above its threshold, the label is "high volatility". Otherwise ADX and breadth decide between "trend" and "range".
4. Action. One line per label, written in advance: which rules run, and at what size. Nothing else is decided at the time of reading.

Keep a log of the daily labels. After a few months you can see how often the label changed, how long each regime lasted, and whether the actions you wrote down matched what you actually did. That log is also the raw material for the next round of testing, because it tells you how many regime changes your thresholds have really seen.

What this means if you use AlphaPrime: a regime filter is an entry condition like any other, so you can include trend-strength and volatility conditions with fixed parameter ranges in strategy generation, and let strategy validation run the filtered and unfiltered candidates through the same walk-forward windows and thresholds, so the filter is judged on out-of-sample data with its parameters counted as part of the strategy.

Frequently asked questions

What are the three market regimes?

The three practical regimes for index futures are trending, ranging and high volatility. Trending means price keeps moving in one direction with normal bar sizes. Ranging means price swings back and forth with normal bar sizes. High volatility means bars are much larger than their recent average, in either direction, and it takes priority over the other two labels.

Which indicator shows the current market regime?

No single indicator does, because the regime has two dimensions. ADX, or the distance of price from a long moving average, measures how directional the market is. ATR compared with its own longer average measures how large the bars are. Breadth adds whether the move is carried by most stocks or a few. Read all three against fixed thresholds.

Is a regime filter the same as a trend filter?

A trend filter is one kind of regime filter. It allows entries only when a trend measure, such as ADX or the slope of a moving average, is above a threshold. A full regime filter also checks volatility and often breadth, so it can switch a strategy off in a fast, wide-ranging market even when a trend measure looks fine.

How often should the regime label be updated?

Once per completed bar of the data the label is built on, and most traders use the daily bar. Breadth is a daily cash-market number, so a daily update is the natural rhythm. The label computed at the close applies to the next bar, never to the bar that produced it.

Can a regime filter be overfitted?

Yes, and easily, because a decade of data contains only a handful of regime changes for the thresholds to learn from. Fix the thresholds before testing, compare the filtered strategy with its unfiltered baseline on out-of-sample data, check that the removed trades were mostly losers, and confirm that nearby threshold values give similar results.

References

• Bailey, D. H., Borwein, J., López de Prado, M., & Zhu, Q. J. (2014). Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance. Notices of the American Mathematical Society, 61(5), 458–471.