Backtest Metrics II

In the previous lesson we learnt about backtesting and three important metrics that help us decide whether a backtested strategy is any good: the average return, the win rate, and R.

There are other metrics that matter just as much, and this lesson is about them.

Let's take a sports example again. How do we decide whether a player is a legend?

Simple. They performed across years, consistently. Someone who was good for a single season is not a legend.

It works the same way for strategies. If a strategy performs well for one month, that does not make it a good strategy. It might just be a fluke.

Or think about ordering food online. You look at the restaurant's rating, but at the same time you look at how many people actually rated it. Four and a half stars from nine people is a different thing from four and a half stars from four thousand.

How many trades, and over how long

For a strategy backtest to mean anything, it has to have been tested across years, and it needs to have taken many trades. Not two or three.

So the first metric here is the number of trades, and it is one of the most important on the whole scorecard.

• For strategies working on intraday timeframes, a good backtest should have hundreds of trades.
• For a day-timeframe strategy, a lower number is reasonable, because each trade takes days rather than minutes.

The second is the evaluation window: how long a stretch of history the test covers. For a day-timeframe strategy this should ideally run to more than ten years, so the strategy has lived through more than one kind of market.

Both of these are really one idea, sample size, seen from two angles. A great average over five trades is noise. The same average over five hundred is evidence.

Two different kinds of pain

Now for a metric that is very often explained wrong, including by people who should know better. There are two separate things here, and they get called by the same name.

A losing streak is the number of losing trades that came one after another with no winner in between. Think of a team losing seven matches in a row.

Maximum drawdown is the deepest fall from a high point in your account down to the low that follows, before you make a new high. It is measured in money, not in trades.

They are not the same number and they do not measure the same thing.

A losing streak tests your patience. Seven losses in a row is seven chances to decide the strategy is broken and abandon it, even if each loss was small and the strategy is fine.

Maximum drawdown tests your capital, and your stomach. A strategy with a 50% drawdown can take your account from $100,000 to $50,000 without a single thing having gone wrong with the rules. It is behaving exactly as designed. You just have to survive it.

You can have a long streak with a shallow drawdown, if your losses are small. You can also have a brutal drawdown from only two or three trades, if your losses are large. Read both.

Profit factor

There is one more number worth knowing, and it answers a question expectancy does not.

Profit factor is everything the strategy won divided by everything it lost.

Profit factor = gross profit / gross loss

If a strategy made $12,000 across all its winners and lost $8,000 across all its losers, the profit factor is 1.5. Every dollar of loss bought you a dollar fifty of profit.

• Below 1: the strategy lost money.
• At 1: it broke even.
• Above 1: it made money, and the higher the number the more comfortably.

Expectancy tells you what one average trade is worth. Profit factor tells you how hard the whole strategy had to work for it. A strategy can have a positive expectancy and still have a profit factor barely above 1, which means it is scraping through, and a small change in costs or slippage could tip it under.

This is the number to compare two strategies on, along with expectancy per trade. Comparing them on total return is close to meaningless, because total return depends on how long the test ran and how often the strategy traded.

What the scorecard looks like now

So across the last two lessons we have collected:

1. Average return per trade
2. Win rate
3. R, the risk to reward ratio
4. Expectancy, in R
5. Number of trades
6. Evaluation window
7. Longest losing streak
8. Maximum drawdown
9. Profit factor

None of them is the single number to optimise, and pretending there is one is how traders talk themselves into fragile systems. A higher expectancy often comes with a deeper drawdown. A smoother account curve often comes with a smaller edge.

The honest question is not "is this strategy good?" It is "is this strategy good, and can I actually sit through its worst stretch without quitting?" Those are different questions, and the second one is the one that decides whether you make any money.

In the next lesson we look at why the same strategy can pass all of these tests in one kind of market and fail in another.