Bug - momentum indicator backtest

The following strategy produces crazy results in the backtest (over 1000%) per month but needless to say never works in real life:

German30
Entry - momentum changes direction upward; apply to open; period 20
Exit - momentum changes direction downward; apply to open; period 20
Stop loss - 600 but really never used
Trading frequency- 30 min

5 Likes

It seems that somehow momentum calculation in backtest is wrong.

Yes, I was hoping that @Nikola would respond to say what the problem was and what they were going to do about it. I suspect that people are losing money running strategies that look good in the backtest but fail in reality

On second thought, it might be slippage that causes momentum strategy to get such bad results in real life? I’ve heard that in very extreme cases it can be up to 40 seconds.

I find it hard to believe that slippage could account for such a large difference. Anyway best wait for a comment from PQ

@Nikola is this a problem with the momentum indicator? Or a problem with the backtest that effects all indicators to some extent?

I’ll try to look at Momentum on Monday. Been terribly busy with engine improvements, sorry!

1 Like

Thanks, I wasn’t expecting a reply today. Thanks also for all your (and anyone else working on ProQuant) efforts. to be clear I get the same results across all time frames and all instruments that I’ve tried when momentum is applied to open.

from my testing its usually most broken with level 100 when the level line is used, and a period of 1

I’ve discovered a bug related to Momentum backtesting, specifically when “Apply to” is set to “Open”, which leads to a broken backtest due to the signal calculator using the “close” price of the newly opened candle to calculate Momentum, which would be looking into the future, as, obviously, a freshly opened candle cannot have a close price.

It will be fixed and deployed live today. I will write here whenever we do so.

Apologies for the bug and the long time it took to get around to discovering and fixing it.

3 Likes

Great, thanks for the explanation. I look forward to the update.

The bug has been fixed and strategies that include the Momentum indicator with “Apply to” set to “Open” should no longer produce incorrect backtests.

2 Likes

Awesome! It might be a good idea to send a notification advising users to double check their backtests since we didn’t have to do an update in the app so they might not know. I’m looking through some long backtest strats, so many backtests have crashed!

2 Likes

Hey Nikola,

Does this affect the generator?

Cheers

Sure, the bug has been fixed for strategies produced by the generator as well.

1 Like

Dear Gents,

I believe that there was no bug with the indicator. The only issue with the backtest was that it showed exactly the entry in the candle which is in real trading considered as the signal. So while the backtest shows an entry at 11:00 the real trade would take place at 12:00.
Now with the fix the results have nothing to do anymore with the real trading. My real trading results don‘t match the backtest results.

Best
Aykut

@TheAyk there was definitely a bug. Had you created the strategy above before the fix you’d have seen a backtest with extraordinary results in terms of monthly returns and win/loss ratio. When run that didn’t materialise as you would expect from such a simple strategy.

I don’t know if fixing it has caused further bugs, whether PQ has fixed the wrong thing or something else is at play with your results. Perhaps @Nikola can shed some more light on that.

2 Likes