Your interactive trading laboratory!
 • 
7 users online

non-random backtesting for production
This is a Feature Request with 1 vote

I'm running a Meta model in production. I have it set up as a backtest over the past several months (it uses some indicators that require that much lookback). The trouble is that the backtest is not precisely repeatable. So one day I place some buy orders (ex. Buy AAPL 24 shares at market open), and the next day the backtest produces some sell orders that don't precisely match the open long positions (ex. Sell AAPL 22 shares at market open). I have several models running on the same universe of stocks, and it can be very confusing to figure out exactly which open positions to sell. In most cases it is not critical which gets sold, but it is inconvenient that my signals do not align with my open positions.

Request: either advice to help me find a way around this, or perhaps we could have a more deterministic backtesting mode, so that I can somehow get more accurate signals for closing positions that match the number of shares I have open.

Attachment

Cancel

Responses

Some tips:

  1. The starting date of your backtest must be fixed i.e. not some Most Recent N ... Better put a fixed Date Range with the To date set to the future.

  2. If using indicators calculated using their own previous values (like RSI or EMA), don't rely on their initial values. Estimate: after 3-4 times the longest period of such indicator.

  3. Avoid a Percent Equity position sizing that might result in near 100% equity usage. Better use Fixed Amount / Number of Shares for production, readjusting the number on a daily (weekly...) basis.

Some tips: 1. The starting date of your backtest must be fixed i.e. not some Most Recent N ... Better put a fixed Date Range with the To date set to the future. 2. If using indicators calculated using their own previous values (like RSI or EMA), don't rely on their initial values. Estimate: after 3-4 times the longest period of such indicator. 3. Avoid a Percent Equity position sizing that might result in near 100% equity usage. Better use Fixed Amount / Number of Shares for production, readjusting the number on a daily (weekly...) basis.

Thanks Atmosfear,

  1. I do that.
  2. I'll try making my start date even farther back but set a standard StartIndex for all of the component models. Currently each starts when it is ready.
  3. Ahhh, good idea. Fixed #shares won't work for me, but Fixed Amount readjusted periodically would be ok.

Back to experimenting, thanks much.

Thanks Atmosfear, 1. I do that. 2. I'll try making my start date even farther back but set a standard StartIndex for all of the component models. Currently each starts when it is ready. 3. Ahhh, good idea. Fixed #shares won't work for me, but Fixed Amount readjusted periodically would be ok. Back to experimenting, thanks much.

Thank you again. With change #3 alone it seems more stable. I'll add #2 changes as well. So far so good.

Thank you again. With change #3 alone it seems more stable. I'll add #2 changes as well. So far so good.

Yes I can see it can be confusing. I'm glad the tips are helping, and possibly we can add some portfolio synch to try and match alert quantity with what's in a connected Broker account in the future.

Yes I can see it can be confusing. I'm glad the tips are helping, and possibly we can add some portfolio synch to try and match alert quantity with what's in a connected Broker account in the future.

Yes! I hope so (portfolio sync). I'm currently testing a MetaModel with several component models each using the same Universe of only about 30 stocks, and it's quite confusing to keep track of trades... I might have two or three different long positions open with say 100 total shares, and I get a signal to sell to close 33 shares, I have to double check everything (including the signals from several days back) to make sure that's the right amount of shares and from which component model did it originate.

Yes! I hope so (portfolio sync). I'm currently testing a MetaModel with several component models each using the same Universe of only about 30 stocks, and it's quite confusing to keep track of trades... I might have two or three different long positions open with say 100 total shares, and I get a signal to sell to close 33 shares, I have to double check everything (including the signals from several days back) to make sure that's the right amount of shares and from which component model did it originate.
Forum Tips

Please sign in if you want to participate in our forum.

Our forum uses Markdown syntax to format posts.

To embed code snippets, enclose them in [CODE][/CODE] tags.