fix fetch_ohlcv bug

[boutchitos]

I was refreshing candles downloaded for RENDER (bitget) and did notice a weird bug. I was downloading since July 23, and near Novembre 11, 12, 13, etc. I got empty data, and then data starts coming in again.

I did debug ccxt and put it on verbose. I notice that fetch_ohlcv switch between two endpoints /api/v2/mix/market/candles and /api/v2/mix/market/history-candles while downloading a period.

So, for old candles they use the history endpoint, and then they switch to the other endpoint. So, for Novembre 11, 12, etc. they just switch to using /api/v2/mix/market/candles and this doesn't work.

I found an issue on for this: ccxt/ccxt#24890 and they have merge a workaround. So, I updated ccxt and use useHistoryEndpoint.

[RobotTraders-old]

Okay that's strange, I had never witnessed that. I must use that workaround for my new code version. Any chance you could copy paste your updated fetch_olcv method please?

[boutchitos]

Okay that's strange, I had never witnessed that.

This could be a new bug on Bitget side, or may be a change in their limitation. For /api/v2/mix/market/candles their is a 83 days limitation. Could they have change that recently?

Anyway, a safe approach could be considered. I was already considering a candles validator to make sure my CSV files are complete. Could worth it if more exchanges are to be integrated in the code base. The exchange could be buggy, and the ccxt layer also.

I must use that workaround for my new code version. Any chance you could copy paste your updated fetch_olcv method please?

fetch_ohlcv is in the ccxt package. I just update to most recent release. See requirements.txt.

[boutchitos]

For your other code bases, if it is for recent candles, like trade computing with average period like 4 to 20, you will not experience the bug. With /api/v2/mix/market/candles endpoint there is a limit of 1000 candles.

[RobotTraders-old]

No I don't think it's the exchange that is buggy, but rather ccxt (they tend to often change versions, which can add an extra layer of complications).

That is not to say that sometimes some candles are badly recorded by the exchanges in their history, or you can have several recordings at the same timestamp. I've seen this in binance data for example.

Anyway, in my mind, if I went for private methods (with or without ccxt) for backtesting I would stick to the https://www.bitget.com/api-doc/contract/market/Get-History-Candle-Data endpoint, while for a live trading bot https://www.bitget.com/api-doc/contract/market/Get-Candle-Data could be more efficient.

I've encorporated a check for missing data method in the new backtest code.

Note that the /api/v2/mix/market/candles endpoint does not have a limitation of a 1000 candles actually, but rather a past days limit which depends on the timeframe.