Backtesting in Crypto: Why Your Data Source Decides the Result
Every backtest produces a number. Most of them are wrong, and the reason is rarely the strategy.
A backtest runs a trading strategy against historical data to estimate how it would have performed. The logic is sound. The problem is that the result is only as good as the data it runs on, and in crypto the data has failure modes that most backtesting guides never mention. A strategy can look brilliant on a dataset that quietly excludes every coin that died, or that reflects one exchange's quirks instead of the market, or that assumes fills at prices that were never actually available.
This article covers the biases that distort crypto backtests, with particular attention to the one that gets the least discussion: where the price data came from.
What a backtest measures
A backtest asks one question: if this strategy had been running over this period, what would have happened?
Answering it honestly means the data has to represent what a trader in that period would actually have seen and been able to trade. Every departure from that pushes the result away from reality, almost always in the flattering direction: every piece of information the trader couldn't have had, every price they couldn't have got, every asset they couldn't have known would survive.
The biases below are the main ways that happens. The first three are well known in traditional finance and apply with extra force in crypto. The fourth is specific to how crypto markets are structured, and most backtests ignore it entirely.
Survivorship bias in crypto backtesting
If you download today's list of the top hundred cryptocurrencies and backtest a strategy on their history, you are testing on winners. Every coin in that list survived. The ones that collapsed, were delisted, or went to zero aren't there, and a strategy that would have held them is spared the loss.
The effect was first quantified for mutual funds by Elton, Gruber and Blake in 1996, in the Review of Financial Studies, who showed that databases which drop dead funds overstate the performance of the survivors. In crypto the scale is unusual. A study by researchers at the University of St. Gallen and the University of Liechtenstein, Survivorship and Delisting Bias in Cryptocurrency Markets, found that only about one in four cryptocurrencies that existed in 2014 was still listed by the end of 2021. When the authors included delisted coins in their analysis, the measured size premium, the excess return from holding smaller coins, fell from 2.84% to 1.89%. A third of the apparent effect was survivorship.
A separate dataset built for a 2025 academic study covering December 2013 to September 2025 catalogued 18,622 active coins and 29,230 that had become inactive or been delisted. More coins have died than survived.
The fix is point-in-time universe construction. At each date in the backtest, the strategy should only see the coins that were listed and tradable on that date, and it should experience what happened to them afterwards, including delisting. More work than downloading a current list, and it separates a real result from a fictional one.
Look-ahead bias in crypto backtesting
Look-ahead bias occurs when a backtest uses information that would not have been available at the time of the decision. Banz and Breen identified it in 1986 in the Journal of Finance, showing that apparent anomalies in stock returns could be produced simply by using accounting data before it was actually published. The mechanism is often subtle.
A strategy that rebalances at the daily close, using the day's closing price to decide, is using information the trader could not have acted on, because the close is not known until the day is over. A strategy that filters for coins above a certain market cap, using today's market cap data, is applying a filter with hindsight. A strategy that uses a "clean" price series where anomalies have been retrospectively corrected is trading on a version of history that never existed in real time.
In crypto, a common form is using data that was revised after the fact. Exchanges correct erroneous trades, data providers backfill gaps and fix outliers, and reference rates are recalculated when methodologies change. A backtest on the corrected series performs better than a live strategy would have, because the live strategy would have seen the errors.
The fix is to ensure every input to a decision at time t was actually available at time t, and to be suspicious of any data series that looks unusually clean.
Transaction costs and slippage in crypto backtests
Most backtests assume trades execute at the recorded price. Real trades do not.
Every trade pays fees, crosses a spread, and, if it is large relative to available liquidity, moves the market against itself. A backtest that ignores these costs overstates returns, and the overstatement grows with trading frequency. A strategy that trades once a month can absorb realistic costs; one that trades hourly may be entirely consumed by them.
Crypto adds a complication. Spreads and liquidity vary enormously across assets and across venues. After the January 2024 approval of US spot Bitcoin ETFs, Kaiko's research found BTC-USD spreads below one basis point on the largest US exchanges in normal conditions, while thin tokens on small venues can carry spreads of a percent or more. A cost model that applies one assumption across the whole universe is wrong for most of it.
Slippage, the difference between the expected price and the actual fill, is worse to model, because it depends on order size relative to the liquidity that existed at that moment. Historical trade data shows what prices traded at. It does not show what would have traded if your order had been in the book. The same problem sits at the centre of measuring live execution, which we cover in Execution Quality in Crypto.
The fix is a cost model that varies by asset and venue, uses historical bid/ask spread data instead of a flat assumption, and applies a conservative slippage estimate scaled to order size. When in doubt, overestimate costs; a strategy that survives pessimistic assumptions is more likely to survive reality.
Single-venue bias: why one exchange is not the market
Of the four, this bias is specific to crypto, and most guides skip it.
Crypto has no consolidated tape. The same asset trades on dozens of exchanges at once, each with its own order book, its own liquidity, and its own price. These prices are usually close and sometimes not. They diverge during volatility, differ by region, and gap wildly on thin pairs. We explain the structure of this in Rethinking Crypto Data.
The consequence for backtesting is direct. Much of the historical crypto data in circulation comes from a single exchange, because that is how exchanges publish it. A strategy backtested on one exchange's price history has learned that exchange: its outages, its liquidity profile, its moments of divergence from the wider market, its regional flow. The backtest treats all of that as "the market," and the strategy optimises against it.
When the strategy goes live, it trades against the actual market, which is larger than any one exchange. Or it trades on that same exchange, but the signals it learned were partly exchange-specific noise that does not repeat. Either way, performance degrades, and it is hard to see why, because the backtest looked clean.
It takes several concrete forms. Exchange-specific events get read as signals: a flash crash on one venue, such as Ether falling to roughly $700 on Kraken on 22 February 2021 while trading above $1,400 elsewhere, appears in that exchange's history as a real price move, and a strategy trained on that data learns a pattern that existed on one venue for a few minutes. Liquidity assumptions fail to transfer, so a strategy sized for one exchange's order book depth mis-sizes on another, or in aggregate. Regional venues carry persistent premiums or discounts to the global market, the Korean "Kimchi premium" among the best documented, and a strategy trained on such a venue learns the premium as a feature of the asset when it belongs to the venue. And missing data gets treated as no data: an exchange outage leaves a gap in that exchange's history, and a single-venue backtest either skips those periods, removing exactly the volatile moments a strategy most needs to be tested against, or fills them with interpolated values that never traded.
The fix is to backtest against data that represents the market instead of one participant in it: a price aggregated across venues, with outliers filtered and single-exchange anomalies removed, built the way we describe in Real-Time Crypto Data Aggregation: How It Works. For execution modelling, use historical bid/ask data from the venue you actually intend to trade on. The two serve different purposes. The aggregated series tells you whether the signal is real; the venue-specific series tells you what it would have cost to act on it. A rigorous backtest uses both.
Overfitting on short crypto history
Overfitting is the general failure of which the biases above are special cases. A strategy with enough parameters, tuned against enough history, will find patterns in any dataset, including patterns that are noise. Bailey, Borwein, López de Prado and Zhu made the point sharply in 2014 in the Notices of the American Mathematical Society: given enough trials against the same data, a researcher will always find a strategy that backtests well, and the more configurations tried, the higher the in-sample result that can be reached by chance alone.
Crypto makes this easier than most markets because the usable history is short. Bitcoin has meaningful trading history from around 2013, most major assets from 2017 or later, and much of today's market from 2020 or later. A strategy tuned over three years of data has been tuned over perhaps one full market cycle. It has learned that cycle, and mistaken it for the market.
The fix is the standard discipline: hold out a portion of data the strategy never sees during development, test on it once, and treat a strategy that performs well in-sample and poorly out-of-sample as a failed strategy, whatever the temptation to tune it further. Simpler strategies with fewer parameters generalise better. And a strategy whose backtest depends on a specific data source is a warning sign. If it works on aggregated data and not on single-venue data, or the reverse, the result is telling you something about the data, and nothing about the strategy.
What a backtest needs from its data
Pulling the above together, a crypto backtest that can be trusted needs data with five properties: a point-in-time universe, meaning the set of tradable assets at each date including the ones that later disappeared; unrevised history, meaning prices as they were seen at the time and not as they were corrected afterwards; realistic costs, meaning bid/ask spreads and slippage estimates that vary by asset, venue and order size; a market-wide price series aggregated across venues and filtered for anomalies, so the strategy learns the market and not one exchange; and a documented methodology, so that when a result is questioned, the data behind it can be explained.
Free datasets rarely offer more than one of these. Commercial datasets typically offer some. Knowing which are missing before trusting a result is worth more than any single test statistic.
Blocksize data for crypto backtesting
Blocksize's contribution to this problem is the market-wide price series. Its VWAP and consolidated bid/ask are aggregated in real time across more than 70 centralised and decentralised exchanges, with anomalous trades filtered out and drifting venues excluded, and the methodology for each product is published. What each of those price types measures, and which fits which purpose, is covered in Crypto Prices Explained.
For backtesting, that provides the reference a strategy should be tested against: the price the market as a whole was trading at, and not the price on one venue. Historical VWAP and bid/ask snapshots are available, with the 30-minute VWAP struck on the clock for repeatable, timestamped analysis; depth of history depends on plan.
The same principle applies here as in valuation and execution. A strategy validated against the market has learned something real. One validated against a single exchange has learned that exchange, and will find out the difference when it goes live.
Frequently asked questions
What is survivorship bias in crypto backtesting? Survivorship bias occurs when a backtest uses only assets that still exist today, excluding those that were delisted or went to zero. Because most cryptocurrencies ever listed have since become inactive, this can materially inflate backtest returns.
What is look-ahead bias? Look-ahead bias is using information in a backtest that would not have been available at the time of the decision: closing prices before the close, retrospectively corrected data, or filters applied with hindsight.
Why does the choice of exchange matter for crypto backtesting? Because crypto has no consolidated tape, each exchange's price history reflects that exchange's liquidity, outages and regional flow as well as the market. A strategy backtested on one venue learns that venue's quirks and may not generalise.
Should I backtest on aggregated or single-exchange data? Both, for different purposes. Aggregated data across many venues tests whether a signal is real at the market level. Single-venue bid/ask data from the exchange you intend to trade on estimates what executing the signal would have cost.
How should transaction costs be modelled in a crypto backtest? With asset- and venue-specific bid/ask spreads instead of a flat assumption, plus a slippage estimate scaled to order size. Crypto spreads range from under a basis point on major pairs to a percent or more on thin ones, so one number does not fit.
How much history is enough for a crypto backtest? More than most strategies have. Most crypto assets have only a few years of meaningful history, often covering a single market cycle. Out-of-sample testing on held-back data matters more than the length of the in-sample period