Real-Time Crypto Data Aggregation: How It Works
"Real-time" is the most common phrase in crypto market data and the least examined. Every provider claims it. Almost none explain what it means when the same asset is trading on seventy exchanges at once, each publishing its own stream, in its own format, on its own clock.
Real-time from one exchange is easy. Real-time across the whole market is an engineering problem with trade-offs that most descriptions skip entirely. This article walks through how it actually works: what a single exchange feed looks like, why seventy of them don't simply add up, what has to be filtered out, and why the fastest possible price is rarely the right one.
What real-time means at a single exchange: WebSocket vs REST
Start with one venue. A modern exchange publishes market data two ways.
REST is request-and-response: ask for the current order book, get a snapshot back. Simple, stateless and slow, since every update requires a new request and rate limits cap how often you can ask.
WebSocket is a persistent connection: subscribe once, and the exchange pushes updates as they happen. In practice, real-time means WebSocket. Binance's spot WebSocket, for example, publishes trades and best bid/ask the instant they occur, order book depth updates at 100 millisecond or 1,000 millisecond intervals depending on the stream, and candle updates roughly every second.
The volume is substantial. In one published measurement, a Coinbase ticker subscription for Bitcoin delivered 136 messages in a minute, while a Binance raw trade stream for the same asset delivered 1,382 in the same minute, with bursts arriving inside the same millisecond. Multiply that across thousands of pairs and dozens of exchanges, and an aggregator is ingesting on the order of millions of messages per minute before it has done anything with them.
Even at a single exchange, real-time has a floor. Measured from an AWS datacentre in Tokyo, Binance's WebSocket data arrived with an average latency of about 4 milliseconds, with 99% of messages under 13 milliseconds. That figure is the physical minimum, the time for a packet to cross the network, and everything an aggregator does happens on top of it.
Why seventy exchange feeds don't add up
The naive picture of aggregation is: connect to every exchange, receive every price, average them. The reality is that every exchange's feed is different in ways that matter, and reconciling them is most of the work.
Start with recovery. When a connection drops, and connections drop, the question is how you know what you missed, and three large exchanges give three different answers. Kraken's WebSocket sends a full snapshot of the order book on subscribe, so a reconnect self-heals: current state, then deltas. Coinbase stamps every message with a sequence number, so a gap is detectable even if it isn't automatically filled. Binance's raw trade streams send no snapshot on subscribe, but each message carries a monotonically increasing per-symbol trade ID, so a gap can be detected but must be filled from REST. An aggregator needs a strategy for each.
Then there are forced disconnects and limits. Binance closes every WebSocket connection after 24 hours regardless of activity, so an aggregator that isn't expecting this loses the feed once a day and must reconnect and re-synchronise. Binance also caps incoming messages at five per second per connection and 1,024 streams per connection, with a limit of 300 new connections per five minutes per IP, and every other exchange has limits of its own. Stay within all of them, for every exchange, simultaneously, or lose data.
Symbols are the next problem. Bitcoin appears as BTC on most exchanges and has been listed as XBT on some. Ether on Ethereum is ETH; wrapped Ether on a DEX is WETH; they trade at nearly but not exactly the same price. USDT on Tron and USDT on Ethereum are the same token on different chains, listed as separate pairs on some venues. An aggregator that fails to normalise symbols either drops data it should include or merges data it should keep apart.
Clocks differ too. Each exchange timestamps its own messages using its own clock. Some stamp the time the trade matched; some stamp the time the message was emitted; those can differ by tens of milliseconds. Clocks drift. An aggregator computing a price "as of" a specific instant has to decide whose instant it is, and typically has to stamp every message with its own receive time as well, so it can reason about the market on one consistent timeline.
And quality differs, which is the largest problem of all and deserves its own section.
Wash trading and fake volume in exchange data
Not every trade reported by an exchange is a genuine trade.
In March 2019, the asset manager Bitwise presented an analysis to the US Securities and Exchange Commission, studying reported Bitcoin volume across 81 exchanges. Its headline finding was that 95% of reported Bitcoin trading volume was fake or non-economic, meaning wash trading or volume manufactured to inflate an exchange's apparent liquidity. Bitwise identified just ten exchanges with genuine volume, and estimated the real market at around $273 million a day against reported figures of roughly $6 billion.
That number is a 2019 snapshot of a younger, less regulated market, and the share of fake volume has declined as regulation and scrutiny have increased. The peer-reviewed evidence since then still puts it high on the venues that matter least: Cong, Li, Tang and Yang, in a 2023 study in Management Science titled Crypto Wash Trading, applied statistical and behavioural tests to exchange trade data and estimated that wash trading averaged over 70% of reported volume on unregulated exchanges in their sample, while regulated exchanges showed little. In August 2020, South Korean regulators charged the exchange Coinbit with faking more than 99% of its reported volume.
The practical point stands. An aggregator that treats all reported volume as real will weight fake trades alongside genuine ones and produce a price that reflects the wash trader's intent. Fake volume remains a live problem on smaller and less regulated venues, and any aggregation method has to account for it.
Filtering anomalous trades and drifting venues
Before any aggregation, bad data has to be removed. There are two distinct failure modes, and they need different filters.
The first is the individual anomalous trade: a fat-finger order, a wash trade at an artificial price, a stale message arriving late. A single print far from the market should not move the aggregate. A price-based filter handles this: for each pair, compute the statistical distribution of recent prices and reject any new trade that falls outside a confidence interval around the mean. Blocksize's published methodology uses a threshold of 3.5 standard deviations for VWAP and 3.0 for bid/ask; anything beyond is discarded as anomalous.
That filter has a known edge case. If the market genuinely jumps, the first trades at the new level look like outliers relative to the old distribution, and a naive filter would reject them all. The fix is a jump detector: if several consecutive trades fall on the same side of the interval with meaningful volume behind them, the filter recognises a real move and re-initialises around the new level instead of excluding it.
The second failure mode is a whole exchange that has drifted. A venue with a regional premium, thin liquidity, or a stale feed can sit consistently off-market. An exchange-based filter handles this: compute the venue's own VWAP, compare it against the VWAP of every other venue combined, and exclude the venue if the gap exceeds a threshold, typically two standard deviations of the combined sample. The venue is re-included when it returns.
This filter also has limits worth knowing. It only works with at least three venues reporting; with fewer, there is no "rest of the market" to compare against. And if it flags multiple venues at once, it stands down, because a wide spread across many exchanges is usually a genuine market dislocation and not several simultaneous errors.
Volume weighting and liquidity-adjusted windows
After filtering, the surviving data has to be combined. How it is combined determines what the aggregate means.
A simple average treats every venue equally. A thin exchange with a handful of trades gets the same weight as a deep one with thousands. The obvious problem is that small venues move the result disproportionately; the less obvious one is worse: it makes the aggregate easy to manipulate by trading on the smallest, cheapest venue in the set.
Volume weighting gives each trade influence proportional to its size, and each venue influence proportional to its traded volume. A trade of ten Bitcoin moves the aggregate ten times more than a trade of one. "Volume-weighted average price" means exactly this, and it is the standard for a reason: it makes the aggregate reflect where real money is actually changing hands. How VWAP relates to the other price types is covered in Crypto Prices Explained.
Liquidity-adjusted windows handle the fact that pairs differ enormously in activity. Bitcoin against USD trades thousands of times a minute; a small altcoin might trade a dozen times an hour. A fixed time window that works for one is meaningless for the other. Instead, the window for each pair adapts to the number and volume of events observed. Blocksize's methodology requires at least ten events and USD 1,000 of volume before applying statistical filters, extends the window up to 120 minutes for thin pairs, and caps it at 1,000 events for active ones.
Speed vs completeness in real-time aggregation
Every step above costs time.
Waiting for messages from seventy exchanges takes longer than reading one. Normalising and timestamping takes processing. Filtering requires enough data to establish a distribution. Weighting requires the data to arrive. An aggregated price is, by construction, slower than any single input to it.
Here sits the trade-off at the heart of real-time aggregation: completeness and speed pull in opposite directions. The most complete price is never the fastest, and the fastest is never the most complete. A provider claiming both has either sacrificed one quietly or is measuring latency from a point after the hard work is already done. We set out the same tension as breadth, depth and latency in What Is Price Accuracy in Crypto.
The honest question is where on that curve a given use case needs to sit. For a high-frequency trader arbitraging between two venues, milliseconds matter and a single-exchange feed is the right tool. For anyone valuing a portfolio, benchmarking execution, or settling a contract, how right the price is matters more than how fast it arrives, and a few hundred milliseconds spent getting it correct costs nothing, because nothing downstream needs it faster.
Blocksize's aggregated VWAP is published at sub-250 millisecond latency, orders of magnitude slower than a colocated equity feed and orders of magnitude faster than anything that consumes it: blockchain settlement takes seconds, oracle feeds update on deviation thresholds or heartbeats measured in seconds or minutes, and valuation runs on schedules measured in hours. Past the point where the consumer can act, additional speed has nothing to act on.
Real-time aggregation vs periodic aggregation
Precision about what "aggregated" means matters, because the word is used loosely.
Some widely used aggregated price feeds update every few seconds instead of continuously. CoinMarketCap's documented WebSocket price channel, for example, pushes aggregated prices roughly every five seconds for the top 500 assets and every fifteen seconds for the rest. For a price display, that cadence is perfectly reasonable. Treating it as a real-time aggregated feed, for execution, for benchmarking, for anything time-sensitive, produces a price that is systematically stale.
The distinction is between an aggregate computed periodically from cached inputs and one recomputed continuously as inputs arrive. The first is cheaper to run and fine for many purposes. The second is what real-time aggregation should mean.
The result: one market-wide price per pair
The result of all of this is one price per pair, per moment, that reflects trading across the whole market instead of one venue's view of it, filtered of anomalies, weighted by where real volume sits, and timestamped on a consistent clock.
For Bitcoin against USD, that aggregate sits within a corridor about 0.167 basis points wide at the 95th percentile, which is to say it is extremely stable. Individual exchanges, even the deepest, fall inside that corridor only about a quarter of the time, as our study across 23 exchanges in Rethinking Crypto Data shows. The aggregate is a more precise estimate of the market than any single exchange can produce alone, because each of them is one input to it.
Blocksize real-time market data aggregation
Blocksize does exactly what this article describes, across more than 70 centralised exchanges and DEXes and 9,000+ trading pairs. Each data type is taken from its own source: VWAP from executed trades, consolidated bid/ask from top-of-book updates, state prices read directly from pool contracts. Each is normalised, filtered with the price-based and exchange-based methods above, volume-weighted, and published over WebSocket at sub-250 millisecond latency with a 99.9% uptime guarantee.
The methodology for every product is published, including the filter thresholds, the liquidity-adjustment rules and the edge-case handling. The same aggregated feed is published into decentralised oracle networks, where independent operators validate it against consensus before accepting it on-chain, a second layer of quality control that no single-exchange feed has and that we describe in How Oracle Price Feeds Work.
Frequently asked questions
What is real-time crypto data aggregation? The process of collecting price and trade data from many exchanges simultaneously, normalising it into a common format and timeline, filtering out anomalous trades and drifting venues, and combining what remains, typically volume-weighted, into a single market-wide price, continuously and with minimal delay.
Why not just use one exchange's real-time feed? Because no single exchange is the market. Crypto has no consolidated tape, the same asset trades at different prices on every venue, and any one venue can diverge from the rest during outages, thin liquidity, or regional imbalances. A single-exchange feed is fast and precise about that exchange, and silent about everything else.
What is the difference between WebSocket and REST for crypto data? REST is request-and-response: you ask, the exchange answers, and rate limits cap how often. WebSocket is a persistent connection where the exchange pushes updates as they happen. Real-time data in practice means WebSocket; REST is used for snapshots, history and gap-filling.
How does aggregation handle wash trading and fake volume? Through filtering. Individual anomalous trades are rejected by statistical outlier detection; venues whose whole price series drifts from the rest of the market are excluded until they return. Volume weighting also limits the influence of small venues where fake volume is most common.
Is aggregated data slower than single-exchange data? Yes, necessarily. Waiting for multiple sources, normalising, filtering and weighting all take time. The question is whether the extra latency matters for the use case. For most valuation, benchmarking and settlement purposes it does not, because the downstream process runs far slower than the feed.
What is a liquidity-adjusted time window? A calculation window that adapts to how actively a pair trades, so that filters have enough data to be statistically meaningful. Active pairs use short windows; thin pairs use longer ones, subject to caps on age and event count.