Why Rithmic Connection Does Not Support Market Order Aggregation

Article author
Arthur
  • Updated

Rithmic Market Data — Aggregated Market Big Trades

Warning: ExoCharts does not support Aggregated Market mode for Big Trades on Rithmic data connections. On Rithmic, Big Trades always runs in Limit mode, because Rithmic data can never match sweeps identically live and on reload. If you need reproducible aggregated sweeps, switch your data connection to dxFeed.

Here's why.

How Rithmic timestamps a trade

Rithmic provides both real-time and historical data through two completely different protocol messages, stamped by two separate clocks.

Real-time. A live tick comes in the LastTrade message (template 150). That message contains two different time sources:

  • ssboe / usecs — Rithmic's gateway clock, set when the gateway receives the message.
  • source_ssboe / source_usecs / source_nsecs — the exchange (CME) clock, passed along from the matching engine.

When processing the live feed, Rithmic timestamps each trade using its own gateway clock (ssboe * 1000 + usecs / 1000; microseconds truncated). It has the source clock available but does not use it for trade timestamps.

Historical. Recorded trades come in the ResponseTickBarReplay message (template 207), stamped with data_bar_ssboe / data_bar_usecs — Rithmic's data_bar clock, set when it writes the trade to its history database. This message does not carry the source clock. There is no source_ssboe, no nanoseconds, no trade id or order id. Its time-of-trade is given by two fields, data_bar_ssboe and data_bar_usecs, translated to milliseconds exactly like the live feed: data_bar_ssboe * 1000 + data_bar_usecs / 1000, truncated to the millisecond.

So the same CME trade gets timestamped twice — by two different Rithmic subsystems, at two separate points in the ingest process. There are two different clocks on Rithmic, not one clock read at two different times.

Why this produces a discrepancy

Both paths truncate microseconds the same way, so truncation is not to blame. The two feeds are given slightly different input. The trade is stamped several microseconds apart at the gateway versus when it's written to the history data bar, so ssboe / usecs and data_bar_ssboe / data_bar_usecs disagree before any truncation happens. Most of the time both truncate to the same millisecond. Occasionally the two timestamps land on opposite sides of a 1-millisecond boundary, and the truncated results come out one millisecond apart — one side sees the trade at …267, the other at …268.

This cannot be fixed after the fact, because:

  • The historical message does not contain the exchange's source clock, so the two time streams share no common reference.
  • The historical message contains no trade id or order id, so there's no way to match a live trade to its counterpart in the history feed.
  • Historical trades are stored at millisecond precision only; the microseconds were thrown away when the database was written.

There is nothing left to compare against to realign the feeds.

And to be clear, CME is not at fault. Its MDP 3.0 matching engine timestamps every trade with a nanosecond TransactTime (microsecond precision) and assigns each trade a Globex order number. Both reach Rithmic on the live feed. Neither makes it out of Rithmic's historical trade message. The information needed to correlate live and historical trades exists at the exchange — it's lost inside Rithmic.

Effect on Big Trades

In Aggregated Market mode, Big Trades combine all trades at the same instant into a single volume "sweep." Grouping is done at the millisecond, so shifting a timestamp by one millisecond flips a trade's placement in the grouped data.

An extreme case: 8 contracts BUY at one price in one burst. By luck of the draw they all share one millisecond live, and ExoCharts shows a single 8-contract sweep. After reloading, Rithmic shifted the millisecond one way on some of those trades and the other way on the rest — five land on one side of the boundary, three on the other. The 8-contract burst you saw live is now two separate sweeps on reload, one of 5 and one of 3. If your display filter is set to "only show sweeps of 6 or more," you'll see one bubble live and zero bubbles after reload. Both feeds contain the same trades, at the same price, with the same volume, by the same aggressor side. The only difference is how they were grouped.

These are reproducibility issues, not data-accuracy issues, and they affect only Aggregated Market Big Trades, which buckets at the millisecond. Candles, footprint, volume profile, CVD, and Limit Mode Big Trades are all unaffected, because those group on price, aggressor side, size and sequence — and Rithmic reports all of those consistently live and on reload.

dxFeed is not affected

dxFeed does not have this problem. Each TimeAndSale event is given a single index when it enters the ticker plant, and the historical contract replays those same indexed events back out. One clock, one timestamp, both live and on replay. Truncation is deterministic — the same input always produces the same millisecond, and there is no second timestamp to contradict it. Aggregated Market sweeps reproduce exactly between live and reload on dxFeed.

This is why Aggregated Market mode is available on dxFeed. If your analysis requires reproducible aggregated market sweeps, dxFeed is the recommended connection type.

A note on Rithmic R | API+

R | API+ is Rithmic's native (non-WebSocket) API, offering microsecond-granularity live timestamps and access to advanced live market-data features. It does not resolve this issue. R | API+ and the R | Protocol API used here are two interfaces onto the same R | Trade Execution Platform — the same ticker plant and the same history plant. Historical ticks on R | API+ are served from that same history database, stamped by the same data_bar clock, with no source clock and no trade id. The divergence is a property of Rithmic's plant architecture, not of the API used to reach it, so R | API+ carries the identical live-vs-reload mismatch and aggregated sweeps still cannot be reproduced. It is an alternative for richer live data only — not a fix for aggregation. For reproducible aggregation, use dxFeed.

Reference

Sierra Chart reached the same practical conclusion. They do not support Rithmic market data, historical or real-time, and advise customers to use their own direct-from-the-exchange feed instead. Sierra Chart lists historical bid/ask volume — the aggressor-side data this feature relies on — as unsupported on Rithmic.

https://www.sierrachart.com/index.php?page=doc%2FRithmic.php

0 likes