The fidelity scorecard

The scorecard answers the campaign’s one question: does live behavior match the backtest? It compares what a paper session actually did (persisted equity snapshots + fills) against a shadow backtest — the same engine, the exact running strategy config, over the identical calendar window — and reports the gaps honestly. It is the go/no-go instrument before real money.

Find it on the Sessions tab, under the portfolio card. Hit Recompute to refresh; the worker refreshes stale daily bars (free vendors), replays each leg, and caches the result for six hours.

Where the data comes from

• Live side — the alert relay persists an equity snapshot from the telemetry heartbeat every ~5 minutes (plus a forced row at session open — the starting-cash anchor — and on a clean stop). Fills come from the session history. Restarted sessions with the same sleeve composition are stitched into one curve; a config change starts a fresh comparison window.
• Shadow side — run_book_backtest over the same window, starting from the leg’s first snapshot equity, using the exact params recorded when the session started. Sessions from before params capture fall back to the strategy registry defaults and are flagged with a warning (params_source: defaults).

Reading the verdict

Verdict	Meaning
green	Return gap ≤ 1pp AND trades match ≥ 80% (or both sides took no signals).
yellow	Return gap ≤ 3pp with a reasonable trade match — watch it.
red	Larger gap, or the backtest traded and live did not (missed signals) — investigate.
insufficient	Under two daily points — nothing honest to say yet.

Other honesty rules:

• Sharpe is gated until 20 daily points exist. An annualized Sharpe from a handful of days is noise; return, drawdown, and trade matching are the meaningful early metrics.
• Trade matching pairs live fills with shadow fills per (symbol, side) with a one-trading-day tolerance (a Friday-close signal filled live Friday matches the model’s Monday-open fill). Partial IB fills are aggregated to VWAP before comparison. The per-pair slippage (bps) blends timing and execution cost — it is labeled “vs model”, not “vs market”.
• A zero-trade leg still renders. If neither side traded, the flat overlay is itself evidence the gate/warm-up behave (“no signals yet — the backtest agrees”). If the shadow traded and live did not, that is a red verdict, not a blank card.
• The combined view is a dollar sum of the legs’ actual curves (live and shadow separately). Real capital embeds the risk-parity allocation, so no weight modeling is applied; equity-market holidays forward-fill while crypto keeps moving.

Intraday legs

A leg whose sleeves run at an intraday cadence (today: the 1-minute family) is compared per bar, not per day. Everything above still applies; the differences:

• Snapshot cadence follows the bar spec. The relay persists one equity snapshot per bar (60 s floor, 5 min ceiling) instead of every ~5 minutes, each stamped with the heartbeat frame’s own timestamp. Fills carry their true execution time.
• As-of alignment. Each shadow bar close pairs with the last live snapshot at or before it (plus a ~30 s heartbeat grace). A close with no snapshot within one bar + one snapshot period drops from both curves — a data gap is a gap, not a divergence.
• Per-bar trade matching. Fills aggregate per bar (only true same-bar partials fuse) and match within a one-bar tolerance on the shadow’s bar grid — a 10:02 entry and a 14:30 entry are separate trades, not one blended daily row.
• Verdict allowance. Intraday legs trade far more often, so per-fill costs compound: each matched trade widens the acceptable return gap by 0.02 pp (green; yellow gets 3×). The allowance widens tiers — it never rescues a blowout.
• Sharpe gates harder. Intraday Sharpe needs a full session’s worth of points and at least 5 distinct UTC days; tracking error is scaled to daily-equivalent bps so legs stay comparable.
• Data policy (important). Crypto 1-minute data refreshes automatically and free (Binance CDN daily ZIPs, published ~08:00 UTC for the prior day — so an intraday crypto leg honestly compares through yesterday’s close; data_through shows it). Equity intraday data is billed (Databento) and is never pulled by a background job — the card shows a warning and you ingest it deliberately from the Data tab’s cost-confirmed flow.
• The intraday overlay downsamples to ~1500 points for display and caps the trade table at 500 rows (flagged when cut); metrics are computed on the full series first.

scripts/smoke-intraday.sh runs the free end-to-end check on the crypto sandbox: start a 1-min paper leg, watch per-bar snapshots land, stop, recompute, and confirm the per-bar overlay. It refuses to run while another crypto session (e.g. a live campaign leg) is up.

Limits

• Snapshots begin at the first instrumented session — history before that cannot be reconstructed.
• The shadow runs on catalog bars at the leg’s own cadence; data_through in the card shows the honest coverage if a vendor refresh failed or lagged.
• Mixed-cadence books (a daily sleeve plus an intraday one) fall back to the daily comparison with a warning — homogeneous cadence per leg is the current contract.
• The cached payload is a point-in-time computation (computed_at), not a live stream.