Тема
Liveness Audit — ADR-001..017 + FSD strategies (2026-05-06)
Status: Complete (analysis only — no code changes). Trigger: P1 incident incident-2026-05-06-rtds.md §6.3 — RTDS ADR-004 silently broken for 4+ days; 1561 tests stayed green. The principle in memory feedback_silent_feature_broken (every claimed feature must have a liveness test observing an end-to-end boundary effect) had not been audited against the rest of the ADR portfolio.
Methodology + scope: 18 ADRs (docs/adr/) + 5 FSD strategy specs (docs/strategies/). For each, identify (a) the headline feature claim from the Status: IMPLEMENTED/ACCEPTED block, (b) the observable boundary at which a runtime failure would manifest, (c) whether at least one automated test in crates/pm-bot/tests/ or in-crate #[cfg(test)] modules observes that boundary at runtime, vs only asserting on a pure-function return shape.
1. Methodology — what counts as "liveness"
We discovered during the RTDS post-mortem that the project carries a healthy unit-test surface for pure evaluators (evaluate_signal, evaluate_rebate_pair, should_cancel_basket_on_drift) but a thinner one for the wiring between those evaluators and the I/O boundary. The RTDS parser had 11 unit tests; not one drove RtdsClient::process_messages end-to-end with a wire payload. Hence the 4-day silent failure.
This audit uses the following classification:
| Class | Definition | Example | Sufficient? |
|---|---|---|---|
| Spec test | Asserts on a pure function's return value over a constructed input. No I/O, no executor, no journal, no side-channel observed. | t_evaluate_returns_some_signal_when_prob_in_range | NO (alone) |
| Liveness test | Drives input through the production wiring up to a side-effect boundary (OrderClient::place_order, journal row, SSE envelope, log line, metric counter), and asserts on what was observed at that boundary. | engine_handle_full_fill_with_autosell_places_sell (asserts mock.place_calls.lock().unwrap() length+content) | YES |
| Compile-only | event_hub.rs / local_api.rs — #[tokio::test] bodies exercise types but assertions are conditional or guarded; no observable boundary verified at runtime in CI. | executor_with_null_hub_zero_overhead_smoke | NO |
| Static fixture replay | E2E replay against frozen Parquet — drives full pipeline, asserts on Decision-stream SHA. Strong liveness for wiring; weak for external-format drift (Polymarket changes wire schema). | t_bucket_arb_full_cyyz_3day_replay | YES for wiring, NO for external-contract drift |
The silent-broken risk is the union of:
- Wiring path that has no liveness coverage at all (RTDS-class)
- Wiring path covered only by static replay (drifts when Polymarket changes the wire and replay still passes against stale capture)
- Wiring path gated by an env flag whose default-off branch is the only one tested
A liveness test must observe a boundary effect that a cargo test run can fail on without external network. The five admissible boundaries used below:
MockOrderClient.place_calls / cancel_calls / merge_callsrecords (Decision → Executor wiring)- SQLite journal row count via
read_conn.query_row(BotEvent → JournalWriter wiring) - SSE/REST response body via
axum_test_helperorreqwestagainstlocal_api::serve(BotEvent → SSE wiring) - Tracing log line capture via
tracing_subscriber::fmt::TestWriter(only for warn/error escalations likepaired_imbalance_escalation) Metrics::snapshot()/MetricsTickprojection counter delta (counter-bumping wire)
2. ADR-by-ADR analysis
ADR-001 / strategy-isolation-design.md — Strategy Isolation Refactor (Phase 4)
| Field | Value |
|---|---|
| Feature claim | New Strategy pipeline (strategies/{bucket_arb,directional,maker,maker_rebate}.rs + dispatcher.rs + executor.rs + engine.rs) is sole executor; legacy OrderManager.discover_and_place only kept for MakerRebate path; bit-identical hot loop. |
| Observable boundary | Decision::PlaceOrder / Decision::CancelOrder reach OrderClient::place_limit_order / cancel_order, with strategy-tagged client_id. |
| Liveness test exists? | YES — tests/strategy_engine_integration.rs (6 #[tokio::test] cases) drive StrategyEngine over MockOrderClient and assert place_calls/cancel_calls content. engine_handle_full_fill_with_autosell_places_sell asserts side, price, size, order_type. |
| Gap | None for this ADR. |
| Risk | LOW |
ADR-002 / phase-5-scanner-fsm-extraction.md — Scanner thinning + bucket-arb FSM extraction
| Field | Value |
|---|---|
| Feature claim | scanner.rs becomes thin generic event router; bucket-arb FSM lives at strategies/bucket_arb/{fsm,driver,scanner_hooks}; LegacyStrategy trait + strategies/legacy/ deleted; ScannerHooks trait dispatch. |
| Observable boundary | (a) ScannerHooks::on_ws_book -> Decision::PlaceOrder reach OrderClient for bucket-arb; (b) on_new_market registers candidate -> FSM transitions Candidate → Tracking. |
| Liveness test exists? | PARTIAL — tests/scanner_dispatch_test.rs::t_external_signal_reaches_hooks proves webhook path reaches a hook; tests/event_fsm_transitions.rs covers FSM but as pure unit. tests/e2e_bucket_arb_replay.rs::t_bucket_arb_full_cyyz_3day_replay drives the entire pipeline against frozen-v1 Parquet. No test fires ScannerHooks::on_ws_book and asserts a Decision reaches the executor end-to-end — replay test asserts on Decision SHA, not on per-hook counter. |
| Gap | Add a per-hook liveness test: feed a synthetic WsEvent::Book into the application scanner with a BucketArbScannerHooks registered, assert at least one Decision::PlaceOrder reaches the bound MockOrderClient. |
| Risk | MEDIUM — ScannerHooks pipeline silent failure would surface only via integration replay or production miss. |
ADR-003 / webhook-signal-source.md — External Webhook Signal Source
| Field | Value |
|---|---|
| Feature claim | Opt-in HMAC-signed POST /v1/signals (axum on 127.0.0.1:31338); accepted signal flows into ScannerHooks::on_external_signal → Vec<Decision> via the same Phase 4 pipeline. |
| Observable boundary | (a) HTTP 202 Accepted, (b) HMAC verification, (c) signal reaches the in-crate hook (capture via mpsc receiver), (d) metrics counters bump. |
| Liveness test exists? | YES — tests/e2e_webhook_smoke.rs::t_e2e_signal_dry_run_smoke boots a real axum server, posts an HMAC-signed payload, awaits the deserialized ExternalSignal on the receiver, asserts metrics counters. Also tests/scanner_dispatch_test.rs::t_external_signal_reaches_hooks and t_full_channel_maps_to_503. |
| Gap | One indirect gap: no test asserts that an accepted signal that lands in on_external_signal produces a Decision reaching OrderClient (the path stops at the receiver). When the strategy hook silently returns Vec::new() (e.g. min_probability filter mistuned), the signal is observably "received" but not "actioned". |
| Risk | LOW for receive-and-decode wiring; MEDIUM for receive-to-Decision wiring. |
ADR-004 / rtds-integration.md — RTDS toxic-flow stream
| Field | Value |
|---|---|
| Feature claim | Always-on wss://ws-live-data.polymarket.com subscription; events → ScannerHooks::on_rtds_trade / on_rtds_orders_matched → strategies' toxic-flow detector → optional Decision::CancelOrder. |
| Observable boundary | (a) RTDS-events parser yields RtdsEvent::Trade from a wire payload, (b) ScannerHooks::on_rtds_* is invoked by the application scanner, (c) on toxic-flow active mode, Decision::CancelOrder reaches OrderClient, (d) rtds_events_received_* counters bump in MetricsTick. |
| Liveness test exists? | PARTIAL/BROKEN — Spec tests in io/rtds/parse.rs (11 fixtures), subscribe.rs (10 builder/parser cases), metrics.rs (8 snapshot cases). tests/metrics_loop.rs::metrics_loop_projects_rtds_counters_when_wired confirms counter projection but uses manually seeded counters. No test feeds a wire payload through RtdsClient::process_messages and asserts the parsed event reaches ScannerHooks::on_rtds_trade. The Q3 timestamp invariant (the very thing that broke) was a docstring-only safeguard. The active-cancel branch (RTDS_TOXIC_FLOW_ACTIVE_CANCEL_ENABLED=true) has zero coverage. |
| Gap | Three liveness tests required (see §4 / TD-31, TD-32, TD-33 below). |
| Risk | HIGH — root cause of the incident; parser-to-hook-to-Decision path entirely uncovered. |
ADR-005 / pm-bot-dashboard-integration.md — Local-Bot Mode (SSE+REST)
| Field | Value |
|---|---|
| Feature claim | Opt-in BOT_LOCAL_API_ENABLED=true boots axum SSE+REST on 127.0.0.1:31339; BotEvent reaches the wire as EventEnvelope JSON; Last-Event-ID replay supported; rotate flow protected by ROTATE token. |
| Observable boundary | (a) SSE GET /api/local/stream emits the next-emitted BotEvent as JSON; (b) GET /seed returns last-seq snapshot; (c) POST /rotate requires confirm token; (d) Journal row written atomically before broadcast. |
| Liveness test exists? | YES for seed, health, bind-policy, rotate-confirm (tests/local_api.rs 5 tests). YES for journal commit-then-broadcast contract (tests/journal_writer.rs::single_event_atomic_commit, ..._idempotent, ..._under_load — 9 tests asserting both row exists AND broadcast envelope arrives). PARTIAL for SSE — stream_replays_from_last_event_id checks status only with comment "frame assertion would require..."; stream_emits_heartbeat is similarly compile-only on the body shape. |
| Gap | SSE end-to-end frame assertion: spawn server, connect a real reqwest::get SSE client, emit a BotEvent::OrderPlaced via JournalWriter, assert the frame's data: JSON deserializes back to the same envelope. |
| Risk | MEDIUM — SSE wire-frame schema drift would silently strand the dashboard. Journal layer well-covered. |
ADR-006 / adr-006-archive-extraction.md — Archive 7 dormant crates
| Field | Value |
|---|---|
| Feature claim | Workspace reduced to 7 CORE; archived crates moved to sibling repo. |
| Observable boundary | cargo metadata lists 7 members; grep -r pm_orchestrator crates/ returns 0. |
| Liveness test exists? | N/A — pure workspace structural change, not a runtime feature. |
| Gap | None. |
| Risk | LOW — compile-time enforced. |
ADR-007 / adr-007-pm-bot-modular-restructure.md — 4-layer KISS layout
| Field | Value |
|---|---|
| Feature claim | domain//application//io//persistence/ directories enforced; no DI traits beyond existing. |
| Observable boundary | Module path enforcement; cargo check for misplaced imports. |
| Liveness test exists? | N/A — pure layout invariant. The layering rule "domain ⊥ io" is not asserted by a test (e.g. via cargo modules lint). |
| Gap | Optional: a cargo modules-based layering test (treat as Tier-3 polish). |
| Risk | LOW — compile-time enforced for naming; semantic layering depends on reviewer discipline. |
ADR-008 / adr-008-config-split-per-strategy.md — config/ folder split
| Field | Value |
|---|---|
| Feature claim | Pure structural file split; BotConfig API preserved bit-identical. |
| Observable boundary | All 898+ existing tests pass after split; cargo test -p pm-bot --lib baseline. |
| Liveness test exists? | YES — the 898-test baseline covers behavioural parity; webhook.rs + rtds.rs co-locate 13 unit tests with their config struct. |
| Gap | None. |
| Risk | LOW. |
ADR-009 / adr-009-capture-replay-architecture.md — Capture & Replay
| Field | Value |
|---|---|
| Feature claim | Capture taps in io/ write JSONL.zstd hot stream (when --capture set); pm-analytics replay/sweep/whatif/compact operate on Parquet+ZSTD; deterministic replay (10× identical SHA-256). |
| Observable boundary | (a) Capture writer flushes valid JSONL.zstd round-trip; (b) Parquet schema parity; (c) ReplayEngine emits identical Decision SHA across 10 runs; (d) frozen-v1 Parquet fixtures replay the 9 cyyz E2E tests. |
| Liveness test exists? | YES, well-covered — tests/capture_smoke.rs (round-trip + Parquet schema), tests/replay_determinism.rs + replay_determinism_smoke.rs (10× identical SHA-256), tests/capture_replay_parity.rs (acceptance gate), 9 cyyz replays in tests/e2e_*_replay.rs. CI .github/workflows/parity.yml runs the gate per PR. |
| Gap | One drift risk: capture writes happen on --capture only — when the path is disabled (production), the NullSink branch has only a #[cfg(test)] no-panic check; no test asserts the production hot loop emits 0 capture writes. Bit-identical contract is asserted by replay parity, indirectly. |
| Risk | LOW — strong direct coverage. |
ADR-010 / adr-010-order-client-trait-unified.md — Unified OrderClient trait in pm-sdk
| Field | Value |
|---|---|
| Feature claim | Trait moved to pm-sdk/src/clob/order_client_trait.rs, expanded to 8 methods; SdkOrderClient/DryRunOrderClient/MockOrderClient re-impl through new trait. |
| Observable boundary | The 8 methods are reachable through Arc<dyn OrderClient> from application::engine and tests/e2e_*_replay.rs. |
| Liveness test exists? | YES — tests/strategy_engine_integration.rs exercises place_limit_order and cancel_order. The 6 newer methods (queries) are exercised by E2E replay through MockOrderClient. |
| Gap | The "wiring-first, no premature deletion" deferred 6 methods to unimplemented!-style defaults on test adapters. No test asserts which method is actually wired today — a method renamed in the trait but stubbed in adapters compiles and runs without surfacing the gap until live. |
| Risk | LOW (5 of 8 methods drive replay). MEDIUM if a query method is added later without test alignment. |
ADR-011 / adr-011-paired-maker-rebate-emission.md — Paired YES+NO maker-rebate + on-chain merge
| Field | Value |
|---|---|
| Feature claim | MAKER_REBATE_PAIRED_ENABLED=true drives RebatePair state machine Pending → OneFilled → BothFilled → Merged; BothFilled emits Decision::CallMergePositions; imbalance timeout escalates to ManualReconcile with paired_imbalance_escalation warn. |
| Observable boundary | (a) MakerRebateStrategy::evaluate_periodic returns paired [YES_BUY, NO_BUY] Decisions when paired_enabled=true; (b) on_full_fill advances state; (c) Decision::CallMergePositions emitted on BothFilled; (d) ManualReconcile warn line on timeout. |
| Liveness test exists? | YES — 16 integration-shape tests against MockRepository (per backlog X5.3 entry). 17 unit tests on the pure evaluate_rebate_pair. The paired_enabled=false default-off path is bit-identical-asserted via existing single-side tests. |
| Gap | The Decision::CallMergePositions reaching OrderClient::merge_positions is asserted only at unit level (merges.len() == 1 via `decisions.iter().filter( |
| Risk | MEDIUM — production-flag-gated; default-off; flip-day risk on first activation. |
ADR-012 / adr-012-pm-dashboard-redesign.md — pm-dashboard Vue 3 SPA + axum backend
| Field | Value |
|---|---|
| Feature claim | Split into crates/pm-dashboard/ (axum REST+SSE proxy) + frontend/pm-dashboard-web/ (Vue 3 SPA, separate pnpm workspace); 3 routes; PrimeVue Aura dark; SSE-only data plane; cancel via REST. |
| Observable boundary | Backend: GET /api/local/seed, GET /api/local/stream, POST /api/local/orders/:id/cancel. Frontend: store reactivity to incoming envelope. |
| Liveness test exists? | Backend: 28 pm-dashboard integration tests (per backlog X2.4 entry) — but crates/pm-dashboard/tests/ does not exist on disk; tests likely live in-crate #[cfg(test)] modules. Frontend: no automated test surface in frontend/pm-dashboard-web/__tests__/ is referenced from this audit. |
| Gap | (a) Frontend has no asserted runtime boundary in CI — the lesson from the RCA §5.5 (truthful-indicator policy) is that a unit test asserting rtdsStatus(metricTape) returns 'success' for a known-bad tape would have caught the false-green. No such tests exist. (b) The proxy-only nature of /api/local/stream is not asserted by an integration test that runs the axum proxy + a fake pm-bot upstream + verifies bytes pass through. |
| Risk | MEDIUM — Vue layer ships untested, dashboard severity logic uniquely vulnerable to the silent-broken pattern (RCA §4.3). |
ADR-013 / adr-013-dashboard-cancel-ui-invariants.md — Cancel UI hardening
| Field | Value |
|---|---|
| Feature claim | Bulk cancel + per-market cancel + paranoid CANCEL token gate at threshold 5 + 5s soft-delete undo + cancel-reason taxonomy. |
| Observable boundary | Frontend: store transitions, dialog confirmation, undo timer, REST POST count. Backend: POST /api/local/orders/:id/cancel reaches pm-bot OrderClient::cancel_order once per row in cohort. |
| Liveness test exists? | Per backlog claim "12 pm-dashboard tests"; not yet inspected from disk. Frontend store/composable tests claimed but unverified. |
| Gap | (a) Backend: integration test posting a 6-row bulk cancel without CANCEL token must 4xx. (b) Frontend: store reducer test for the 5s undo branch (commit cancellation only after window). |
| Risk | MEDIUM — cancel volume blast radius is operator-protective; silent-broken paranoid gate would let an accidental "cancel all" through. |
ADR-014 / adr-014-dashboard-risk-metrics.md — Risk metric annualisation
| Field | Value |
|---|---|
| Feature claim | Annualisation = √(31 536 000 / median_dt); risk-free default 0.0; per-strategy slicing; PrimeVue badges + EquityCurve sparkline + DrawdownChart. |
| Observable boundary | Frontend: metricsByStrategy projection over fills ring; badge severity. |
| Liveness test exists? | Unverified from disk in this audit; backlog claims it. |
| Gap | If absent: a Vitest-style assertion that feeds a known-shape fills array into risk.ts and asserts the Sharpe/Sortino/MaxDD numerical output (Maths invariant, not severity). The RCA §5.5 lesson applies: a "freshness-derived severity" badge for the risk panel needs a known-bad input test. |
| Risk | LOW for math (frontend-only, no order-routing impact). MEDIUM for the dashboard-truthfulness pattern recurrence. |
ADR-015 / adr-015-multi-wallet-and-strategy-config.md — Encrypted wallets.db + plaintext config.db
| Field | Value |
|---|---|
| Feature claim | Two SQLite stores; BOT_WALLET_LABEL / BOT_STRATEGY_LABEL opt-in; INV-3 cross-check (strategy.wallet_label == BOT_WALLET_LABEL); env-fallback bit-identical when unset; FE-5 budget predicate at provisioning time. |
| Observable boundary | (a) pm-bot startup with selectors loads correct keys; (b) INV-3 violation aborts startup; (c) pm-cli wallet/strategy subcommands round-trip; (d) WebUI configurator REST. |
| Liveness test exists? | YES — backlog claims 27 pm-cli integration tests + 28 pm-dashboard integration tests + +57 wallets unit tests. INV-3 cross-check at main.rs:210-228 — but no test in crates/pm-bot/tests/ named for that invariant; it is a startup panic. |
| Gap | The fail-fast INV-3 panic is not asserted by a test — startup failure is the only signal in CI that the invariant fires. A #[should_panic] test on validate_required_env_or_panic (or whatever the invariant evaluator is) would catch a regression where the INV-3 branch is silently elided. |
| Risk | MEDIUM — if the cross-check is silently disabled, two strategies share a wallet's budget bookkeeping incorrectly; financial-system risk. |
ADR-016 / adr-016-timezone-handling-policy.md — Timezone handling
| Field | Value |
|---|---|
| Feature claim | TimezoneCache activation predicate = resolution_source contains wunderground.com; disk-only cache; raw UTC fallback when ICAO miss. |
| Observable boundary | compute_deadline returns local-day midnight for known ICAO; UTC end_date for unknown. |
| Liveness test exists? | YES — 13 unit tests in crates/pm-bot/src/timezone/mod.rs cover extract_icao, compute_deadline, cache load/save. |
| Gap | The activation predicate is not asserted at the strategy boundary — no test confirms that directional::evaluate_signal invokes tz_cache.compute_deadline and respects the result for late-entry-gate decisions. The directional or(end) chain at directional.rs:1050 could regress to always-fallback without any test failing. |
| Risk | LOW for cache mechanics, MEDIUM for the strategy-side activation predicate (silently falling back to UTC = misclassifies "today vs tomorrow" markets per memory feedback_max_hours_expiry). |
ADR-017 — REMOVED 2026-05-20
ADR-017 (EV cap для directional WatchArmFak) удалён полностью; раздел аудита более не релевантен. См. историческую запись в архиве openspec / git history. Причина: Gamma.outcomePrices[i] ≡ mid делает формулу EV cap тождественной константному ценовому потолку — независимой защиты не было.
merge-arb-strategy.md — Merge-Arb Strategy v2
| Field | Value |
|---|---|
| Feature claim | New MergeArbStrategy with FSM `Candidate → BothPosted → OneFilled → BothFilled → MergeSubmitted → {Confirmed |
| Observable boundary | (a) Decision::CallMergePositions emitted on BothFilled; (b) executor wires it to on-chain pm-sdk::onchain::ctf::merge_positions or neg_risk_adapter::merge_positions; (c) on-chain receipt → MergeReceiptRequest → receipt_watcher → confirm/revert/timeout BotEvent. |
| Liveness test exists? | YES — merge_arb.rs::t_band_exit_emits_cancel_merge_reconcile (line 1536) asserts exactly one CallMergePositions Decision in band-exit branch. 17 in-crate t_* tests. tests/receipt_watcher.rs covers Decision-emit-on-receipt for confirmed/reverted/timeout. |
| Gap | The Decision → executor → on-chain wiring (Decision::CallMergePositions reaches pm-sdk merge_positions call) is asserted only by collected decisions count, not by a recorded "merge call sent" boundary. The neg_risk: true branch (INegRiskAdapter) has no specific test ensuring the neg-risk adapter path is invoked vs the standard CTF path — the variant tag is inspected, but executor call-site dispatch is not. |
| Risk | MEDIUM — financial-system path; live merge of $X funds the merge call must hit the right contract, neg-risk inversion bug history (memory project_neg_risk_audit). |
bucket-arbitrage.md — Bucket arbitrage strategy FSD
| Field | Value |
|---|---|
| Feature claim | INV-5/6/7/20/21/22/23/25/26/27 enforced at FSM and executor layer; basket FSM Candidate → Tracking → Participating; holds-to-resolution; pre-flight atomic. |
| Observable boundary | (a) FSM transitions match snapshot; (b) Decision::PlaceOrder for parallel basket placement; (c) Decision::CancelOrder on drift / timeout / partial reject; (d) basket ledger rows in journal. |
| Liveness test exists? | YES — extensive: tests/bucket_arb_no_exit_sell.rs, tests/parallel_placement.rs, tests/basket_covered_on_full_fill.rs, tests/dirty_basket_retry.rs, tests/abandon_cancels_open_orders.rs, tests/manual_cancel_attempt.rs, tests/preflight_best_bid_ask.rs, tests/expiry_sync_attempt.rs, tests/restart_attempt_recovery.rs, tests/crash_recovery.rs, tests/partial_fill_history.rs, tests/event_fsm_transitions.rs, tests/e2e_bucket_arb_replay.rs. |
| Gap | None significant. |
| Risk | LOW. |
maker-rebate.md — Maker-rebate FSD
Covered by ADR-011 above + tests/e2e_maker_rebate_replay.rs. Risk: MEDIUM carried from ADR-011.
directional.md — Directional strategy FSD
Covered by tests/e2e_directional_replay.rs (4 profile cases) + 100+ in-crate tests at domain/strategies/directional.rs. The entry_flow=watch_arm_fak Phase 1 has 14+ in-crate tests; the recovery path has tests/watch_arm_fak_recovery.rs. Risk: LOW for live evaluation; MEDIUM for the periodic-scan path (evaluate_periodic invocation cadence is governed by BOT_FALLBACK_SCAN_SECS and has no liveness test asserting "scan fires every N seconds in the running loop"; the loop is not driven in CI).
maker.md — Maker strategy FSD
Covered by tests/e2e_maker_replay.rs (3 profile cases) + 30+ in-crate tests. Risk: LOW.
merge-arb.md — Merge-arb strategy FSD
Covered by ADR merge-arb-strategy.md above. Risk: MEDIUM.
3. Risk-ordered gap list
High-risk gaps (silent-broken precedent in RCA)
- ADR-004 RTDS parser → hook wiring — no liveness test feeds a wire payload through the full
RtdsClient::process_messages→RtdsEvent::Trade→ScannerHooks::on_rtds_tradechain. RCA §5.3. - ADR-004 RTDS active-cancel branch —
RTDS_TOXIC_FLOW_ACTIVE_CANCEL_ENABLED=truepath emitsDecision::CancelOrderfromon_rtds_trade; coverage 0%. - ADR-004 RTDS connection-state freshness — RCA §4.2 watchdog + §4.3 dashboard truthfulness; both are live-only, no automated assertion.
Medium-risk gaps
- ADR-002 ScannerHooks per-hook E2E — a synthetic WS event end-to-end through
BucketArbScannerHookstoMockOrderClientwould catch regressions that frozen-v1 replay misses. - ADR-005 SSE wire-frame schema — backend test boots the proxy, decodes one
EventEnvelopefrom the SSE byte stream. - ADR-011 paired maker-rebate executor wiring —
Decision::CallMergePositionsfrom the paired flow reaches a recording executor /merge_calls. - ADR-011
paired_imbalance_escalationwarn-log capture —tracing_subscriber::fmt::TestWriterassertion. - ADR-012 dashboard truthfulness pattern — frontend Vitest tests asserting status-function output for known-bad metric tapes (recurrence guard for RCA §4.3).
- ADR-013 paranoid bulk-cancel gate — backend test rejects a 6-row cancel cohort missing
CANCELtoken. - ADR-015 INV-3 cross-check assertion —
#[should_panic]test for the "wallet_label mismatch" branch. - ADR-016 directional uses TZ at evaluate-signal — assert
directional::evaluate_signalconsultstz_cache.compute_deadlinefor awunderground.comsource and rejects late-entry beyond local-midnight. - merge-arb neg-risk vs standard CTF dispatch — assert
neg_risk: trueDecision lands onNegRiskAdapter.merge_positions, notConditionalTokens.merge_positions(financial-system inversion risk per memoryproject_neg_risk_audit).
Low-risk gaps
- ADR-007 layering lint — optional Tier-3 polish.
- ADR-009 NullSink no-op proof in production hot loop — replay parity already infers this indirectly.
- ADR-010 trait method coverage matrix — guard for future trait additions.
- directional.md periodic-scan cadence —
evaluate_periodicfires-every-N-secs assertion.
4. Recommendations — High-risk gap remediation (test designs)
For each High-risk gap, the test design enumerated below is the minimum that would have caught the documented incident class. Implementation is queued in the tech-debt-backlog (see TD-31..TD-36 entries added in §5).
Recommendation A — ADR-004 RTDS parser → hook end-to-end (TD-31)
Test name (proposed): tests/rtds_hook_e2e.rs::t_wire_payload_drives_on_rtds_trade.
Inputs: a fixture JSON payload matching the RTDS activity/trades schema with timestamp in both ms and seconds magnitude (parametrised); a synthetic BucketArbScannerHooks recorder that captures invocations.
Observable boundary: after one RTDS message is fed through RtdsClient::process_messages (or an in-process loopback WS server), the recorder has at least one entry; the captured RtdsEvent::Trade fields equal the source payload modulo the magnitude conversion.
When to fire: every PR touching io/rtds/ or domain/strategies/scanner_hooks.rs::on_rtds_*. Plus daily CI cron run that probes the live wss://ws-live-data.polymarket.com endpoint with a known-busy slug whitelist; assertion that trades_parsed >= 1 within 30s. (RCA §6.2 — already required for P0 closure.)
Recommendation B — ADR-004 RTDS active-cancel branch (TD-32)
Test name: tests/rtds_active_cancel.rs::t_toxic_flow_emits_cancel_when_active.
Inputs: RTDS_TOXIC_FLOW_LOG_ONLY=false + RTDS_TOXIC_FLOW_ACTIVE_CANCEL_ENABLED=true env in the test's BotConfig; sliding-window seeded with N synthetic large trades exceeding the toxic threshold; one live ManagedOrder for the affected token.
Observable boundary: mock.cancel_calls.lock().unwrap().len() == 1 for that order_id, with reason matching the toxic-flow taxonomy.
When to fire: every PR touching the toxic-flow window code or the env knobs.
Recommendation C — ADR-004 RTDS health metric → AlertRule (TD-33)
Test name: tests/alerts_rtds_health.rs::t_rtds_health_alert_fires_on_stale_dropped.
Inputs: a constructed MetricsTick series with rtds_events_dropped_stale ≥ 1 over a 5-minute synthetic window.
Observable boundary: a new RtdsHealthRule: AlertRule registered in application/alerts/engine.rs produces one Alert{ severity: Critical, rule: "rtds_health" } per the rule's contract.
When to fire: PRs touching application/alerts/ or MetricsTick schema. Also pairs with the dashboard-truthfulness recurrence guard (Recommendation §3 / TD-34).
5. Tickets surfaced (see tech-debt-backlog.md updates)
- TD-31 — Liveness test for RTDS parser → ScannerHooks::on_rtds_trade wiring (HIGH).
- TD-32 — Liveness test for RTDS toxic-flow active-cancel branch (HIGH).
- TD-33 — Stock
AlertRule::rtds_health+ corresponding test (HIGH). - TD-34 — Bundle of medium-risk liveness gaps: ScannerHooks E2E, SSE wire-frame, paired maker-rebate executor, paired escalation log, INV-3 panic assertion, neg-risk vs standard CTF dispatch, TZ used at evaluate_signal, dashboard truthfulness recurrence guard, paranoid bulk-cancel gate (MEDIUM).
Estimated complexity per memory feedback_no_man_hours: see backlog entries (LOC + iteration count, no human-hour bookkeeping).
6. Verdict
18 ADRs audited; 4 FSD strategy specs cross-referenced.3 High-risk gaps, all in ADR-004 (RTDS) — direct continuation of the 2026-05-06 incident. 9 Medium-risk gaps, distributed across ADR-002/005/011/012/013/015/016 + merge-arb. None individually production-blocking, but the ensemble matches the silent-broken pattern that produced the RTDS incident. 4 Low-risk gaps, optional polish.
The pattern is consistent: pure evaluators have rich unit coverage; wiring between evaluators and observable boundaries (executor calls, journal rows, SSE frames, log lines, metric counters) has thin or absent direct coverage. Frozen-v1 replay tests catch internal regressions but not external-contract drift — the failure mode that broke RTDS.
Top-3 most urgent (block before next live trading session, per RCA §6.2):
- TD-33 —
AlertRule::rtds_health(already P0 in RCA). - TD-31 — RTDS parser → hook E2E (closes the literal silent-broken hole that surfaced).
- TD-32 — RTDS active-cancel branch (zero coverage on a code path that would emit live cancels).
The remaining 9 Medium gaps should be triaged as TD-34 bundle. Implementation gating: feedback_no_autonomous_changes — no test added without explicit user sign-off on the specific list.