Scanner Service Production Validation: 9.4M Quotes, 106-Hour Continuous Run, and Multi-DEX Arbitrage Signal Detection

TL;DR

• Production Grade Confirmed: Scanner service achieves a 9.8/10 overall score over a 5-day, 8.26M-quote continuous session — zero errors, zero reconnections, zero stale quotes
• Multi-DEX Arbitrage Detection Working: 4,216 structured arb signals detected from Orca Whirlpool concentrated liquidity pools (65shmpuY + 7qbRF6Ys) as SOL rallied 10% — signals from different pools and DEXes correctly surfaced for the downstream pipeline
• Zero Critical Deviations: No local quote ever breached the 10% oracle deviation threshold across the entire 9.3M-quote Epoch 2 dataset
• Per-Pair Routing Insight: SOL→USDC local routing wins 40–43% of comparisons; SOL→USDT should route exclusively to Jupiter dark pools (84–94% external wins)

Background: The Complete Quality Assessment Journey

Over 52 days (January 19 to March 12, 2026), the Solana Trading System’s quote infrastructure has been subjected to rigorous continuous testing — starting from an 18-second burst to a 106.4-hour multi-day soak. The Final Consolidated Quality Assessment covers 11 test sessions, ~9,456,069 quotes total, and represents the most comprehensive validation of the quote + scanner pipeline to date.

This post summarises the full journey, focuses on the landmark 5-day extended session, and explains what the results mean for multi-DEX arbitrage detection going into the strategy execution phase.

Two Epochs of Architecture

Testing revealed a clear architectural divide, which shapes how we interpret all results:

┌─────────────────────────────────────────────────────────────────┐
│  EPOCH 1 — Confidence-Weighted Scoring (Jan 19 – Jan 24)        │
│                                                                  │
│  • Single gRPC stream with embedded comparison                   │
│  • Oracle factor (0.6–1.0) applied to confidence scores          │
│  • Route complexity penalised (1-hop preferred)                  │
│  • Local wins heavily biased by scoring model (81–100%)          │
│  • Sessions: 644–1,065 quotes, 19–45 seconds each               │
└─────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────┐
│  EPOCH 2 — Dual Direct-Connection Streaming (Feb 27 – present)  │
│                                                                  │
│  • local-quote-service  :50052  ─── Orca, Meteora, Raydium      │
│  • external-quote-service :50053 ─ Jupiter (20+ DEX protocols)  │
│  • Comparison: raw output amounts only — no bias                 │
│  • Winner = true best price                                      │
│  • Sessions: 1,001 to 8,256,940 quotes, 71s to 106.4 hours      │
└─────────────────────────────────────────────────────────────────┘

All production readiness decisions are based on Epoch 2 data. Epoch 1 data is preserved for historical context only.

Session Timeline: From 19 Seconds to 106 Hours

The progression from seconds to days validates that the system is not just correct at startup — it remains correct under sustained load across real market conditions including a 10% SOL price rally.

The 5-Day Extended Session: Definitive Production Proof

The March 8–12 session is the centrepiece of the assessment.

Session:     March 8–12, 2026
Duration:    106.4 hours (6,384 minutes) — uninterrupted
Quotes:      8,256,940
Files:       8,257 JSONL log files
Errors:      0
Reconnects:  0
Stale quotes: 0
SOL price:   $80.51 – $88.60 (avg ~$85.0, +10% range captured)

Infrastructure Health

No errors. No reconnections. No stale data. 8.26 million quotes delivered cleanly over five days.

Oracle Coverage: TASK-001 Resolved and Sustained

The oracle oracle_price_usd field was completely broken at project start (0% coverage on January 19). It was fixed prior to the February 27 session and has held 100% across every subsequent quote:

Epoch 2 oracle coverage:
  Feb 27:     1,001 / 1,001    → 100%  ✅
  Mar 2:     22,672 / 22,672   → 100%  ✅
  Mar 3:      8,417 / 8,417    → 100%  ✅
  Mar 7 AM:  42,148 / 42,148   → 100%  ✅
  Mar 7–8: 1,120,084 / 1,120,084 → 100% ✅
  Mar 8–12: 8,256,940 / 8,256,940 → 100% ✅
  ─────────────────────────────────────────
  Total Epoch 2: 9,313,022 / 9,313,022 → 100%

The fix is confirmed stable at multi-day scale. Deviation gating is fully operational.

Deviation Policy: Reclassifying Arb Signals

A key finding from the 5-day session was that the original >5% = critical threshold was too conservative. With the downstream pool-filtering strategy pipeline in place, deviations in the 5–10% range represent real, exploitable price dislocations — not service defects.

Revised Threshold Policy

The service correctly surfaces 5–10% deviations. The downstream pipeline decides whether to act on them.

Multi-DEX Arbitrage Signal Detection

How the Scanner Detects Arb Signals

The scanner service streams from both local and external services simultaneously, pairs forward (A→B) and reverse (B→A) quotes, computes the round-trip profit, and classifies deviations against oracle prices. When a local quote diverges 5–10% from oracle, the pool has moved outside its concentrated liquidity tick range — a dislocation that can be exploited by routing through the alternative source.

Signals Detected in the 5-Day Session

Both primary pools are Orca Whirlpool concentrated liquidity positions whose active tick ranges were exited as SOL rallied. The deviation onset correlates tightly with specific price levels:

SOL avg price:      $82.37 → $84.01 → $86.57 → $86.14 → $85.98
Arb signals/day:       1   →   567  →  1,047  →  1,561  → 1,041*

(* partial day)

The signals are predictable and structured — they turn on as SOL crosses a threshold and persist for hours. This is exactly the kind of opportunity the downstream strategy pipeline is built to exploit.

Aquifer Protocol Bug (External, Isolated)

One external protocol — Aquifer pool CNC5TaeNQEoSPfQKZ7GgfM4R8WYAJRKRSHFCHkf2H7ko — produces 100k–241k% deviations at ~1.1 events/hour. This is a protocol-level calculation bug, not a scanner defect. The comparison system isolates it correctly. Frequency is declining (45 events on March 8 → 6 on partial March 12). Explicit pool exclusion is recommended in external quote routing config.

Protocol Coverage: 8 Local + 20+ External

The 5-day session shows the most balanced local protocol mix to date:

Local Protocols (8 active)

raydium_amm has grown from near-zero in early sessions to dominate local routing (25%). The emergence of raydium_clmm (16 quotes) hints at future CLMM routing capability.

External Protocols (20+, via Jupiter)

The external service aggregates across 20+ protocols simultaneously. The top contributors:

HumidiFi         →  35.34%  (dark pool)
GoonFi V2        →  24.80%  (dark pool)
SolFi V2         →   9.7%
BisonFi          →   8.6%
TesseraV         →   4.3%
PancakeSwap, 1DEX, WhaleStreet, Byreal, Manifest,
DefiTuna, Meteora DAMM v2, Raydium CLMM, ...

Dark pool concentration (HumidiFi + GoonFi) is 60.1% — down from 67.7% as newer protocols grow. Well below the 80% concentration alert threshold.

Competitive Analysis: Local vs External

Win Rate Trend (Epoch 2)

The 26.7% local win rate from the 5-day session is the first statistically robust long-run baseline. Daily win rate improved across the session (21.9% → 32.2% → 30.7%) as the SOL rally rebalanced local AMM pools.

Win Rate by Token Pair

Critical routing insight confirmed at 5-day scale: SOL→USDC and SOL→USDT have structurally different dynamics. The distinction is not random variance — it held across every sustained session.

Routing Strategy Recommendations

Based on 9.4M quotes, the scanner service produces the following routing policy for the strategy layer:

┌──────────────────────────────────────────────────────────────┐
│  SOL → USDC (all amounts)                                     │
│    Primary:  Local quote service (40–43% win at 5-day scale) │
│    Fallback: External Jupiter when local unavailable          │
│    Note:     0.1 SOL has highest local win rate (43%)         │
├──────────────────────────────────────────────────────────────┤
│  SOL → USDT (all amounts)                                     │
│    Primary:  External quote service (84–94% external wins)   │
│    Local:    Price floor check only; skip if external avail.  │
│    Note:     10 SOL has worst local rate (6.4%)              │
├──────────────────────────────────────────────────────────────┤
│  Arb Signal Handling (5–10% deviation)                        │
│    Action:   Forward to downstream arbitrage pipeline         │
│    Source:   Concentrated liquidity pools at tick boundaries  │
│    Pattern:  Onset correlates with SOL price moves >3%        │
│    Duration: Signals persist ~19 hours post tick-range exit   │
└──────────────────────────────────────────────────────────────┘

Quality Scorecard: Final

Final Grade: 9.8 / 10 ⭐ — PRODUCTION READY

What Has Changed Since Day 1

The improvement from the January 19 baseline to today quantifies 52 days of infrastructure hardening:

Architecture Diagram

                     ┌─────────────────────────────────┐
                     │       Scanner Service            │
                     │                                  │
  local-quote-service│ ─────► Quote Comparator ──────►  │─── Arb Opportunity
    :50052           │         ▲           ▲             │    (FlatBuffers)
  (Orca, Raydium,    │         │           │             │
   Meteora, Pump)    │    Oracle Dev    Forward/Reverse  │
                     │   Classification   Pair Match     │
  external-quote-svc │ ─────► Quote Comparator           │
    :50053           │                                   │
  (Jupiter: 20+      │  ┌─────────────────────────────┐  │
   DEX protocols)    │  │  Deviation Thresholds:       │  │
                     │  │  0–5%   → normal             │  │
                     │  │  5–10%  → ARB SIGNAL ──────►  │──► Downstream
                     │  │  >10%   → CRITICAL           │  │   Strategy
                     │  └─────────────────────────────┘  │   Pipeline
                     └─────────────────────────────────┘

Remaining Open Items

Impact and Next Steps

What We’ve Confirmed

The scanner service and quote pipeline are operating at production quality across all measurable dimensions:

• Arbitrage detection is working: 99.918% forward/reverse match rate over 8.26M quotes means almost every genuine opportunity is being paired and surfaced
• Multi-pool, multi-DEX signal generation: The system correctly identifies price dislocations across different Orca Whirlpool pools as SOL moves, generating structured arb signals that scale predictably with price movement
• Reliable at scale: 106.4 continuous hours with no errors, no reconnects, and no stale quotes proves the infrastructure is ready for live trading operation

What’s Next

1. Per-pair routing policy implementation — wire the SOL→USDC / SOL→USDT routing distinction into the strategy service
2. Arb signal pipeline integration — connect 5–10% deviation signals to the strategy execution layer
3. Aquifer pool exclusion — add explicit config exclusion for the Aquifer buggy pool
4. Strategy Service — define profit thresholds, slippage tolerance, execution logic
5. Executor Service — Jito bundle submission for MEV-protected trade execution
6. Paper Trading — validate the full pipeline end-to-end without real capital at risk

Conclusion

The quote and scanner pipeline has earned its production grade through sustained validation — not just a snapshot. Starting from a broken oracle and 19-second test bursts, the system now delivers 21+ QPS of oracle-validated, deviation-classified quotes across 8 local protocols and 20+ external protocols, continuously, for days at a time.

The 5-day extended session captured a real 10% SOL price rally and correctly generated 4,217 structured arbitrage signals from concentrated liquidity pools whose tick ranges were exited by the move. The scanner is doing exactly what it was built to do: detect price dislocations across different DEXes and pool types, and surface them for the strategy pipeline.

The infrastructure phase of this project is complete. The next posts will cover building the strategy and execution layers that turn these signals into profitable trades.

Related Posts

• Quote Service Production Validation: Zero Critical Deviations & a Chinese New Year Milestone - The Feb 27 Epoch 2 baseline validation
• Scanner Service Deep Dive: Arbitrage Detection Engine with Full Observability - Scanner service architecture overview
• Project Milestone Complete: Infrastructure Ready for Arbitrage Phase - Overall project status

Technical Documentation

• Quote Service Quality Assessment — Final Consolidated Report
• gRPC Test Client
• Scanner Service Architecture

Connect

• GitHub: guidebee/solana-trading-system
• LinkedIn: James Shen

This is post #29 in the Solana Trading System development series. A final consolidated quality assessment spanning 52 days, 11 test sessions, and 9.4M quotes confirms the scanner service is production ready — with multi-DEX arbitrage signal detection working correctly across different pool types and protocols during a live 10% SOL price rally.