Documentation

Autonomous Market

Documentation

Autonomous Market Strategy

The AI prediction engine (arb_bot/ai/) runs three flows via headless codex exec — no OpenAI API key, $0 per token. All LLM calls shell out to the local Codex CLI.

Transport: CodexRunner

arb_bot/ai/codex_runner.py — wraps codex exec --sandbox read-only --skip-git-repo-check --ephemeral --json -m <model> -o <out.json> -. The --search flag (if web_search=True) is placed before --sandbox. Token counts are parsed from JSONL type=token_count lines — input_tokens, output_tokens, and cached_tokens (prompt-cache hits) are all captured end-to-end and persisted. Model is set via Config.CODEX_MODEL (default gpt-5.5).

Auth & retries: the bot authenticates with the host's free ChatGPT login mounted at ~/.codex — no OpenAI API key, no per-token cost, and no budget vars. When CODEX_USE_FREE_CHATGPT_AUTH=true (default), the runner strips any stray OPENAI_API_KEY from the subprocess environment so the free login is used untouched. Transient failures (HTTP 429 rate-limit or a subprocess timeout) are retried up to CODEX_MAX_RETRIES times with CODEX_RETRY_BACKOFF_SEC backoff.

Enricher resilience: the three data enrichers (US markets, NSE technicals, FII/DII flows) share an in-process TTL cache (ENRICHER_CACHE_TTL_MIN, default 120 min) plus retry (ENRICHER_RETRY_COUNT, default 2). A transient yfinance/NSE outage falls back to the most recent good value instead of None, so a single provider hiccup no longer forces a NO_TRADE from missing data.

Data Collection

Market data is collected deterministically each cycle by MarketDataCollector (Dhan-side: spot, ATM strike, near expiry, IV, straddle, PCR, GIFT Nifty, RSI-14, MACD signal, S/R levels) and assembled into a MarketDataBundle. Before prompting Codex, MarketDataBundle.to_prompt_dict() normalizes that raw bundle into the schema the AI prompts expect: gift_nifty, us_markets, commodities_fx, asia_markets, domestic_flows, technicals, options_data, and event_risk. Macro data is fetched once per calendar day via Firecrawl (http://localhost:3002) and cached in-process. Per-cycle news is also fetched from Firecrawl. The MacroSearchCollector (arb_bot/ai/macro_search.py) handles both: Firecrawl for raw snippets, Codex (no web search) for structured extraction.

Flow 1 — AI_MARKET Single-Leg Trade

Two-stage workflow for the directional NIFTY single-leg option:

  1. 09:05 IST premarket thesis — run_analysis() calls Codex with the ai_market prompt task. Produces BULLISH / BEARISH / NO_TRADE verdict.
  2. 09:20 IST live confirmation — run_confirmation() fetches live spot/chain context, re-runs Codex, derives strike and index-based exit levels. Only the confirmation stage creates the dry-run trade.
  3. 15:25 IST forecast score — score_today_forecast() records directional precision and excursion stats.

Data-quality gate & data_quality block

After the premarket Codex call, _critical_coverage() checks five groups — us_markets, technicals, options, gift_nifty, domestic_flows. If 3+ are missing, the verdict is forced to NO_TRADE. The gate result is embedded into the persisted analysis_json under data_quality: {"covered": [...], "missing": [...], "coverage_pct": 40, "pcr_source": "oi"|"derived"|"missing"}. pcr_source is "oi" when Dhan OI-based PCR/max-pain is present; "derived" is a soft-miss guard that keeps the options group covered when the option chain itself is usable (spot + near straddle valid) even though the Dhan-side OI gap left PCR null — this prevents a single missing PCR from collapsing the verdict.

Flow 2 — Market View (shared prediction)

run_market_view() runs every 30 minutes (configurable via AI_PREDICT_INTERVAL_MIN). It builds a MarketDataBundle, enriches with macro/news, runs the market_view Codex task, and persists the result to the ai_predictions table (task = market_view). Each strategy spec may declare an ai_hint_consumer callable; at the start of each scan cycle bot.py loads the latest prediction (max age AI_PREDICT_MAX_AGE_MIN = 35 min) and applies the consumer to the signal. Consumers may set blocked_by_ai=True to veto entry.

Market view output schema: direction, conviction, regime, expected_range, vol_outlook, rationale.

Flow 3 — Breakout-Risk Exit Overlay

run_breakout_risk() runs every 30 minutes alongside market view. It runs the breakout_risk Codex task and persists to ai_predictions (task = breakout_risk). During monitor_open_positions(), before the normal exit check, _ai_breakout_exit() checks each range-bound trade (DC / DCS / IC) against the latest breakout-risk prediction. If breakout_expected=True, confidence ≥ AI_EXIT_MIN_CONFIDENCE, and spot is within AI_EXIT_TENT_BUFFER_PCT% of a short strike, the trade is flagged for exit. DDC and AI_MARKET are exempt (directional). When AI_EXIT_AUTO_CLOSE=True the trade is closed immediately; when False an ADVISORY_ONLY row is written to ai_exit_advisories and a Telegram alert is sent.

Flow 4 — Market Sentiment (deterministic)

SentimentService (arb_bot/ai/sentiment.py) runs every AI_SENTIMENT_INTERVAL_MIN minutes (default 10). Unlike Flows 1–3 it is fully deterministic — no LLM, no Codex call, $0 cost. Each run collects a MarketDataBundle, scores five weighted components into a transparent composite in [−100, +100], and persists one row to the ai_market_sentiment table.

Scoring components and default weights (the composite renormalizes over whichever components are present, so a missing input never corrupts the score):

  • Momentum (0.35) — tanh(spot_pct_move_vs_prev_close), capped at ±1.
  • PCR (0.25) — put/call OI level stance plus 2h trend; bullish above 1.0, bearish below.
  • Volatility (0.15) — rising IV/VIX is bearish (negated); currently fed via technicals cache until a live IV-delta source is wired.
  • Technical (0.15) — 200-EMA stance (60%) and RSI zone (40%); RSI ≥ 55 = bullish, ≤ 45 = bearish.
  • Global (0.10) — GIFT Nifty gap vs spot.

Each row records score, label (STRONGLY_BEARISH ≤ −50, BEARISH < −20, NEUTRAL, BULLISH < 50, STRONGLY_BULLISH ≥ 50), strength_pct (60% magnitude + 40% component agreement), coverage_pct (share of total weight contributed by present inputs), and the raw spot / prev_close / pcr / iv plus a components_json breakdown.

Sentiment-Risk Exit Overlay

During monitor_open_positions(), alongside the breakout overlay, _sentiment_exit() checks the latest sentiment row (max age AI_PREDICT_MAX_AGE_MIN) against each range-bound trade (DC / DCS / DCS_SKEW / IC / MDC). It auto-closes only when all four conditions hold (a deliberately high bar):

  1. The label is a hard flip — STRONGLY_BULLISH or STRONGLY_BEARISH.
  2. strength_pct ≥ AI_SENTIMENT_EXIT_MIN_STRENGTH (default 70) and coverage_pct ≥ AI_SENTIMENT_EXIT_MIN_COVERAGE_PCT (default 40).
  3. Spot is within AI_EXIT_TENT_BUFFER_PCT of a short strike (the tent is under live threat).
  4. A confirmed adverse move of ≥ AI_SENTIMENT_EXIT_ADVERSE_MOVE_PCT (default 0.6%) from prev_close.

DDC and AI_MARKET are exempt (directional — they benefit from trends). When AI_SENTIMENT_EXIT_AUTO_CLOSE=True the trade is closed immediately and an advisory row is written with reason="AI_SENTIMENT_RISK"; otherwise an ADVISORY_ONLY row plus a Telegram alert is emitted. AI_SENTIMENT_EXIT_SHADOW=True forces advisory-only regardless of the auto-close flag, for observation before trusting the overlay.

Promotion Readiness

AI_MARKET has an extra promotion gate beyond the normal 15-trading-day confidence check. GoLiveManager calls load_ai_market_promotion_report() before allowing /shadow AI_MARKET or /promote AI_MARKET.

Dashboard Surfaces

  • GET /api/ai-market/latest — latest AI Market analysis + trade.
  • GET /api/ai-market/performance — dry-run / live performance stats.
  • GET /api/ai-market/data-quality?date=YYYY-MM-DD — data-quality block (covered / missing inputs, coverage_pct, pcr_source) embedded in the day's analysis_json; null when the block is absent.
  • GET /api/ai-market/calibration — forecast calibration (direction accuracy, Brier score, MFE/MAE) from load_ai_forecast_performance().
  • GET /api/ai/predictions — latest market_view and breakout_risk predictions.
  • GET /api/ai/exit-advisories?date=YYYY-MM-DD — advisory rows for the given date.
  • GET /api/sentiment/latest — most recent market-sentiment reading (score, label, coverage, strength); {} when no fresh row exists.
  • GET /api/sentiment/history?hours=N — recent sentiment readings for the gauge sparkline.
  • React — AI Market page: AIPredictionsPanel (market-view + breakout-risk predictions, exit advisories) plus DataQualitySection and CalibrationSection cards, all themed via CSS variables.
  • React — main Dashboard: SentimentGaugePanel (diverging-bar gauge, refreshes every 60 s) and a per-trade SENT ±N badge on open DC/DCS/SDCS/IC/MDC rows in TradeTable.

Operator Commands

  • /ai_market — stored execution status and rejection reason.
  • /ai_promotion_report — promotion report.
  • /rerun_ai_market — requests OTP; /rerun_ai_market <otp> forces same-day rerun.
  • /shadow AI_MARKET / /promote AI_MARKET — rejected if promotion report fails.