Skip to content

Audit Thresholds Playbook

Purpose

Define default, reusable thresholds used in docs/audits/audit_template.md.

These are defaults, not guarantees. The auditor may tighten thresholds based on: - leverage / account constraints (prop vs live) - liquidity and spread regime - strategy capacity - tail risk profile

If a strategy claims an exception (e.g., lower Sharpe but strong convexity), it must justify it explicitly in the audit.

How to Use

In each audit report, set these variables: - {{IS_SHARPE_MIN}} - {{OOS_SHARPE_MIN}} - {{OOS_DEGRADATION_MAX}} - {{WF_CONSISTENCY_MIN}} - {{MAX_DD_MAX}} - {{PF_MIN}} - {{MIN_OOS_TRADES}}

Then choose a profile below and copy the defaults into the audit.

Global Minimums (Hard Gates)

These apply to all strategies unless explicitly waived in writing.

Metric Default Hard Gate Rule
Anti-bias tests All PASS Any FAIL → FAIL verdict
OOS trade count ≥ 200 If < 100 → FAIL (insufficient evidence)
OOS degradation < 40% If ≥ 60% → FAIL (edge likely unstable)
Max Drawdown < 15% If > 20% → FAIL (unless explicitly "high-risk sleeve")
Cost realism Declared + tested Missing costs/slippage assumptions → PASS-WITH-FIXES minimum

Notes - "OOS degradation" means drop in key metric (Sharpe, PF, expectancy) from IS to OOS. - If OOS is very small, treat results as low confidence and demand more evidence.

Profiles (Choose One)

Profile A: Intraday Trend / Breakout (FX, Indices, Futures)

Good for NASDAQ futures, major FX pairs, liquid futures.

Defaults - {{IS_SHARPE_MIN}} = 1.2 - {{OOS_SHARPE_MIN}} = 0.9 - {{OOS_DEGRADATION_MAX}} = 35% - {{WF_CONSISTENCY_MIN}} = 60% - {{MAX_DD_MAX}} = 12% - {{PF_MIN}} = 1.25 - {{MIN_OOS_TRADES}} = 200

Extra gates - Must survive chop regimes without catastrophic DD. - Must include volatility/range filter or risk throttle.

Profile B: Intraday Mean Reversion (Indices/Futures, Some FX)

Mean reversion often looks great in-sample and dies out-of-sample, so be stricter.

Defaults - {{IS_SHARPE_MIN}} = 1.4 - {{OOS_SHARPE_MIN}} = 1.0 - {{OOS_DEGRADATION_MAX}} = 30% - {{WF_CONSISTENCY_MIN}} = 65% - {{MAX_DD_MAX}} = 10% - {{PF_MIN}} = 1.35 - {{MIN_OOS_TRADES}} = 250

Extra gates - Must pass stress tests: spread spikes + gap scenarios. - Must show parameter stability across bands/thresholds.

Profile C: Swing Trend (FX/Crypto/Indices)

Lower trade count, longer holds, higher variance. Require longer OOS window.

Defaults - {{IS_SHARPE_MIN}} = 1.0 - {{OOS_SHARPE_MIN}} = 0.7 - {{OOS_DEGRADATION_MAX}} = 40% - {{WF_CONSISTENCY_MIN}} = 55% - {{MAX_DD_MAX}} = 15% - {{PF_MIN}} = 1.20 - {{MIN_OOS_TRADES}} = 80 (but require long time window)

Extra gates - OOS must cover multiple volatility regimes (at least 2). - Must define event/news risk rules if applicable.

Profile D: Crypto 24/7 Momentum / Trend (BTC/ETH + majors)

Crypto has regime shifts, funding effects, weekend behavior. Require regime testing.

Defaults - {{IS_SHARPE_MIN}} = 1.1 - {{OOS_SHARPE_MIN}} = 0.8 - {{OOS_DEGRADATION_MAX}} = 40% - {{WF_CONSISTENCY_MIN}} = 55% - {{MAX_DD_MAX}} = 15% - {{PF_MIN}} = 1.20 - {{MIN_OOS_TRADES}} = 150

Extra gates - Must include funding/fees assumptions. - Must test weekends vs weekdays. - Must test high-vol crash weeks.

Profile E: Prop-Firm Conservative (Any Asset)

This profile is designed to minimize rule violations and smooth equity curve. It sacrifices raw return to avoid daily loss breaches.

Defaults - {{IS_SHARPE_MIN}} = 1.3 - {{OOS_SHARPE_MIN}} = 1.0 - {{OOS_DEGRADATION_MAX}} = 30% - {{WF_CONSISTENCY_MIN}} = 70% - {{MAX_DD_MAX}} = 8% - {{PF_MIN}} = 1.30 - {{MIN_OOS_TRADES}} = 150

Extra gates - Worst day must not breach representative daily loss limit under baseline costs. - Must include strict kill-switch rules and scaling ladder. - Must cap consecutive losses and max trades/day if needed.

Cost & Slippage Standards (Baseline + Stressed)

Audits must declare BOTH baseline and stressed assumptions.

FX (majors)

  • Baseline: 0.5–1.5 pips + typical commission
  • Stressed: 2–5 pips during events

Indices / Futures (e.g., NQ)

  • Baseline: 1–2 ticks slippage + commission
  • Stressed: 3–8 ticks (news / open)

Metals (Gold/Silver futures)

  • Baseline: 1–3 ticks
  • Stressed: 5–12 ticks around data releases

Crypto (spot/perps)

  • Baseline: 2–10 bps + fees + funding estimate
  • Stressed: 20–80 bps in volatility spikes

If the strategy fails under stressed costs, it must either: - add execution filters, or - reduce trade frequency, or - redesign the entry/exit model.

Walk-Forward Consistency Definition

A "walk-forward window" is counted as positive if it meets BOTH: - PF ≥ ({{PF_MIN}} - 0.10) OR Sharpe ≥ ({{OOS_SHARPE_MIN}} - 0.1) - DD within 1.25× of target (e.g., if MAX_DD_MAX=10%, WF window DD ≤ 12.5%)

Then: - {{WF_CONSISTENCY_MIN}} is the % of positive windows.

When to FAIL immediately (Non-Negotiables)

  • Any confirmed lookahead/leakage/repainting used for signals
  • Missing cost/slippage assumptions after audit request
  • OOS results based on tiny sample size with no plan to expand
  • Extreme sensitivity: small parameter changes flip profitability
  • Worst-week or worst-day breaches typical risk constraints with no mitigation plan

When PASS-WITH-FIXES is acceptable

  • Strategy is sound but missing:
  • explicit cost model tables
  • stress tests
  • reproducibility evidence
  • kill-switch table
  • parameter sensitivity sweep

Must include a deadline and rerun evidence.

Notes on Interpretation

  • A lower Sharpe can be acceptable if the strategy is uncorrelated and improves portfolio-level Sharpe.
  • Strategies should be judged not only on return but on:
  • stability
  • drawdown behavior
  • tail risk
  • execution robustness
  • capacity

If claiming portfolio value, the audit must include correlation reasoning.