Skip to content

Validation Checklist

Overview

This checklist must be completed for every strategy before deployment. ALL items must pass for approval. Any single failure blocks deployment.


Section 1: Anti-Bias Tests

1.1 Lookahead Detection

Test Method Pass Criteria Result
Truncation test Run on data[0:N], then data[0:N+100]; signals for 0:N must match 100% match [ ] PASS / [ ] FAIL
Bar index audit Verify all feature calculations use [1] or earlier No [0] in features [ ] PASS / [ ] FAIL
High/Low usage Check if same-bar high/low used for entry decisions Not used [ ] PASS / [ ] FAIL
Future price reference Grep for forward-looking array indices None found [ ] PASS / [ ] FAIL

Lookahead Audit Commands:

# Check for common lookahead patterns
grep -n "\[0\]" strategy.pine | grep -v "close\[0\]"  # Suspicious
grep -n "high\[0\]\|low\[0\]" strategy.pine  # Red flag for intrabar

1.2 Label Leakage Detection

Test Method Pass Criteria Result
Feature-label correlation Check single-feature AUC No feature AUC > 0.8 [ ] PASS / [ ] FAIL
Temporal ordering Verify label created AFTER all features Confirmed [ ] PASS / [ ] FAIL
Information flow Trace data dependencies No circular refs [ ] PASS / [ ] FAIL

1.3 Repaint Detection

Test Method Pass Criteria Result
Historical vs realtime Compare chart replay to static Signals identical [ ] PASS / [ ] FAIL
Bar close lock Verify signals computed at bar close only Confirmed [ ] PASS / [ ] FAIL
barstate.isconfirmed usage Check if used correctly Correct or N/A [ ] PASS / [ ] FAIL
request.security lookahead Verify lookahead=barmerge.lookahead_off Confirmed [ ] PASS / [ ] FAIL

Repaint Detection Protocol: 1. Load strategy on chart 2. Note current signals 3. Wait for 10+ new bars 4. Check if historical signals changed 5. If changed → FAIL


Section 2: Sensitivity Analysis

2.1 Parameter Sensitivity

Test Method Pass Criteria Result
±10% parameter sweep Vary each param ±10% Sharpe change < 30% [ ] PASS / [ ] FAIL
±20% parameter sweep Vary each param ±20% Still profitable [ ] PASS / [ ] FAIL
Correlation of params Test pairs of parameters No cliff edges [ ] PASS / [ ] FAIL

Parameter Stability Score:

Stability = 1 - (Sharpe_std / Sharpe_mean) across parameter variations
Target: Stability > 0.7

2.2 Time Period Sensitivity

Test Method Pass Criteria Result
Different start dates Shift start by 3, 6, 12 months Results consistent [ ] PASS / [ ] FAIL
Different end dates Test on recent 1yr vs 2yr vs 3yr No major divergence [ ] PASS / [ ] FAIL
Year-by-year breakdown Performance per calendar year Profitable in 60%+ years [ ] PASS / [ ] FAIL

Section 3: Stress Tests

3.1 Market Stress Events

Event Date Range Test Requirement Result
COVID crash 2020-02-15 to 2020-04-15 Survives or exits cleanly [ ] PASS / [ ] FAIL
Flash crash 2010-05-06 Handles gaps [ ] PASS / [ ] FAIL
Brexit vote 2016-06-23 to 2016-06-27 No catastrophic loss [ ] PASS / [ ] FAIL
Crypto crash 2022-05-01 to 2022-06-30 (If crypto) Manages DD [ ] PASS / [ ] FAIL
SVB/Banking crisis 2023-03-08 to 2023-03-15 Handles volatility [ ] PASS / [ ] FAIL

3.2 Adverse Conditions

Condition Simulation Pass Criteria Result
2x slippage Double slippage model Still profitable [ ] PASS / [ ] FAIL
2x spread Double spread costs Still profitable [ ] PASS / [ ] FAIL
50% fill rate Random order rejection Degrades gracefully [ ] PASS / [ ] FAIL
10-bar execution delay Entry delayed 10 bars Edge survives [ ] PASS / [ ] FAIL

3.3 Gap Handling

Test Scenario Pass Criteria Result
Weekend gap Gap through SL Fills at open, limited loss [ ] PASS / [ ] FAIL
News gap 5% gap No position explosion [ ] PASS / [ ] FAIL
Circuit breaker Trading halt Handles gracefully [ ] PASS / [ ] FAIL

Section 4: Regime Robustness

4.1 Volatility Regimes

Regime Definition Requirement Result
Low vol ATR < 25th percentile Not negative [ ] PASS / [ ] FAIL
Normal vol ATR 25th-75th percentile Profitable [ ] PASS / [ ] FAIL
High vol ATR > 75th percentile Not catastrophic [ ] PASS / [ ] FAIL
Extreme vol ATR > 95th percentile Survives [ ] PASS / [ ] FAIL

4.2 Trend Regimes

Regime Definition Requirement Result
Strong uptrend ADX > 30, +DI > -DI Profitable or flat [ ] PASS / [ ] FAIL
Strong downtrend ADX > 30, -DI > +DI Profitable or flat [ ] PASS / [ ] FAIL
Ranging ADX < 20 Not catastrophic [ ] PASS / [ ] FAIL
Choppy ADX 20-25, frequent crosses Minimal losses [ ] PASS / [ ] FAIL

4.3 Cross-Regime Requirement

Strategy must be profitable in at least 2 of 3 major regime types.
If profitable in only 1 regime: Requires regime filter before deployment.

Section 5: Cost Realism

5.1 Commission Model

Market Model Used Verified Against Result
FX $ per lot Broker actual [ ] PASS / [ ] FAIL
Crypto % of notional Exchange fees [ ] PASS / [ ] FAIL
Futures $ per contract Broker actual [ ] PASS / [ ] FAIL

5.2 Spread Model

Market Spread Used Actual Observed Within 20%? Result
EUR/USD [ ] PASS / [ ] FAIL
BTC/USD [ ] PASS / [ ] FAIL
NQ [ ] PASS / [ ] FAIL
Gold [ ] PASS / [ ] FAIL

5.3 Slippage Model

Condition Model Assumption Realistic? Result
Normal market X ticks [ ] PASS / [ ] FAIL
High volatility X ticks [ ] PASS / [ ] FAIL
Low liquidity X ticks [ ] PASS / [ ] FAIL

Section 6: Walk-Forward / OOS Validation

6.1 Data Split

Split Period Bars Purpose
In-Sample YYYY-MM-DD to YYYY-MM-DD Training/optimization
Out-of-Sample YYYY-MM-DD to YYYY-MM-DD Final validation
OOS Ratio Target: ≥25%

6.2 Walk-Forward Results

Window Train Period Test Period Sharpe Profitable?
1 [ ] Yes / [ ] No
2 [ ] Yes / [ ] No
3 [ ] Yes / [ ] No
4 [ ] Yes / [ ] No
5 [ ] Yes / [ ] No
6 [ ] Yes / [ ] No

Walk-Forward Consistency: ___/6 profitable (target: ≥4/6 = 67%)

6.3 OOS Degradation

Metric In-Sample Out-of-Sample Degradation Pass (<30%)?
Sharpe % [ ] PASS / [ ] FAIL
Win Rate % [ ] PASS / [ ] FAIL
Profit Factor % [ ] PASS / [ ] FAIL
Max DD % [ ] PASS / [ ] FAIL

Section 7: Final Verdict

Summary

Section Pass/Fail
1. Anti-Bias Tests [ ] PASS / [ ] FAIL
2. Sensitivity Analysis [ ] PASS / [ ] FAIL
3. Stress Tests [ ] PASS / [ ] FAIL
4. Regime Robustness [ ] PASS / [ ] FAIL
5. Cost Realism [ ] PASS / [ ] FAIL
6. Walk-Forward/OOS [ ] PASS / [ ] FAIL

Overall Verdict

  • PASS: All sections pass → Proceed to audit
  • FAIL: One or more sections fail → Return to development

Validator

Field Value
Validated by
Date
Signature