Validation Checklist
Overview
This checklist must be completed for every strategy before deployment. ALL items must pass for approval. Any single failure blocks deployment.
Section 1: Anti-Bias Tests
1.1 Lookahead Detection
| Test | Method | Pass Criteria | Result |
| Truncation test | Run on data[0:N], then data[0:N+100]; signals for 0:N must match | 100% match | [ ] PASS / [ ] FAIL |
| Bar index audit | Verify all feature calculations use [1] or earlier | No [0] in features | [ ] PASS / [ ] FAIL |
| High/Low usage | Check if same-bar high/low used for entry decisions | Not used | [ ] PASS / [ ] FAIL |
| Future price reference | Grep for forward-looking array indices | None found | [ ] PASS / [ ] FAIL |
Lookahead Audit Commands:
# Check for common lookahead patterns
grep -n "\[0\]" strategy.pine | grep -v "close\[0\]" # Suspicious
grep -n "high\[0\]\|low\[0\]" strategy.pine # Red flag for intrabar
1.2 Label Leakage Detection
| Test | Method | Pass Criteria | Result |
| Feature-label correlation | Check single-feature AUC | No feature AUC > 0.8 | [ ] PASS / [ ] FAIL |
| Temporal ordering | Verify label created AFTER all features | Confirmed | [ ] PASS / [ ] FAIL |
| Information flow | Trace data dependencies | No circular refs | [ ] PASS / [ ] FAIL |
1.3 Repaint Detection
| Test | Method | Pass Criteria | Result |
| Historical vs realtime | Compare chart replay to static | Signals identical | [ ] PASS / [ ] FAIL |
| Bar close lock | Verify signals computed at bar close only | Confirmed | [ ] PASS / [ ] FAIL |
barstate.isconfirmed usage | Check if used correctly | Correct or N/A | [ ] PASS / [ ] FAIL |
request.security lookahead | Verify lookahead=barmerge.lookahead_off | Confirmed | [ ] PASS / [ ] FAIL |
Repaint Detection Protocol: 1. Load strategy on chart 2. Note current signals 3. Wait for 10+ new bars 4. Check if historical signals changed 5. If changed → FAIL
Section 2: Sensitivity Analysis
2.1 Parameter Sensitivity
| Test | Method | Pass Criteria | Result |
| ±10% parameter sweep | Vary each param ±10% | Sharpe change < 30% | [ ] PASS / [ ] FAIL |
| ±20% parameter sweep | Vary each param ±20% | Still profitable | [ ] PASS / [ ] FAIL |
| Correlation of params | Test pairs of parameters | No cliff edges | [ ] PASS / [ ] FAIL |
Parameter Stability Score:
Stability = 1 - (Sharpe_std / Sharpe_mean) across parameter variations
Target: Stability > 0.7
2.2 Time Period Sensitivity
| Test | Method | Pass Criteria | Result |
| Different start dates | Shift start by 3, 6, 12 months | Results consistent | [ ] PASS / [ ] FAIL |
| Different end dates | Test on recent 1yr vs 2yr vs 3yr | No major divergence | [ ] PASS / [ ] FAIL |
| Year-by-year breakdown | Performance per calendar year | Profitable in 60%+ years | [ ] PASS / [ ] FAIL |
Section 3: Stress Tests
3.1 Market Stress Events
| Event | Date Range | Test Requirement | Result |
| COVID crash | 2020-02-15 to 2020-04-15 | Survives or exits cleanly | [ ] PASS / [ ] FAIL |
| Flash crash | 2010-05-06 | Handles gaps | [ ] PASS / [ ] FAIL |
| Brexit vote | 2016-06-23 to 2016-06-27 | No catastrophic loss | [ ] PASS / [ ] FAIL |
| Crypto crash | 2022-05-01 to 2022-06-30 | (If crypto) Manages DD | [ ] PASS / [ ] FAIL |
| SVB/Banking crisis | 2023-03-08 to 2023-03-15 | Handles volatility | [ ] PASS / [ ] FAIL |
3.2 Adverse Conditions
| Condition | Simulation | Pass Criteria | Result |
| 2x slippage | Double slippage model | Still profitable | [ ] PASS / [ ] FAIL |
| 2x spread | Double spread costs | Still profitable | [ ] PASS / [ ] FAIL |
| 50% fill rate | Random order rejection | Degrades gracefully | [ ] PASS / [ ] FAIL |
| 10-bar execution delay | Entry delayed 10 bars | Edge survives | [ ] PASS / [ ] FAIL |
3.3 Gap Handling
| Test | Scenario | Pass Criteria | Result |
| Weekend gap | Gap through SL | Fills at open, limited loss | [ ] PASS / [ ] FAIL |
| News gap | 5% gap | No position explosion | [ ] PASS / [ ] FAIL |
| Circuit breaker | Trading halt | Handles gracefully | [ ] PASS / [ ] FAIL |
Section 4: Regime Robustness
4.1 Volatility Regimes
| Regime | Definition | Requirement | Result |
| Low vol | ATR < 25th percentile | Not negative | [ ] PASS / [ ] FAIL |
| Normal vol | ATR 25th-75th percentile | Profitable | [ ] PASS / [ ] FAIL |
| High vol | ATR > 75th percentile | Not catastrophic | [ ] PASS / [ ] FAIL |
| Extreme vol | ATR > 95th percentile | Survives | [ ] PASS / [ ] FAIL |
4.2 Trend Regimes
| Regime | Definition | Requirement | Result |
| Strong uptrend | ADX > 30, +DI > -DI | Profitable or flat | [ ] PASS / [ ] FAIL |
| Strong downtrend | ADX > 30, -DI > +DI | Profitable or flat | [ ] PASS / [ ] FAIL |
| Ranging | ADX < 20 | Not catastrophic | [ ] PASS / [ ] FAIL |
| Choppy | ADX 20-25, frequent crosses | Minimal losses | [ ] PASS / [ ] FAIL |
4.3 Cross-Regime Requirement
Strategy must be profitable in at least 2 of 3 major regime types.
If profitable in only 1 regime: Requires regime filter before deployment.
Section 5: Cost Realism
5.1 Commission Model
| Market | Model Used | Verified Against | Result |
| FX | $ per lot | Broker actual | [ ] PASS / [ ] FAIL |
| Crypto | % of notional | Exchange fees | [ ] PASS / [ ] FAIL |
| Futures | $ per contract | Broker actual | [ ] PASS / [ ] FAIL |
5.2 Spread Model
| Market | Spread Used | Actual Observed | Within 20%? | Result |
| EUR/USD | | | | [ ] PASS / [ ] FAIL |
| BTC/USD | | | | [ ] PASS / [ ] FAIL |
| NQ | | | | [ ] PASS / [ ] FAIL |
| Gold | | | | [ ] PASS / [ ] FAIL |
5.3 Slippage Model
| Condition | Model Assumption | Realistic? | Result |
| Normal market | X ticks | | [ ] PASS / [ ] FAIL |
| High volatility | X ticks | | [ ] PASS / [ ] FAIL |
| Low liquidity | X ticks | | [ ] PASS / [ ] FAIL |
Section 6: Walk-Forward / OOS Validation
6.1 Data Split
| Split | Period | Bars | Purpose |
| In-Sample | YYYY-MM-DD to YYYY-MM-DD | | Training/optimization |
| Out-of-Sample | YYYY-MM-DD to YYYY-MM-DD | | Final validation |
| OOS Ratio | | | Target: ≥25% |
6.2 Walk-Forward Results
| Window | Train Period | Test Period | Sharpe | Profitable? |
| 1 | | | | [ ] Yes / [ ] No |
| 2 | | | | [ ] Yes / [ ] No |
| 3 | | | | [ ] Yes / [ ] No |
| 4 | | | | [ ] Yes / [ ] No |
| 5 | | | | [ ] Yes / [ ] No |
| 6 | | | | [ ] Yes / [ ] No |
Walk-Forward Consistency: ___/6 profitable (target: ≥4/6 = 67%)
6.3 OOS Degradation
| Metric | In-Sample | Out-of-Sample | Degradation | Pass (<30%)? |
| Sharpe | | | % | [ ] PASS / [ ] FAIL |
| Win Rate | | | % | [ ] PASS / [ ] FAIL |
| Profit Factor | | | % | [ ] PASS / [ ] FAIL |
| Max DD | | | % | [ ] PASS / [ ] FAIL |
Section 7: Final Verdict
Summary
| Section | Pass/Fail |
| 1. Anti-Bias Tests | [ ] PASS / [ ] FAIL |
| 2. Sensitivity Analysis | [ ] PASS / [ ] FAIL |
| 3. Stress Tests | [ ] PASS / [ ] FAIL |
| 4. Regime Robustness | [ ] PASS / [ ] FAIL |
| 5. Cost Realism | [ ] PASS / [ ] FAIL |
| 6. Walk-Forward/OOS | [ ] PASS / [ ] FAIL |
Overall Verdict
Validator
| Field | Value |
| Validated by | |
| Date | |
| Signature | |