Skip to content

Release Process Runbook

Overview

This runbook defines the process for releasing strategies and code changes to production. All releases require audit approval and staged rollout.

Release Types

Type Description Approval Required Rollout
Strategy Release New or modified trading strategy Full audit Staged
Model Update ML model retrain/update Model audit Staged
Bug Fix Non-trading code fix Code review Direct
Hotfix Critical live issue Expedited review Immediate
Config Change Parameter adjustment Risk review Direct

Versioning

Semantic Versioning

MAJOR.MINOR.PATCH

MAJOR: Breaking changes, strategy logic change
MINOR: New features, parameter additions
PATCH: Bug fixes, documentation updates

Version Tracking

# version.yaml (root of repo)
version: 1.2.3
release_date: 2024-01-15
changelog:
  - "1.2.3: Fixed timeout handling in webhook"
  - "1.2.2: Added kill switch API endpoint"
  - "1.2.1: Updated slippage model for crypto"
  - "1.2.0: Added session breakout strategy"
  - "1.1.0: Implemented paper trading layer"
  - "1.0.0: Initial release with ICT QM"

Strategy Versioning

# strategies/pine_v6/ict_qm.pine header
// @version=6
// Strategy: ICT_QM
// Version: 2.1.0
// Audit: AUD-2024-01-15-001 (PASS)
// Released: 2024-01-20

CI/CD Pipeline

Pre-Release Checks

# ci/release_checks.yaml
name: Release Checks

on:
  push:
    tags:
      - 'v*'

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Lint Python
        run: ruff check src/ tests/
      - name: Type check
        run: mypy src/

  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Run unit tests
        run: pytest tests/ -v --cov=src
      - name: Run anti-bias tests
        run: pytest tests/test_bias.py -v

  security:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Scan for secrets
        run: detect-secrets scan --all-files
      - name: Dependency audit
        run: pip-audit

  audit_check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Verify audit approval
        run: python scripts/verify_audit.py --version=${{ github.ref_name }}

Audit Verification Script

# scripts/verify_audit.py
import sys
import yaml

def verify_audit(version, strategy=None):
    """Verify audit approval for release."""

    # Load audit log
    with open('docs/audits/audit_log.md') as f:
        audit_log = f.read()

    # Check for PASS verdict
    if strategy:
        required_audit = f"{strategy}.*PASS"
        if not re.search(required_audit, audit_log):
            print(f"ERROR: No PASS audit found for {strategy}")
            sys.exit(1)

    # For any release, check version is documented
    if version not in audit_log:
        print(f"WARNING: Version {version} not found in audit log")

    print("Audit verification passed")
    sys.exit(0)

if __name__ == "__main__":
    version = sys.argv[1] if len(sys.argv) > 1 else "unknown"
    verify_audit(version)

Release Workflow

Phase 1: Preparation

1. Create release branch
   git checkout -b release/v1.2.3

2. Update version.yaml
   version: 1.2.3
   release_date: YYYY-MM-DD

3. Update CHANGELOG.md
   ## [1.2.3] - YYYY-MM-DD
   ### Changed
   - Description of changes

4. Run full test suite
   pytest tests/ -v

5. Verify audit status
   python scripts/verify_audit.py --strategy=ICT_QM

Phase 2: Review & Approval

1. Create pull request
   - Title: "Release v1.2.3"
   - Description: Changelog + audit reference

2. Required approvals:
   - [ ] Code review (any team member)
   - [ ] Audit approval (for strategy changes)
   - [ ] Risk review (for config changes)
   - [ ] Founder approval (for production)

3. CI checks must pass:
   - [ ] Lint
   - [ ] Tests
   - [ ] Security scan
   - [ ] Audit verification

Phase 3: Merge & Tag

# Merge to main
git checkout main
git merge --no-ff release/v1.2.3

# Tag release
git tag -a v1.2.3 -m "Release v1.2.3: Description"

# Push
git push origin main
git push origin v1.2.3

# Clean up
git branch -d release/v1.2.3

Phase 4: Staged Rollout

Day 0: Deploy to paper trading
├── Monitor for 24 hours
├── Check for errors
└── Verify signal generation

Day 1-3: Paper trading validation
├── Minimum 10 trades
├── No critical errors
└── Performance within expected range

Day 4: Deploy to live (minimal size)
├── 25% of target position size
├── Monitor closely
└── Kill switch ready

Day 7: Increase to 50% size
├── If Day 4-6 stable
├── Continue monitoring

Day 14: Full deployment
├── If Day 7-13 stable
├── 100% of target size
├── Standard monitoring

Audit Sign-Off Requirements

For Strategy Releases

Requirement Evidence Location
Audit report PASS or PASS-WITH-FIXES docs/audits/{strategy}_audit.md
All fixes complete Verified Audit report Section 5
Anti-bias tests All pass CI logs
Walk-forward > 60% consistency Audit report Section 6
Paper results Within 20% of backtest Paper reconciliation report

Audit Sign-Off Checklist

## Release Audit Sign-Off: v1.2.3

### Strategy: ICT_QM v2.1.0

- [ ] Audit report: docs/audits/ict_qm_audit.md
- [ ] Verdict: PASS
- [ ] Audit date: 2024-01-15
- [ ] Within 30 days: Yes

### Validation
- [ ] Anti-bias tests: PASS (CI run #123)
- [ ] Paper trading: 2 weeks, 25 trades
- [ ] Paper vs backtest: Sharpe 1.2 vs 1.4 (14% divergence)

### Approvals
- [ ] Auditor: _________________ Date: _______
- [ ] Risk: _________________ Date: _______
- [ ] Founder: _________________ Date: _______

### Release Decision
- [ ] APPROVED for production
- [ ] HOLD pending: _____________
- [ ] REJECTED reason: _____________

Rollback Procedure

When to Rollback

Condition Action
Live errors > 3 Immediate rollback
Unexpected losses Pause + review
Signal divergence > 30% Rollback within 24h
System instability Immediate rollback

Rollback Steps

# 1. Activate kill switch (if trading)
python -m execution.mt5.kill_switch --activate --reason "Rollback"

# 2. Identify previous stable version
git log --oneline --tags | head -5

# 3. Checkout previous version
git checkout v1.2.2

# 4. Deploy previous version
# (For Pine Script: revert in TradingView)
# (For Python: restart services with old code)

# 5. Verify rollback
python scripts/verify_deployment.py --version=v1.2.2

# 6. Deactivate kill switch (if safe)
python -m execution.mt5.kill_switch --deactivate --authorized-by="founder"

# 7. Document incident
# See docs/runbooks/incident_response.md

Release Checklist Template

# Release Checklist: v{VERSION}

## Pre-Release
- [ ] Version.yaml updated
- [ ] CHANGELOG.md updated
- [ ] All tests passing
- [ ] Audit approval (if strategy)
- [ ] Paper trading complete (if strategy)

## Review
- [ ] PR created
- [ ] Code review approved
- [ ] Risk review approved (if needed)
- [ ] Founder approval

## Deployment
- [ ] Tag created
- [ ] Deployed to paper
- [ ] Paper validation (24h min)
- [ ] Deployed to live (minimal)
- [ ] Monitoring configured
- [ ] Kill switch tested

## Post-Release
- [ ] Announcement sent
- [ ] Documentation updated
- [ ] Metrics baseline recorded

## Sign-Off
Released by: _____________
Date: _____________

Emergency Release (Hotfix)

Expedited Process

1. Create hotfix branch from production tag
   git checkout -b hotfix/critical-fix v1.2.3

2. Make minimal fix

3. Expedited review (Founder + 1 other)

4. Tag and deploy
   git tag -a v1.2.4-hotfix -m "Hotfix: description"

5. Deploy directly to live (skip paper)

6. Post-mortem within 24 hours

7. Proper release within 48 hours

Hotfix Criteria

Only these qualify for hotfix: - Active money at risk - Critical security vulnerability - Complete system failure - Regulatory requirement

All others follow standard release process.