Review and Merge Runbook¶

Overview¶

This runbook defines the process for reviewing AI-generated code and documentation before merging into the main codebase.

Review Requirements by File Type¶

File Type	Required Reviews	Auto-Merge Allowed
Strategy code (Pine/Python)	Auditor + Founder	No
ML model code	Auditor + Founder	No
Feature engineering	Quant review	No
Documentation	Light review	Yes (non-critical)
Configuration	Risk review	No
Tests	Quant review	Yes

Review Checklist by Category¶

Strategy Code Review¶

## Strategy Code Review Checklist

### Correctness
- [ ] Entry logic matches documented hypothesis
- [ ] Exit logic handles all scenarios (TP, SL, timeout)
- [ ] Position sizing respects risk limits
- [ ] No division by zero or null handling issues

### Anti-Bias
- [ ] No lookahead: signals use only past data
- [ ] No repainting: signals don't change after bar close
- [ ] Time alignment: correct bar indexing
- [ ] Data handling: missing data handled gracefully

### Code Quality
- [ ] Follows naming conventions (see repo_tree.md)
- [ ] Parameters are configurable (no hardcoded values)
- [ ] Comments explain non-obvious logic
- [ ] No dead code or unused variables

### Risk Compliance
- [ ] Stop loss always defined
- [ ] Position size calculations are correct
- [ ] Max drawdown scenario is survivable
- [ ] Compatible with prop firm rules

### Testing
- [ ] Compiles without errors
- [ ] Basic backtest runs
- [ ] Edge cases handled (gaps, holidays, etc.)

ML Model Review¶

## ML Model Review Checklist

### Data Integrity
- [ ] Training data is point-in-time correct
- [ ] No label leakage in features
- [ ] Train/test split respects time ordering
- [ ] Purge/embargo gaps are sufficient

### Model Quality
- [ ] OOS performance within acceptable degradation
- [ ] Calibration error < 10% (if probability model)
- [ ] Feature importance is reasonable
- [ ] No single feature dominates

### Documentation
- [ ] Model card is complete
- [ ] Training config is documented
- [ ] Validation report exists
- [ ] Kill triggers are defined

### Deployment Readiness
- [ ] Model artifact is versioned
- [ ] Rollback procedure exists
- [ ] Monitoring is configured

Configuration Review¶

## Configuration Review Checklist

### Risk Parameters
- [ ] Max drawdown limits are appropriate
- [ ] Position size limits are set
- [ ] Daily loss limits are set
- [ ] Prop firm compliance verified

### Alert Thresholds
- [ ] Warning thresholds are meaningful
- [ ] Critical thresholds trigger appropriate action
- [ ] No missing alert configurations

### Credentials
- [ ] No secrets in config files
- [ ] Environment variables used for sensitive data
- [ ] API keys are not exposed

Merge Process¶

Step 1: Pre-Merge Validation¶

# 1. Ensure branch is up to date
git fetch origin
git rebase origin/main  # Or merge, per team preference

# 2. Run automated checks
pytest tests/  # All tests pass
python -m src.validation.anti_bias_check  # If applicable

# 3. Verify no secrets
git diff origin/main --name-only | xargs grep -l "api_key\|password\|secret" || echo "No secrets found"

Step 2: Review Submission¶

# Review request format
review_request:
  branch: feature/strat-001-ict-qm
  type: strategy | model | config | docs
  description: |
    Brief description of changes
  files_changed:
    - strategies/pine_v6/ict_qm.pine
    - docs/strategies/ict_qm.md
  reviewers:
    - independent_auditor (required)
    - founder (required for strategy/model)
  audit_report: docs/audits/ict_qm_audit.md (if applicable)

Step 3: Review Execution¶

## Reviewer Actions

1. **Pull branch locally**
   git checkout feature/strat-001-ict-qm

2. **Run through checklist**
   - Use appropriate checklist above
   - Document findings

3. **Test changes**
   - Run relevant tests
   - Manual inspection if needed

4. **Provide feedback**
   - APPROVE: All checks pass
   - REQUEST CHANGES: Issues found, list them
   - COMMENT: Questions or suggestions

Step 4: Merge Execution¶

# Only after all required approvals

# 1. Final rebase
git checkout main
git pull origin main
git checkout feature/strat-001-ict-qm
git rebase main

# 2. Squash if many commits (optional)
git rebase -i main  # Squash to meaningful commits

# 3. Merge
git checkout main
git merge --no-ff feature/strat-001-ict-qm

# 4. Push
git push origin main

# 5. Clean up
git branch -d feature/strat-001-ict-qm
git push origin --delete feature/strat-001-ict-qm

Special Merge Scenarios¶

Scenario: Hotfix to Production Strategy¶

process:
  1. Create hotfix branch from main
  2. Minimal change to fix issue
  3. Expedited review (Founder + Auditor)
  4. Direct merge to main
  5. Immediate paper testing
  6. Post-merge full audit within 48 hours

Scenario: Breaking Change¶

process:
  1. Document breaking changes in PR description
  2. Update all dependent code
  3. Migration guide if needed
  4. Extended review period
  5. Staged rollout plan

Scenario: Reverting a Merge¶

# If issues discovered post-merge

# 1. Identify merge commit
git log --oneline --merges -10

# 2. Revert the merge
git revert -m 1 <merge_commit_hash>

# 3. Push revert
git push origin main

# 4. Document in incident log
# See docs/runbooks/incident_log.md

Automated Checks (CI)¶

Pre-Merge CI Pipeline¶

# ci/pre_merge.yaml
name: Pre-Merge Checks

on:
  pull_request:
    branches: [main, develop]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Lint Python
        run: ruff check src/ tests/
      - name: Lint Pine (syntax check)
        run: python scripts/validate_pinescript.py

  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Run tests
        run: pytest tests/ -v

  anti_bias:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Anti-bias checks
        run: pytest tests/test_bias.py -v

  secrets_scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Scan for secrets
        run: |
          pip install detect-secrets
          detect-secrets scan --all-files

Review Metrics¶

Track These Metrics¶

Metric	Target	Review Frequency
Review turnaround time	< 24 hours	Weekly
First-pass approval rate	> 70%	Monthly
Post-merge issues	< 5%	Monthly
Audit pass rate	> 80%	Monthly

Review Log¶

Maintain log at docs/runbooks/review_log.md:

| Date | Branch | Type | Reviewers | Outcome | Notes |
|------|--------|------|-----------|---------|-------|
| 2024-01-15 | strat-001 | strategy | Auditor, Founder | Approved | |
| 2024-01-16 | model-002 | model | Auditor, Founder | Changes requested | Calibration issue |

Next 7 Days: Action Plan¶

Set up pre-merge CI pipeline in ci/pre_merge.yaml
Create review request template in .github/PULL_REQUEST_TEMPLATE.md
Initialize docs/runbooks/review_log.md
Configure branch protection rules on main
Create audit report template in docs/audits/template.md
Test full review workflow end-to-end
Document any process gaps discovered