System Architecture¶

Overview¶

The Edge Factory operates as a six-stage pipeline with strict gates between stages.

┌──────────────────────────────────────────────────────────────────────────────┐
│                           EDGE FACTORY PIPELINE                              │
├──────────────────────────────────────────────────────────────────────────────┤
│                                                                              │
│  ┌─────────┐    ┌──────────┐    ┌──────────┐    ┌─────────┐    ┌─────────┐  │
│  │  DATA   │───▶│ RESEARCH │───▶│ BACKTEST │───▶│  PAPER  │───▶│  LIVE   │  │
│  │ LAYER   │    │  LAYER   │    │  LAYER   │    │ TRADING │    │ TRADING │  │
│  └────┬────┘    └────┬─────┘    └────┬─────┘    └────┬────┘    └────┬────┘  │
│       │              │               │               │              │        │
│       └──────────────┴───────────────┴───────────────┴──────────────┘        │
│                                      │                                       │
│                                      ▼                                       │
│                            ┌──────────────────┐                              │
│                            │    MONITORING    │                              │
│                            │      LAYER       │                              │
│                            └──────────────────┘                              │
└──────────────────────────────────────────────────────────────────────────────┘

Layer Definitions¶

Layer 1: Data¶

Component	Purpose	Location
Connectors	Fetch raw data from sources	`src/data/connectors/`
Validators	Check data quality, gaps, anomalies	`src/data/validators/`
Storage	Versioned parquet/CSV files	`data/raw/`, `data/processed/`
Catalog	Metadata registry	`data/catalog.yaml`

Data Flow:

External APIs → Connectors → Validators → Storage → Catalog
     │              │            │           │
     └── Raw ───────┴── Clean ───┴── Versioned

Layer 2: Research¶

Component	Purpose	Location
Feature Engineering	Transform raw data to signals	`src/features/`
Hypothesis Testing	Statistical validation	`research/notebooks/`
Signal Prototypes	Initial strategy logic	`strategies/python/`
Reports	Documented findings	`research/reports/`

Research Flow:

Processed Data → Features → Hypothesis → Prototype → Report
                    │           │            │
                    └── Document everything ─┘

Layer 3: Backtest¶

Component	Purpose	Location
Backtest Engine	Event-driven simulation	`src/evaluation/backtester.py`
Cost Models	Commissions, spreads, slippage	`src/evaluation/costs.py`
Anti-Bias Tests	Detect lookahead, overfitting	`tests/test_bias.py`
Walk-Forward	OOS validation protocol	`src/evaluation/walkforward.py`

Backtest Flow:

Strategy + Data → Engine → Results → Anti-Bias → Walk-Forward → Report
                    │          │         │            │
                    └── Full audit trail ─┴────────────┘

Layer 4: Paper Trading¶

Component	Purpose	Location
Signal Generator	TradingView Pine indicators	`strategies/pine_v6/`
Alert Webhook	JSON signals to execution	TradingView cloud
Paper Executor	Demo account order routing	`execution/paper/`
Reconciliation	Compare signals vs fills	`src/evaluation/reconcile.py`

Paper Flow:

Pine Signal → Webhook → Paper Executor → Fill Log → Reconciliation
                 │            │              │
                 └── Latency measured ───────┘

Layer 5: Live Trading¶

Component	Purpose	Location
Execution Router	Route orders to broker/exchange	`execution/`
MT5 Connector	FX/CFD execution	`execution/mt5/`
TradingView Alerts	Signal source	`execution/tradingview_alerts/`
Position Manager	Track open positions	`src/execution/positions.py`

Live Flow:

Signal → Risk Check → Execution Router → Broker API → Confirmation
            │               │                │
            └── All steps logged ────────────┘

Layer 6: Monitoring¶

Component	Purpose	Location
KPI Tracker	Real-time performance metrics	`src/monitoring/kpis.py`
Drift Detector	Strategy decay alerts	`src/monitoring/drift.py`
Alert System	Notifications (Discord/Telegram)	`src/monitoring/alerts.py`
Dashboard	Visual monitoring	`src/monitoring/dashboard/`

Monitoring Flow:

All Layers → Event Stream → KPI Tracker → Drift Detector → Alerts
                  │              │              │
                  └── Dashboard aggregation ───┘

Directory Structure¶

rqf-ml/
├── config/                    # Configuration files
│   ├── instruments.yaml       # Per-instrument settings
│   ├── prop_firms.yaml        # Prop firm limits
│   └── risk_limits.yaml       # Global risk parameters
│
├── data/                      # Data storage
│   ├── raw/                   # Unprocessed data
│   ├── processed/             # Clean, validated data
│   └── catalog.yaml           # Data registry
│
├── docs/                      # Documentation
│   ├── architecture.md        # This file
│   ├── playbooks/             # Operational playbooks
│   ├── runbooks/              # Incident response
│   └── audits/                # Strategy audits
│
├── execution/                 # Live/paper execution
│   ├── mt5/                   # MetaTrader 5 connector
│   ├── tradingview_alerts/    # Webhook handlers
│   └── paper/                 # Paper trading
│
├── research/                  # Research artifacts
│   ├── notebooks/             # Jupyter notebooks
│   └── reports/               # Documented findings
│
├── src/                       # Source code
│   ├── data/                  # Data pipeline
│   ├── features/              # Feature engineering
│   ├── evaluation/            # Backtest engine
│   ├── execution/             # Order management
│   └── monitoring/            # Live monitoring
│
├── strategies/                # Strategy code
│   ├── pine_v6/               # TradingView indicators
│   └── python/                # Python strategies
│
└── tests/                     # Test suite
    ├── test_bias.py           # Anti-bias tests
    ├── test_backtest.py       # Backtest validation
    └── adversarial/           # Adversarial tests

Integration Points¶

From	To	Method
Data → Research	File read	Parquet/CSV
Research → Backtest	Python import	DataFrame
Backtest → Paper	Manual migration	Pine Script
Paper → Live	Config change	Same code
All → Monitoring	Event emission	JSON logs
Monitoring → Alerts	HTTP webhook	JSON

Technology Stack¶

Function	Technology	Rationale
Data storage	Parquet + Git LFS	Versioned, efficient
Backtesting	Python (vectorbt)	Fast, flexible
Live signals	Pine Script v6	TradingView native
FX execution	MT5 + MQL5	Industry standard
Crypto execution	Exchange APIs	Direct access
Monitoring	Python + SQLite	Simple, portable
Alerts	Discord/Telegram API	Instant delivery
Version control	Git	Standard

Failure Modes¶

Failure	Detection	Response
Data feed down	No new data > 5 min	Alert + pause signals
Backtest anomaly	Sharpe > 5 or < -2	Flag for review
Paper divergence	Fills differ > 20%	Investigate execution
Live order failure	No confirmation	Retry → Alert → Manual
Monitoring down	Health check fails	Redundant alert channel

Security Considerations¶

Asset	Protection
API keys	Environment variables, never in code
Strategy logic	Private repo, access control
Trade data	Local storage, encrypted backups
Alerts	Authenticated webhooks