Backtest Expert

**Judgement** Backtest Expert is a strong, methodology-focused skill for quantitative/systematic traders who want to design, stress test, and validate trading strategies in a robust, pessimistic way. It is best suited to users who already have at least a basic understanding of trading systems and want to avoid typical backtesting traps rather than chase the highest-looking historical P&L. **What it does well** - Enforces a disciplined workflow: clear edge statement → fully codified rules → long-horizon backtest → heavy stress testing → walk-forward/out-of-sample checks → deployment decision. - Strong focus on robustness: parameter plateaus instead of sharp optima, pessimistic friction/slippage, year-by-year and regime-by-regime analysis, and sufficient sample sizes. - Explicitly teaches users to detect and avoid common pitfalls: look-ahead and survivorship bias, over-optimization, fragile parameter dependence, and reliance on a few exceptional periods. - Keeps idea generation and statistical validation conceptually separate, which improves research hygiene and reduces emotional attachment to a favorite strategy. **Key risks / limitations** - Domain risk: Outputs may influence real-money trading decisions. The skill focuses on methodology and robustness but does not replace professional financial advice, risk management, or regulatory/compliance review. - Scope limitation: Oriented toward fully systematic, rule-based strategies on historical price/volume-type data; less applicable to discretionary, news-driven, or deeply fundamental approaches that require human context. - Tool-agnostic: It describes *what* to test rather than providing platform-specific code or implementation details; users must translate the methodology into their chosen backtesting software. **Recommended usage scenarios** - You are designing or refining a rule-based trading system and want to ensure the backtest is realistic and robust. - You have a promising backtest and want to “try to kill it” via pessimistic slippage, parameter sweeps, and regime analysis before risking capital. - You are diagnosing suspiciously good results or inconsistent performance and need a structured checklist for common backtest failure modes. - You are learning or teaching systematic trading research practices and want a concise, professional-grade framework for backtesting discipline.