strategy backtesting

This study addresses the long-standing lack of systematic measurement of “implementation risk” in quantitative investment backtesting—the performance discrepancies arising from differences in backtesting engine implementations. The work formally defines this risk for the first time and proposes four metrological metrics alongside a taxonomy of five failure modes. These are derived from parallel execution of 15 benchmark strategies across five open-source backtesting engines, incorporating transaction cost modeling, non-overlapping stratified asset buckets, and source code defect analysis. Experiments reveal that while engine outputs converge under zero-cost assumptions, performance divergence can reach up to 3.71% when transaction costs are introduced. Crucially, however, the relative ranking of strategy efficacy remains unchanged across engines (conclusion stability index = 1), indicating that implementation risk affects performance attribution but does not alter investment decisions.

Traditional mean-variance portfolio optimization suffers from strong reliance on return normality and stationary covariance structures, resulting in poor cross-market robustness. To address this, we propose a clustering-guided local Sharpe ratio optimization framework: assets are first partitioned via K-means clustering based on historical log-returns; subsequently, Sharpe ratio-maximizing weights are solved independently within each cluster. This approach circumvents global covariance estimation, enhancing interpretability and environmental adaptability. We evaluate the method using rolling-window backtesting across multi-asset classes—equities, bonds, and commodities—and demonstrate a 23.6% improvement in annualized Sharpe ratio and a 17.4% reduction in maximum drawdown relative to an equal-weight benchmark. Our key contribution is the introduction of a novel “clustering–local optimization” hierarchical paradigm, offering a principled pathway for constructing robust portfolios under non-normal, time-varying market conditions.

This study addresses the widespread reliance by institutional investors on marketing-driven backtests to evaluate structured investment strategies, despite concerns about their out-of-sample validity. Leveraging a global dataset of 1,726 structured products issued by institutional firms, the paper systematically examines the translatability of backtested performance into live results through peer benchmarking, macro-factor regime identification, and direct comparison between backtested and realized returns. The findings reveal that backtested returns are primarily driven by common factor conditions prevailing prior to strategy inception rather than genuine strategy-specific skill. Notably, products launched following periods of strong factor performance exhibit significantly degraded out-of-sample results. Building on these insights, the study proposes a novel evaluation framework that dynamically adjusts the credibility assigned to backtests based on the extremity of prevailing factor conditions at product launch, offering a more prudent approach to assessing structured strategies.

Financial regulators face challenges in backtesting risk measures due to the lack of a unified, robust, and manipulation-resistant framework. This paper introduces the first e-value- and e-process-based standard for risk measure evaluation and comparative backtesting, enabling model-free, nonparametric, dynamic validation of widely used measures—including Value-at-Risk (VaR) and Expected Shortfall (ES). By pioneering the application of e-processes to financial backtesting, it achieves unified treatment of both identifiable and incentive-compatible risk measures, bridging statistical rigor with regulatory practicality. The proposed method is inherently resistant to p-hacking, requires no distributional assumptions, and substantially enhances backtest robustness. Extensive experiments on simulated and real-world financial datasets demonstrate its statistical validity, computational feasibility, and direct applicability to regulatory practice.

This work systematically evaluates large language models’ (LLMs) code generation capability for introductory quantitative trading tasks, focusing on end-to-end translation from natural-language strategy descriptions to executable backtesting code. To this end, we introduce MARKET-BENCH—the first benchmark tailored for beginners—covering three canonical strategies: time-based trading, pairs trading, and delta hedging. We propose a novel multi-round pass@k metric that decouples execution success from numerical accuracy, quantifying fidelity across profit-and-loss (P&L), maximum drawdown, and position trajectory error—thereby establishing a verifiable, reproducible evaluation paradigm for financial engineering practice. Experiments span 12 state-of-the-art models: GPT-5.1 Codex-Max achieves 100% pass@1 on the first two strategy classes; Qwen3 Max attains 100% pass@3. However, all models exhibit significant numerical inaccuracies in risk modeling and inventory dynamics reasoning.

This study addresses the challenges in large language model (LLM)-driven automated quantitative research, which are hindered by high technical barriers and the absence of standardized benchmarks for backtesting. To bridge this gap, we introduce BacktestBench, the first large-scale benchmark for automated quantitative backtesting, comprising four task categories and 18,246 annotated samples. We further propose AutoBacktest, a multi-agent system that integrates a Summarizer, Retriever, and Coder to enable end-to-end generation of reproducible backtesting code from natural language strategy descriptions. Leveraging real-market data, SQL queries, and a Python-based backtesting engine, we conduct a systematic evaluation across 23 mainstream LLMs, identifying key factors influencing performance and demonstrating the feasibility and effectiveness of LLM-powered backtesting automation.

This study addresses the low automation and insufficient interpretability in quantitative multi-factor research by proposing an end-to-end framework based on multi-agent collaboration. The framework uniquely integrates reflective planning, chained tool invocation, and an MCP (Model-Controller-Provider) server architecture, incorporating a panel data analysis toolkit to emulate real-world research team workflows. It supports factor replication, novel signal development, and standardized report generation. Departing from purely dynamic code generation, the approach leverages traceable computational chains and narrative-driven analysis to significantly enhance research efficiency, result transparency, and empirical performance on large-scale panel datasets.

Traditional quantitative investment systems typically optimize a single metric—such as the information ratio—and thus struggle to meet professional investors’ multifaceted objectives, including pure alpha generation, style control, drawdown resilience, and turnover and capacity constraints. This work proposes an Objective-Oriented Quantitative Investment (OOQI) framework that formally encodes investment intent as strategy specifications and compiles them into composable, constraint-satisfying strategy assemblies. Key innovations include establishing a dual lattice structure between specifications and assemblies, designing a satisfaction-driven synthesis mechanism, and introducing rolling recertification via e-process-based validation. Empirical results demonstrate that the specification-driven approach satisfies 100% of target constraints across 32 strategies, at the cost of only a 5.5% reduction in information ratio, whereas conventional outcome-oriented methods—despite higher in-sample information ratios—fulfill merely 25% of the specified requirements.

This work addresses the limitations of existing financial agent evaluation frameworks, which often rely on static benchmarks or focus solely on final returns, thereby lacking traceability of decision-making processes and hindering fine-grained, fair performance assessment in dynamic markets. To overcome these challenges, the authors propose a unified performance tracking platform for financial agents that, for the first time, enables persistent logging of the complete decision trajectory—from market observation to trade execution. By integrating a time-consistent market data interface, a multi-agent collaborative architecture, and an end-to-end logging system, the platform supports interactive, cross-market and cross-model attribution analysis. Deployed across Hong Kong, U.S., and A-share markets, the system—augmented with a visual Trading Arena interface—significantly enhances the transparency, interpretability, and diagnostic capability of agent evaluation.

Existing evaluation methods struggle to comprehensively assess large language models’ (LLMs’) ability to translate natural language trading strategies into executable code, particularly lacking multidimensional metrics for system auditability, rule drift, and robustness. This work proposes SysTradeBench—an iterative build-test-repair benchmark that introduces, for the first time, drift-aware diagnostics and a multidimensional scoring framework encompassing specification fidelity, risk discipline, reliability, and out-of-sample robustness. The framework integrates a sandboxed testing environment, rule drift detection algorithms, evidence-bundle feedback mechanisms, and a constrained repair process under frozen semantic constraints, emphasizing human-AI collaborative governance. Experiments across 17 models and 12 strategies demonstrate that top-performing models achieve over 91.7% effectiveness, with code convergence typically attained by the second iteration, highlighting LLMs’ potential in rapid prototyping and shallow repairs while underscoring the necessity of human oversight for critical strategies.