A New Paradigm: AI Agents for Scientific Discoveries in Quantitative Finance

The era of brute-force optimization and hand-crafted heuristics is giving way to a new paradigm — one where Large Language Models (LLMs) act as intelligent evolutionary operators, autonomously exploring, modifying, and optimizing algorithms across scientific and financial domains. In this post, we dive into how LLM-powered agents are transforming algorithm discovery through evolutionary workflows and highlight three groundbreaking publications from ICLR 2025 in Singapore: AlphaSharpe, AlphaPortfolio, and AlphaQuant.

From Random Mutations to LLM-Directed

Traditional evolutionary algorithms — such as genetic programming or symbolic regression — rely on random mutations and crossover strategies to explore solution spaces. While powerful, these methods are limited by the lack of semantic understanding and often suffer from convergence issues or unexplainable outputs. AI agents are enabling a powerful new form of algorithmic evolution. Unlike traditional search-based optimization methods, LLMs bring together several advanced capabilities that make them ideal for driving intelligent, iterative improvement. They integrate domain knowledge from finance, statistics, and optimization, enabling context-aware reasoning. Their semantic understanding of code and mathematics allows them to identify patterns and shortcomings in existing algorithms. Through causal reasoning, they can propose meaningful improvements rather than random tweaks.

Moreover, their ability to automatically implement and debug code makes them not just ideators, but full-cycle developers and optimizers. As evolutionary operators, LLMs replace blind mutation and crossover strategies with intelligent, directed evolution. Rather than relying on randomness, they analyze existing code, metrics, or strategies, deeply understanding their structure and purpose. Then, they hypothesize potential improvements or adaptations, often based on best practices or latent patterns observed in training data. AI agents can generate modified code or new mathematical formulations, ensuring syntactic and semantic correctness. They are also capable of interpreting experimental results and visualizations, offering feedback for refinement. Finally, they can select and evolve the most promising outputs, continuously improving the solution space over time.

Reinventing Human-Made Financial Strategies

The financial industry relies on metrics and heuristics that have remained largely unchanged for decades. LLMs are now challenging this status quo by evolving new financial intelligence.

AlphaSharpe and AlphaPortfolio

The Sharpe Ratio is a widely used metric to assess the risk-adjusted return of investment strategies. However, empirical tests reveal that it correlates weakly (approximately 0.13) with actual future investment performance. AlphaSharpe, discovered using an LLM-driven evolutionary framework, generated new metrics that tripled this correlation to over 0.40. These newly evolved metrics offered significantly better asset ranking, more accurate strategy evaluation, and greater predictive confidence, especially in volatile market conditions. They were implemented in PyTorch, validated for robustness using cross-time validation, and selected through a quality-diversity optimization process—also known as illumination.

Portfolio allocation remains a fundamental yet challenging problem in quantitative finance. Surprisingly, even sophisticated, human-designed strategies often fail to outperform the uniform portfolio — a simple approach that has consistently shown resilience over decades. AlphaPortfolio set out to challenge this benchmark using the same LLM-driven evolutionary framework. It was prompted to design novel allocation strategies, which were then evaluated for robustness and generalization. These strategies were also tested in out-of-distribution market conditions to assess their adaptability. The evolved approaches outperformed the uniform portfolio by more than 2x, demonstrating that autonomous discoveries can not only rival but also exceed the performance of expert-designed methods.

AlphaQuant: Automated feature engineering

While predictive modeling in finance has advanced significantly through AutoML and deep learning, feature engineering remains a persistent bottleneck — demanding manual effort, domain expertise, and extensive validation. AlphaQuant addresses this challenge by automating the entire pipeline, using AI agents as autonomous inventors and evaluators of predictive features.

The system generates thousands of novel features using few-shot prompting combined with financial theory. Then it implements these features in PyTorch using clean, vectorized code optimized for efficiency. Each feature is scored based on its predictive importance (e.g., via SHAP values), its temporal robustness using time-series cross-validation, and its diversity to minimize redundancy and promote uncorrelated signals. Features are then ranked and evolved iteratively through an illumination strategy that favors both quality and variety.

The result is a continually expanding pool of high-performing, uncorrelated features, enabling stronger and more resilient models. This diversity helps reduce overfitting by aggregating a broader range of signals, and it leads to improved alpha generation for asset forecasting and ranking. Notably, even as more features are added, validation errors continue to decline and predictive correlation with future returns increases, clearly demonstrating continuous learning and systematic improvement over time.

Why This Matters

The fusion of LLMs with evolutionary search is redefining what’s possible in algorithm design. Rather than being limited to tweaking existing ideas, LLMs now have the capacity to create and evolve solutions from scratch. These ideas are not only instantly interpretable, but also readily implementable, accelerating the pace from concept to execution. Optimization becomes smarter, faster, and more autonomous, guided by reasoning rather than randomness. In the realm of quantitative finance, where even marginal improvements can yield substantial returns, this advancement represents far more than just an efficiency boost — it’s a fundamental paradigm shift in how algorithms are discovered and deployed.

The breakthroughs behind AlphaSharpe, AlphaPortfolio, and AlphaQuant are just the beginning. For more on our work with AI agents and algorithmic discovery, check out our research publications.