SSFF: Investigating LLM Predictive Capabilities for Startup Success through a Multi-Agent Framework with Enhanced Explainability and Performance

2024-05-29

AI Summary

Existing startup success prediction faces three key challenges: lack of standardized benchmarks, overreliance on founders’ subjective self-reports, and severe class imbalance. To address these, we propose SSFF—a multi-agent framework that emulates venture capital analysts’ decision-making by integrating random forests, feedforward neural networks, and retrieval-augmented generation (RAG) into three core modules: prediction, interpretive analysis, and external knowledge integration. Our work introduces the first fine-grained attribution mechanism based on founder capability levels (L1–L5) and empirically uncovers systematic LLM biases in interpreting founders’ statements—enabling simultaneous gains in both interpretability and accuracy. Experiments demonstrate that SSFF achieves 108.3% and 30.8% higher accuracy than GPT-4o mini and GPT-4o, respectively. Moreover, teams rated at L5 exhibit a 3.79× higher probability of success compared to L1 teams.

Technology Category

• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Machine Learning: Other Foundations of Machine Learning
• Multiagent Systems: Other Foundations of Multi Agent Systems

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

LLM based agents have recently demonstrated strong potential in automating complex tasks, yet accurately predicting startup success remains an open challenge with few benchmarks and tailored frameworks. To address these limitations, we propose the Startup Success Forecasting Framework, an autonomous system that emulates the reasoning of venture capital analysts through a multi agent collaboration model. Our framework integrates traditional machine learning methods such as random forests and neural networks within a retrieval augmented generation framework composed of three interconnected modules: a prediction block, an analysis block, and an external knowledge block. We evaluate our framework and identify three main findings. First, by leveraging founder segmentation, startups led by L5 founders are 3.79 times more likely to succeed than those led by L1 founders. Second, baseline large language models consistently overpredict startup success and struggle under realistic class imbalances largely due to overreliance on founder claims. Third, our framework significantly enhances prediction accuracy, yielding a 108.3 percent relative improvement over GPT 4o mini and a 30.8 percent relative improvement over GPT 4o. These results demonstrate the value of a multi agent approach combined with discriminative machine learning in mitigating the limitations of standard large language model based prediction methods.

Problem

• Predicting startup success accurately with LLM-based multi-agent framework
• Overcoming LLM overprediction and class imbalance issues in startup analysis
• Integrating traditional ML methods with LLMs for enhanced explainability

Innovation

• Multi-agent framework enhances startup success prediction
• Integrates random forests and neural networks
• Retrieval augmented generation with three modules