MBA: Multimodal Benchmark and Agents for Real-World Business Ideation

2026-08-11

AI Summary

This work addresses the limitation of existing commercial idea generation methods, which rely solely on text and fail to leverage multimodal information from real-world scenarios. To bridge this gap, we introduce MBA-Bench, the first multimodal benchmark for real-world business ideation, featuring two evaluation settings: blind assessment (MBA-b) and criterion-aware assessment (MBA-k). We develop a vision-language agent that integrates retrieval augmentation, multi-criterion optimization, and novel reward objectives—such as creativity and feasibility—through techniques including image captioning, GPT-4o–based reference generation, MLLM-as-a-Judge evaluation, LoRA fine-tuning, and grouped relative policy optimization. Experiments demonstrate that our approach outperforms text-only baselines by 63.9% and 77.1% on MBA-b and MBA-k, respectively, and surpasses multimodal baselines by 25.6% and 35.8%, achieving performance close to that of closed-source models across multiple metrics.

Abstract

Agentic systems powered by large language models (LLMs) have opened new opportunities for business ideation. Yet existing approaches remain confined to a text-only paradigm, despite the inherently multimodal nature of real-world contexts. We thus introduce MBA-Bench, the first multimodal benchmark for training and evaluating business ideation agents, comprising 30K samples across six domains, each domain characterized by distinct visual cues not fully conveyed by text alone. Concretely, we automatically caption images and employ GPT-4o to generate five reference ideas for each of three business questions through retrieval query generation, market evidence retrieval, and evidence-augmented synthesis. Following prior work, we evaluate agents across six business-oriented criteria using MLLM-as-a-Judge. To consider settings where criteria are hidden or disclosed, we present MBA-b and MBA-k for blind and known, respectively. We train both with two novel reward objectives---creativity and feasibility---while MBA-k further optimizes the six disclosed criteria for eight in total. Both are trained via LoRA-based supervised fine-tuning followed by group relative policy optimization with these setting-specific rewards. For extensive experiments on MBA-Bench, we set up two baselines accommodating either captions only or multimodal inputs, with the latter nearing closed-source performance on several metrics. MBA-b and MBA-k outperform caption baselines by 63.9% and 77.1%, and multimodal baselines by 25.6% and 35.8%, respectively.

Problem

• multimodal
• business ideation
• agentic systems
• real-world contexts
• visual cues

Innovation

• multimodal benchmark
• business ideation agents
• creativity and feasibility rewards
• MLLM-as-a-Judge
• group relative policy optimization