Search papers, labs, and topics across Lattice.
This paper introduces MBA-Bench, a pioneering multimodal benchmark designed for training and evaluating business ideation agents, leveraging large language models (LLMs) to incorporate visual cues alongside text. By generating business ideas through a combination of automatic image captioning and evidence-augmented synthesis using GPT-4o, the authors demonstrate significant performance improvements over existing text-only approaches. The newly developed agents, trained with innovative reward objectives, achieve up to 77.1% better outcomes compared to traditional multimodal baselines, highlighting the importance of multimodal inputs in real-world business contexts.
MBA-Bench reveals that integrating visual cues into business ideation agents can boost performance by over 77% compared to text-only methods.
Agentic systems powered by large language models (LLMs) have opened new opportunities for business ideation. Yet existing approaches remain confined to a text-only paradigm, despite the inherently multimodal nature of real-world contexts. We thus introduce MBA-Bench, the first multimodal benchmark for training and evaluating business ideation agents, comprising 30K samples across six domains, each domain characterized by distinct visual cues not fully conveyed by text alone. Concretely, we automatically caption images and employ GPT-4o to generate five reference ideas for each of three business questions through retrieval query generation, market evidence retrieval, and evidence-augmented synthesis. Following prior work, we evaluate agents across six business-oriented criteria using MLLM-as-a-Judge. To consider settings where criteria are hidden or disclosed, we present MBA-b and MBA-k for blind and known, respectively. We train both with two novel reward objectives---creativity and feasibility---while MBA-k further optimizes the six disclosed criteria for eight in total. Both are trained via LoRA-based supervised fine-tuning followed by group relative policy optimization with these setting-specific rewards. For extensive experiments on MBA-Bench, we set up two baselines accommodating either captions only or multimodal inputs, with the latter nearing closed-source performance on several metrics. MBA-b and MBA-k outperform caption baselines by 63.9% and 77.1%, and multimodal baselines by 25.6% and 35.8%, respectively.
