Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study explores the effectiveness of various AI-driven workflows in addressing an open problem from the "Stable Menus of Public Goods" paper, focusing on the impact of human intuition in prompts, the benefits of multi-turn interactions, and the comparative performance of a large language model (LLM) versus a first-year PhD student. The findings reveal that incorporating human intuition into prompts enhances the LLM's performance and that multi-turn interactions can foster more ambitious outcomes. However, the LLM was found to be slightly less effective than the first-year PhD student in this context, highlighting the nuanced role of human expertise in AI-assisted research workflows.
Human intuition in prompts boosts LLM performance, but a first-year PhD student still outperforms the AI in complex reasoning tasks.
Using an open problem from the EC 2025 paper "Stable Menus of Public Goods" as a testbed, we conduct experiments to understand the effectiveness of different AI-for-EconCS research workflows. Specifically, we study three questions: Does providing human intuition in the prompt help? Does automated multi-turn interaction help? And, does an LLM outperform a first-year PhD student? Regarding the first two questions, we provide evidence for the following workflow suggestions: (1) prompting with human intuition can encourage the LLM to have better "taste", (2) multi-turn workflows help when the pipeline encourages "ambitious" steps. Regarding the third question, using an unpublished manuscript written by the paper's senior authors prior to collaborating with the first-year PhD student, we compare the effectiveness of the LLM with that of the first-year PhD student, and find that the LLM is slightly less effective.