Search papers, labs, and topics across Lattice.
This paper introduces DocFormBench, a benchmark designed to evaluate document formatting in a content-aware manner, addressing the challenges posed by the lack of dedicated datasets. It also presents DocFormFlow, a novel workflow that separates the processes of identifying formatting targets from executing modifications, leading to improved accuracy and efficiency. Experimental results demonstrate that DocFormFlow enhances formatting performance across various large language models while minimizing token usage, highlighting the importance of precise target localization in achieving better outcomes.
DocFormFlow boosts document formatting accuracy while slashing token consumption, revealing that target localization is the key to success.
Recent advances in large language models (LLMs) have opened up new possibilities for automated document formatting. However, real-world formatting often requires identifying targets based on document content. This content-aware setting remains challenging and underexplored, primarily due to the lack of dedicated evaluation datasets.To enable evaluation in realistic content-aware scenarios, we introduce DocFormBench, a benchmark that extends Text-to-Format evaluation to diverse formatting requirements, along with metrics for both accuracy and efficiency.To mitigate redundant document reading in existing methods during formatting, we propose DocFormFlow, a workflow formatting method that decouples target localization from modification execution into what to format and how. Extensive experiments across multiple LLMs and multimodal models show that DocFormFlow consistently improves formatting accuracy while reducing token consumption compared to representative baselines. Further analysis reveals that precise target localization is the primary factor influencing formatting performance. We hope DocFormBench and DocFormFlow will facilitate future research toward more intelligent and reliable document formatting.
