Search papers, labs, and topics across Lattice.
This study investigates the challenges of summarizing literary works by analyzing a dataset that maps sentences from 150 novel summaries to their corresponding source chapters, utilizing both human and LLM-based annotations. The findings reveal that both annotators struggle with maintaining linearity and uniformity in summaries, leading to significant deviations from the source material. By quantifying these deviations, the research highlights critical differences in narrative expression between original texts and their summaries, shedding light on the complexities of literary representation.
Summaries of literary works often distort the original narrative, with human and LLM annotators both failing to maintain linearity and uniformity in their representations.
Works of literature are complicated; they balance plot, suspense, surprise, and artistic expression. Summaries of literature prioritize plot, and therefore may deviate from their sources. Using a combination of manual and LLM-based annotation, we construct a dataset mapping sentences from 150 novel summaries to their respective source chapters. We find the task unexpectedly difficult for both human and model annotators. Using the sentence-to-chapter mappings, we then measure summary linearity, the degree to which it maintains the source's order of events, and uniformity, the degree to which a summary spreads attention equally across a source. By examining when and how summaries break linearity and uniformity, we identify differences in how literary works and summaries express plot, particularly with regard to the clarity and prominence with which narrative details are described.
