Search papers, labs, and topics across Lattice.
This study investigates cross-domain data-to-text (D2T) generation without relying on in-domain training data, addressing the limitations of existing methods that depend on specific tasks or zero-shot capabilities of large language models. By employing data-driven knowledge distillation (DDKD) and structure-preserving augmentation techniques, the authors demonstrate that DDKD consistently outperforms both fine-tuning on out-of-domain data and zero-shot inference across five benchmarks. Notably, DDKD achieves superior performance with smaller models compared to larger fine-tuned counterparts, highlighting the effectiveness of their approach in diverse D2T tasks.
DDKD outperforms larger fine-tuned models in cross-domain data-to-text generation, revealing that size isn't everything in model performance.
Structured data exists in many forms (tables, knowledge graphs, charts, and time series), and converting it into text may involve different generation tasks. However, most prior work on data-to-text (D2T) generation has focused on specific tasks and datasets, relying either on task-specific training data or on the zero-shot capabilities of large language models. We study cross-domain D2T generation in a setting where neither in-domain training text nor test references are available, and where domains, generation goals, and input structures vary substantially. We compare data-driven knowledge distillation (DDKD) against zero-shot inference and fine-tuning on out-of-domain D2T data, and introduce structure-preserving augmentation via structural subsampling and perturbation. Experiments on five benchmarks show that, at constant model size (1.7B parameters), DDKD consistently outperforms both fine-tuning and zero-shot inference. Moreover, the resulting small models outperform a much larger finetuned model on two of the five domains, achieving comparable performance on the remaining three. We further construct QUINTD-5, a fivefold extension of QUINTD-1, and show that simply scaling real target-domain inputs yields only modest gains, whereas our augmentation strategy remains more effective and more cost-efficient for cross-domain distillation.
