Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
To resolve the structural information loss caused by standard table linearization in LLMs, DeepTable injects hierarchical header tree paths and learnable 2D attention biases directly into parameter-efficient fine-tuning architectures. By combining Tree Path Encoding (TPE) of multi-level header ancestry with Structural Attention Biases (SAB) on attention logits, the model explicitly preserves row-column coordinate relationships and nested header dependencies. Evaluated across three LLM backbones, the method substantially outperforms TableLoRA baselines on table QA tasks, achieving average improvements of 7.42 points on the hierarchical HiTab benchmark and 3.23 points on WikiTQ.
Standard token linearization destroys hierarchical tabular reasoning, but encoding multi-level header tree paths directly into attention logits drives an average 7.4-point gain on complex tables using only parameter-efficient tuning.
Large language models (LLMs) have demonstrated strong performance in table understanding. However, they typically process table content and headers as linearized token sequences. This representation weakens the two-dimensional and hierarchical structural relationships encoded by multi-level row and column headers. Existing parameter-efficient fine-tuning methods incorporate basic row and column information but do not explicitly capture the rich structural dependencies induced by hierarchical table headers. We propose DeepTable, a structure-aware approach for table understanding with LLMs. DeepTable comprises two complementary components. Structural Attention Bias (SAB) introduces learnable biases into the attention logits to explicitly represent whether pairs of table tokens share the same row or column. Tree Path Encoding (TPE) represents each table token using the ancestor paths of its row and column headers, preserving its position within the multi-level table structure. We integrate DeepTable with TableLoRA (He et al., 2025) to inject structural information into parameter-efficient adaptation. Across three LLM backbones, DeepTable consistently improves the corresponding TableLoRA baselines on three table question answering benchmarks, achieving average gains of 7.42 points on HiTab, 3.23 points on WikiTQ, and 2.01 BLEU points on FeTaQA. These results demonstrate the effectiveness of the proposed structural biases across different LLM backbones.