Search papers, labs, and topics across Lattice.
The paper introduces ReSS, a framework that combines symbolic and neural reasoning for tabular data prediction by using decision trees to generate instance-level decision paths as symbolic scaffolds for LLM fine-tuning. This approach guides LLMs to produce grounded natural-language reasoning that adheres to the underlying decision logic, addressing limitations of both symbolic models and general-purpose LLMs. Experiments on medical and financial benchmarks show that ReSS-trained models improve accuracy by up to 10% compared to decision trees and standard fine-tuning, while also producing more faithful and consistent reasoning.
Achieve 10% accuracy gains in tabular data prediction, not by brute-forcing bigger models, but by cleverly scaffolding LLM reasoning with interpretable decision trees.
Tabular data remains prevalent in high-stakes domains such as healthcare and finance, where predictive models are expected to provide both high accuracy and faithful, human-understandable reasoning. While symbolic models offer verifiable logic, they lack semantic expressiveness. Meanwhile, general-purpose LLMs often require specialized fine-tuning to master domain-specific tabular reasoning. To address the dual challenges of scalable data curation and reasoning consistency, we propose ReSS, a systematic framework that bridges symbolic and neural reasoning models. ReSS leverages a decision-tree model to extract instance-level decision paths as symbolic scaffolds. These scaffolds, alongside input features and labels, guide an LLM to generate grounded natural-language reasoning that strictly adheres to the underlying decision logic. The resulting high-quality dataset is used to fine-tune a pretrained LLM into a specialized tabular reasoning model, further enhanced by a scaffold-invariant data augmentation strategy to improve generalization and explainability. To rigorously assess faithfulness, we introduce quantitative metrics including hallucination rate, explanation necessity, and explanation sufficiency. Experimental results on medical and financial benchmarks demonstrate that ReSS-trained models improve traditional decision trees and standard fine-tuning approaches up to $10\%$ while producing faithful and consistent reasoning
