Search papers, labs, and topics across Lattice.
This paper introduces a modular neuro-symbolic system for syllogistic reasoning, combining LLMs for parsing and machine translation with a symbolic theorem prover. The system translates natural language syllogisms into first-order logic, uses a theorem prover to determine validity, and optionally incorporates machine translation and symbolic retrieval. Results on SemEval-2026 Task 11 show competitive accuracy and reduced content effects compared to zero-shot LLMs, although multilingual performance is limited by the small LLM size.
Small LLMs paired with symbolic solvers can outperform larger zero-shot LLMs on formal reasoning tasks, but still struggle with multilingual inputs.
This paper describes our system submitted to SemEval-2026 Task 11: Disentangling Content and Formal Reasoning in Large Language Models. We present an efficient modular neuro-symbolic approach, combining a symbolic prover with small reasoning LLMs (4B parameters). The system consists of an LLM-based parser that translates natural language syllogisms to a first-order logic (FOL) representation, an automated theorem prover, and two optional modules: machine translation for multilingual inputs and a symbolic retrieval component for the identification of relevant premises. The system achieves competitive accuracy and relatively low content effect on most subtasks. Our ablations show that this approach outperforms LLM-based zero-shot baselines in this parameter size range, but also reveal limited multilingual capabilities of small LLMs. Finally, we include a discussion of the task's main ranking metric and analyze its limitations.
