Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The study introduces Engine-Transfer-Bench (ETB), a comprehensive benchmark for evaluating document compilation engines across multiple tasks, including reliability, latency, text consistency, and failures. By analyzing 1,784 open documents and conducting over 4,200 compiles across different operating systems, the results reveal that Tectonic consistently outperforms traditional TeX Live-style engines, which exhibit significant variability in success rates depending on the distribution policy. The findings emphasize that while all engines succeed with portable LaTeX documents, latency emerges as the critical factor for engine selection, highlighting architectural failures related to fonts and layout rather than missing packages.
Tectonic's reliability outshines traditional engines, revealing a stark 12-20 percentage point variability in success rates based on distribution policies.
There is no shared framework for selecting among document compilation engines (pdfLaTeX, XeLaTeX, LuaLaTeX, Tectonic, Typst, and pandoc PDF backends). We present Engine-Transfer-Bench (ETB): 1,784 open documents, four tasks covering reliability, latency, text consistency, and failures, a pinned harness, and host-tagged multi-OS results. On GitHub Actions (N=4,211 compiles per host across macOS, Ubuntu, and Windows), Tectonic success is stable within 0.9 percentage points (96.3-97.2%), whereas classic TeX Live-style engines vary by 12-20 percentage points according to distribution policy (Ubuntu apt, MiKTeX auto-install, or macOS BasicTeX). On 702 portable LaTeX documents, the tested engines succeed at 100%, making latency the primary selection factor; failures concentrate in 107 engine-specific templates. Within ETB, failures are architectural, involving fonts, layout, and assets, rather than missing packages on a provisioned host. A 50-pair validation of the S_pdf text-consistency metric achieves 94% precision for real content divergence. We release ETB, ETB-Porta, a recommender and portability gate, and a public cross-OS harness as shared infrastructure.