Search papers, labs, and topics across Lattice.
This paper introduces BTS-AgentBench, a novel pipeline that transforms read-only telemetry logs into executable multi-turn agent tasks by normalizing metadata and compiling static tasks into structured episodes. The method ensures high fidelity in reproducing tasks while maintaining the integrity of the original telemetry data, as evidenced by the successful reproduction of all logical tool-store exports and the completion of tasks by a retained GPT-5.5 execution. The release includes a comprehensive set of 532 rows with enhancements for clarity, goal revision, and quality reporting, demonstrating the robustness of the construction approach.
BTS-AgentBench enables the seamless transformation of telemetry logs into structured agent tasks, achieving perfect reproducibility in task execution.
Industrial sites contain large volumes of read-only telemetry, but few benchmarks specify how to compile these records into executable multi-turn agent tasks. We present a telemetry-to-episode construction method instantiated as BTS-AgentBench. The pipeline normalizes BTS metadata and raw histories into a read-only tool store, compiles static tasks with tool-derived gold answers and evidence, and lifts retained tasks into typed, bounded operator-facing episodes. The 532-row release adds clarification, goal revision, timestamp policy, quality-gated reporting, and evidence attribution while preserving the source computation and split. Coded contract preflight reports zero findings, and the construction-exclusion controller completes 0/532 rows. Two independent raw-to-episode builds match all 11 logical tool-store exports and reproduce the released 356/87/89 train/dev/test artifact exactly. Applying the shared construction path to XAI4HEAT produces 204 episodes; on its 41-row held-out test split, the controller completes 0 rows and the retained GPT-5.5 execution completes all 41. Code, artifacts, and replay reports are available at https://github.com/kjy7567/BTS-AgentBench.
