Search papers, labs, and topics across Lattice.
This study investigates the testing methodologies for Terminal User Interfaces (TUIs) by surveying 197 real-world applications and discovering that only 12% of test code actively exercises the interface. The authors create a headless benchmark for TUIs across multiple programming languages and compare the performance of four leading LLMs against a random exploration baseline. Key findings reveal that while LLMs are more efficient in reaching input-gated faults, random exploration proves to be a strong competitor, emphasizing the need for reliable baselines in automated TUI testing.
Random exploration outperforms LLMs in TUI testing, revealing that model choice may not be as critical as previously thought.
Terminal User Interfaces (TUIs) combine the stateful, screen-oriented behaviour of GUIs with terminal deployment and are now common in developer tools. Yet they lack a dedicated testing methodology. We survey 197 real-world TUI applications: only 12% of test code exercises the interface, and 45% of those tests never send input, checking a static frame instead. We turn these applications into a headless benchmark spanning ratatui/Rust, bubbletea/Go, textual/Python, and ink/TypeScript, packaging each as an instrumented Docker image. We record line and widget coverage where reliable, rendered terminal states, and crashes. Under equal wall-clock budgets, we compare four frontier LLMs with random exploration. No model dominates. Random is a strong time-budgeted baseline, but its crash advantage comes from higher throughput: per interaction, LLM guidance is more efficient and uniquely reaches input-gated faults. Automatically deriving launch inputs yields the largest practical gain, enabling applications that otherwise never start. Line coverage poorly predicts crash discovery, weakening it as a proxy for test effectiveness. Automated TUI testing is feasible but far from solved, and honest baselines matter more than model choice. We release the coverage tool tuicov at https://github.com/tui-testing/tuicov and the testing framework tuibot at https://github.com/tui-testing/tuibot.
