Search papers, labs, and topics across Lattice.
ATGBuilder introduces a feature-assisted graph-learning framework for constructing Activity Transition Graphs (ATGs) using seed supervision, addressing the limitations of both static and dynamic analysis methods. By leveraging a Large Language Model (LLM) to generate concise textual summaries of UI activity metadata and incorporating widget-trigger information as edge attributes, ATGBuilder enhances the accuracy and completeness of ATG construction. Experimental results reveal that ATGBuilder significantly outperforms existing state-of-the-art approaches, leading to improved performance in automated GUI exploration tools.
ATGBuilder transforms how we construct Activity Transition Graphs by integrating LLM-generated UI summaries and widget-trigger data, achieving superior accuracy in navigation modeling.
Android applications are organized around activities that provide visual Graphical User Interface (GUI) containers that host the UI and handle user interaction events. Activity Transition Graphs (ATGs) have been widely used to model apps' GUI navigation. However, the construction of high-quality ATGs is challenging: ATGs based on static analysis may miss acceptable transitions and may extract infeasible ones; while dynamically explored ATGs can yield incomplete transitions. Recent learning-based approaches can treat ATG construction as a seed-supervised link-prediction task. However, the use of activity-layout and widget-trigger information for ATG construction remains limited. We propose ATGBuilder, a feature-assisted graph-learning approach for seed-supervised ATG construction. ATGBuilder uses a Large Language Model (LLM) to summarize UI activity metadata from layouts into compact textual functionality summaries. ATGBuilder explicitly models widget-trigger information into the edge attribute: It then uses an auxiliary widget-attribute reconstruction objective on this information during model training. ATGBuilder's performance was evaluated across a series of ablations on the frontmatter corpus, and an experiment on benchmark using manually-checked ground-truth ATGs. Experiments on multiple benchmarks show that ATGBuilder significantly outperforms state-of-the-art methods. We further demonstrate its effectiveness by improving automated GUI exploration tools through better navigation guidance.
