Search papers, labs, and topics across Lattice.
The paper addresses the problem of planner bias in goal recognition datasets, which are typically generated using heuristic-based forward search and thus lack diversity in plans. To mitigate this bias, they propose a method using top-k planning to generate multiple diverse plans for each goal, creating more challenging benchmarks. They introduce a new metric, Version Coverage Score (VCS), to evaluate the resilience of goal recognizers to different plan variations, demonstrating that state-of-the-art recognizers' performance degrades significantly under low observability when evaluated on the new dataset.
Current goal recognition benchmarks are too easy: state-of-the-art methods fall apart when faced with diverse plans generated by different planners, revealing a critical lack of robustness.
Autonomous agents require some form of goal and plan recognition to interact in multiagent settings. Unfortunately, all existing goal recognition datasets suffer from a systematical bias induced by the planning systems that generated them, namely heuristic-based forward search. This means that existing datasets lack enough challenge for more realistic scenarios (e.g., agents using different planners), which impacts the evaluation of goal recognisers with respect to using different planners for the same goal. In this paper, we propose a new method that uses top-k planning to generate multiple, different, plans for the same goal hypothesis, yielding benchmarks that mitigate the bias found in the current dataset. This allows us to introduce a new metric called Version Coverage Score (VCS) to measure the resilience of the goal recogniser when inferring a goal based on different sets of plans. Our results show that the resilience of the current state-of-the-art goal recogniser degrades substantially under low observability settings.
