Search papers, labs, and topics across Lattice.
This paper introduces a hybrid task planning framework that combines Large Language Models (LLMs) for failure prediction with Relational Dynamic Influence Diagram Language (RDDL) for probabilistic sequential decision-making. The framework anticipates potential task failures stemming from human limitations or missing objects by leveraging the LLM's predictive capabilities. By integrating these predictions into an RDDL-based planner, the robot proactively executes actions to prevent or recover from failures, leading to improved task completion.
By blending LLM-based prediction with probabilistic planning, robots can now proactively anticipate and mitigate human-related task failures, leading to more robust human-robot collaboration.
Anticipating and adapting to failures is a key capability robots need to collaborate effectively with humans in complex domains. This continues to be a challenge despite the impressive performance of state of the art AI planning systems and Large Language Models (LLMs) because of the uncertainty associated with the tasks and their outcomes. Toward addressing this challenge, we present a hybrid framework that integrates the generic prediction capabilities of an LLM with the probabilistic sequential decision-making capability of Relational Dynamic Influence Diagram Language. For any given task, the robot reasons about the task and the capabilities of the human attempting to complete it; predicts potential failures due to lack of ability (in the human) or lack of relevant domain objects; and executes actions to prevent such failures or recover from them. Experimental evaluation in the VirtualHome 3D simulation environment demonstrates substantial improvement in performance compared with state of the art baselines.
