Search papers, labs, and topics across Lattice.
This paper investigates the role of memory and planning in spatial navigation within dynamic, uncertain environments using a foraging task where the agent must navigate to food through changing barriers with limited sensing. The authors compare various strategies, ranging from simple memory-less approaches to sophisticated architectures that incorporate memory and learning, to determine their effectiveness under different conditions. The key finding is that an agent employing non-stationary probability learning to update episodic memories, build maps, and plan on the fly outperforms simpler agents, especially as task difficulty increases, provided the uncertainty from localization and environmental change remains within reasonable bounds.
Agents that dynamically update episodic memories and plan on imperfect maps significantly outperform simpler, memory-less agents in challenging, changing environments, but only up to a point of uncertainty.
We explore how different types and uses of memory can aid spatial navigation in changing uncertain environments. In the simple foraging task we study, every day, our agent has to find its way from its home, through barriers, to food. Moreover, the world is non-stationary: from day to day, the location of the barriers and food may change, and the agent's sensing such as its location information is uncertain and very limited. Any model construction, such as a map, and use, such as planning, needs to be robust against these challenges, and if any learning is to be useful, it needs to be adequately fast. We look at a range of strategies, from simple to sophisticated, with various uses of memory and learning. We find that an architecture that can incorporate multiple strategies is required to handle (sub)tasks of a different nature, in particular for exploration and search, when food location is not known, and for planning a good path to a remembered (likely) food location. An agent that utilizes non-stationary probability learning techniques to keep updating its (episodic) memories and that uses those memories to build maps and plan on the fly (imperfect maps, i.e. noisy and limited to the agent's experience) can be increasingly and substantially more efficient than the simpler (minimal-memory) agents, as the task difficulties such as distance to goal are raised, as long as the uncertainty, from localization and change, is not too large.
