Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces LeCropFollow, a novel visual navigation framework designed for agricultural robots operating in unstructured crop fields, which often present challenges due to irregular planting and discontinuities. By leveraging a learned latent representation and integrating a self-supervised semantic heatmap extractor with a Model-Based Reinforcement Learning (MBRL) planner, the system optimizes navigation trajectories while preserving critical semantic context. Field experiments demonstrate that LeCropFollow significantly reduces semantic failures in complex environments, achieving a 2.4x improvement over traditional keypoint-based methods, thereby enabling effective zero-shot transfer from simulation to real-world applications.
Latent planning in LeCropFollow reduces semantic failures by 2.4x in challenging agricultural environments, outperforming traditional geometric approaches.
Unstructured navigational features, such as irregular planting or discontinuities, remain the primary failure mode for under-canopy agricultural robots. Existing geometric approaches often fail in these scenarios because they compress high-dimensional visual data into deterministic spatial references, effectively discarding the uncertainty and semantic context required to navigate ambiguous terrain. To address this, we present LeCropFollow, a visual navigation framework that bypasses explicit geometric modeling in favor of a learned latent representation. By integrating a self-supervised semantic heatmap extractor with TD-MPC2, a Model-Based Reinforcement Learning (MBRL) planner, our system optimizes trajectories directly within a latent manifold. The framework operates over the uncompressed heatmap signal, preserving the semantic context that geometric reductions discard. We demonstrate that this representational shift enables zero-shot transfer from simplified simulation to the physical world without fine-tuning. Extensive field experiments in late-stage corn fields show that LeCropFollow matches state-of-the-art baselines in unstructured rows but significantly outperforms them in plantation gaps, achieving a 2.4x reduction in semantic failures compared to keypoint-based methods. These results suggest that latent planning offers a robust alternative to geometric estimation for operations in heterogeneous agricultural environments. Code, models, and data available: https://felipe-tommaselli.github.io/lecropfollow .