Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
This paper introduces ColoACT, an autonomous colon navigation system for a self-propelled endoscopic robot that integrates an Action Chunking Transformer policy. By augmenting RGB input with estimated relative depth and a gradient-based pseudo-elevation map, ColoACT enhances geometric cue saliency for improved fold-ridge detection. The system achieves high success rates in navigating various colon geometries, including 85.4% in straight segments and 72.5% in curved segments, demonstrating robust performance in complex anatomical structures.
A novel Action Chunking Transformer policy enables smooth, continuous autonomous navigation in the colon by fusing overlapping predicted action chunks, overcoming temporal inconsistencies common in existing methods.
Autonomous colonoscopic navigation can reduce operator burden and the risk of loop formation or tissue trauma, but remains challenging due to deformable anatomy, weak-texture and specular endoscopic visuals, and contact-rich viscoelastic interactions. Existing methods either rely on geometry-driven pipelines, which are efficient and interpretable yet brittle due to manually engineered features and switching logic, or adopt learning-based policies, whose inferred depth/geometry can become temporally inconsistent or overly smooth under weak texture and specular highlights while simulation-trained variants (e.g., deep reinforcement learning) may further suffer from a sim-to-real gap. We propose ColoACT, an autonomous navigation system that integrates an RGB-D-E based Action Chunking Transformer policy (ColoACT policy) for a compact self-propelled Bevel-Gear-Based Endoscopic Robot (BGER). The ColoACT policy augments RGB with estimated relative depth and a gradient-based pseudo-elevation map to enhance fold-ridge saliency and other high-frequency geometric cues, and enables smooth continuous control of the BGER by predicting overlapping action chunks and fusing them via temporal ensembling. In different \textit{ex-vivo} porcine colons (approximately 60 cm), our system achieves success rates of 85.4\% and 72.5\% in straight and curved segments, respectively, and achieves 70\% success in 90-degree turns and 60\% in double-bend sequences, with feasibility further demonstrated in challenging triple-bend segments. The project page is available at: https://Adamhu1.github.io/ColoACT/.