Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study introduces tinyDSM, a framework that enables cm-sized millirobots to autonomously explore and develop skills through reinforcement learning, emphasizing minimal initial knowledge and intrinsic motivation. The approach integrates a cognitive architecture with a hierarchical knowledge graph to facilitate skill acquisition while operating within strict resource constraints. Experiments show that the millirobot can progress from basic motor skills to complex movements in just 15 minutes, highlighting the effectiveness of the framework in fostering open-ended skill development.
Millirobots can autonomously learn complex movements in just 15 minutes, thanks to a novel framework that balances minimal initial knowledge with intrinsic motivation.
In this study, we investigate developmental mechanisms that enable small, resource-constrained systems such as cm-sized millirobots to autonomously explore, learn, and adapt their capabilities throughout their lifespan. Reinforcement learning algorithms guide the agent's skill acquisition and adaptation through the interplay of our proposed tinyDSM, which integrates intrinsic motivation and fitness-based assessment. We strive for minimal, hard-wired skills while encouraging the open-ended development of new skills. A key emphasis in our approach is to encode minimal a-priori general knowledge, which serves as a foundational starting point for the system as it further learns system-specific dependencies from the initial knowledge provided. Thus, by design, our approach attempts to cover very generic application domains. The methodology is based on (a) developmental mechanism with intrinsic motivation, and (b) a cognitive architecture (knowledge, reasoning, learning), while (c) utilizing minimal resources. It uses a hierarchical knowledge graph and kinematic reasoners to model and evaluate simple and advanced motion related skills. In our experiments, we use a resource-constrained millirobot with a volume of 36 cm^3 with a Raspberry Pi Pico 32-bit microcontroller (RP2040) that integrates all described features and capabilities except the camera system in 9 kB. Starting with learning the most elementary motor skills the millirobot autonomously progresses from simple linear and angular movements to complex geometric patterns within 15 minutes. To complement the physical experiments, we perform a simulation-based analysis that enables systematic comparisons across learning algorithms and intrinsic motivation parameters.