Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
S2Act is introduced as a computationally efficient framework for deploying reinforcement learning policies using spiking neural networks (SNNs) by training an actor-critic model based on rate-based spiking neurons and transferring the trained weights into physical parameters of leaky integrate-and-fire (LIF) neurons. This approach mitigates the vanishing gradient problem by shaping LIF neuron parameters to approximate ReLU activations, reducing the need for complex SNN-specific hyperparameter tuning. Experimental results in multi-agent stochastic environments and real-world TurtleBot deployments on Intel's Loihi neuromorphic hardware demonstrate that S2Act outperforms relevant baselines in task performance and real-time inference.
S2Act makes SNN-based RL practical by globally shaping LIF neuron parameters to approximate ReLU activations, sidestepping the hyperparameter tuning headaches that plague traditional SNNs.
Spiking neural networks (SNNs) and biologically-inspired learning mechanisms are attractive in mobile robotics, where the size and performance of onboard neural network policies are constrained by power and computational budgets. Existing SNN approaches, such as population coding, reward modulation, and hybrid artificial neural network (ANN)-SNN architectures, have shown promising results; however, they face challenges in complex, highly stochastic environments due to SNN sensitivity to hyperparameters and inconsistent gradient signals. To address these challenges, we propose simple spiking actor (S2Act), a computationally lightweight framework that deploys an RL policy using an SNN in three steps: (1) architect an actor-critic model based on an approximated network of rate-based spiking neurons, (2) train the network with gradients using compatible activation functions, and (3) transfer the trained weights into physical parameters of rate-based leaky integrate-and-fire (LIF) neurons for inference and deployment. By globally shaping LIF neuron parameters such that their rate-based responses approximate ReLU activations, S2Act effectively mitigates the vanishing gradient problem, while pre-constraining LIF response curves reduces reliance on complex SNN-specific hyperparameter tuning. We demonstrate our method in two multi-agent stochastic environments (capture-the-flag and parking) that capture the complexity of multi-robot interactions, and deploy our trained policies on physical TurtleBot platforms using Intel's Loihi neuromorphic hardware. Our experimental results show that S2Act outperforms relevant baselines in task performance and real-time inference in nearly all considered scenarios, highlighting its potential for rapid prototyping and efficient real-world deployment of SNN-based RL policies.