Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper systematically validates the Spiking Actor Network Soft Actor Critic (SANSAC), a spiking neural network (SNN) variant of the Soft Actor-Critic (SAC) algorithm, on conventional hardware for continuous control tasks. The study reveals that SANSAC achieves performance nearly equivalent to traditional SAC networks, despite the inherent differences in their architectures. These findings establish a foundational benchmark for future research into the application of SNNs in reinforcement learning, particularly in neuromorphic computing contexts.
SANSAC matches the performance of traditional SAC networks, challenging assumptions about the limitations of spiking neural networks in continuous control tasks.
Reinforcement learning (RL) algorithms have made strides over the past decade applying them to a wide range of problems and control tasks. However, the deployment of RL on neuromorphic hardware for continuous control tasks remains under-validated. Namely it is unclear whether replacing a conventional actor network with a spiking neural network (SNN) affects the performance of an agent before any hardware-specific benefits manifest. We provide a systematic validation of a minimal, neuromorphically viable spiking actor variant of Soft Actor-Critic (SAC) on conventional hardware, establishing a baseline for future neuromorphic RL research. In this paper, we propose the Spiking Actor Network Soft Actor Critic (SANSAC) to address the use of RL frameworks in continuous environments, designed as a framework that can be implemented on neuromorphic hardware. We compare a traditional Soft Actor Critic (SAC) network to SANSAC in a traditional computer. We demonstrate the near equivalent performance of SANSAC and SAC, while addressing the impact of hidden dimensions. Our results demonstrate the viability of SNN based algorithms in complex continuous environments, as well as competitive performance to traditional neural networks in traditional computers, providing a basis to continue exploring the use of SNNs in continuous RL frameworks.