Search papers, labs, and topics across Lattice.
This paper addresses the challenge of simulation execution bottlenecks in high-performance reinforcement learning (RL) inference by introducing AutoThread, a hybrid adaptive thread-tuning method. By analyzing the ratio of task execution time to scheduling time, the authors develop a Physics-Informed Neural Operator (PINO) to predict optimal thread counts and utilize a finite-source M/M/1 queueing model to enhance prediction accuracy under dynamic workloads. Experimental results demonstrate that AutoThread achieves an average speedup of 18.4% over static strategies and significantly outperforms existing methods in terms of throughput and execution time reduction, with improvements of up to 83.8%.
AutoThread reduces simulation execution time by up to 83.8% while boosting throughput to 1.8x that of existing RL inference methods.
In simulation-in-the-loop decision-making systems, reinforcement learning (RL) inference is often constrained by simulator-side execution overhead, where workloads are highly dynamic and sensitive to runtime thread configurations. Existing multithreaded strategies struggle to match thread resources before or during execution, causing resource contention, scheduling overhead, and reduced throughput. Through empirical analysis, we identify the ratio of task execution time to scheduling time as the key factor determining the optimal thread count. Building on this insight, we propose AutoThread, a hybrid adaptive thread-tuning method for mitigating simulation bottlenecks in RL inference. AutoThread employs a Physics-Informed Neural Operator (PINO) as a thread-count predictor and incorporates a finite-source M/M/1 queueing model to constrain and guide prediction, enabling fast and accurate estimation under dynamic workloads. It further performs load-aware online fine-tuning to compensate for prediction errors and refine resource allocation. Experiments show that AutoThread improves average speedup by 18.4\% over static strategies, achieves average throughput of 1.7x and 1.8x that of XGBoost and Reinforcer, respectively, and reduces execution time by up to 83.8\% compared with state-of-the-art methods. Our code and dataset are publicly available at https://github.com/suchenjm/AutoThread.
