Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
Themis is an innovative framework that integrates explainable AI (XAI) with Reinforcement Learning from Human Feedback (RLHF) to enhance the safety and transparency of RL systems. By supporting over 200 environments and providing a cloud-based platform for human feedback collection, Themis allows researchers to effectively train reward models that align closely with true reward signals. The results indicate that Themis not only matches but can also outperform traditional reward signals, demonstrating its potential to improve RL training methodologies significantly.
Themis can train RL models that align better with true rewards than existing methods, all while being user-friendly and scalable for large experiments.
Training safe Reinforcement Learning (RL) systems is inherently challenging, with no guarantee of avoiding unwanted behaviors. The most effective defenses against this are (i) transparency through explainability and (ii) alignment via human feedback. While both show promising results, no publicly available framework currently combines them. To address this, we introduce Themis, an XAI-enabled testing and evaluation framework for Reinforcement Learning from Human Feedback. Themis supports over 200 widely used environments and is easily configurable for experiments in RL, transparency, and alignment. Our results show that Themis can train reward models that match or outperform the environment's true reward signal using human preferences. We also provide a cloud-based platform for collecting human feedback and managing experiments. It is user-friendly, auto-scalable, and supports large participant groups across multiple experiments without extra development overhead. Tests show Themis can support one thousand users in back-to-back experiments on a modest commercial machine.