Search papers, labs, and topics across Lattice.
This study establishes a robust benchmark for evaluating lightweight reinforcement learning agents in imperfect-information card games by employing a strong, fixed, rule-based expert for Gin Rummy as a yardstick. The authors identify key factors that enhance agent performance, including trust region updates, targeted rewards, and a curriculum of progressively tougher opponents, which collectively improve a self-play champion's win rate from 30% to 36% against the expert. Notably, the findings indicate that increasing model capacity does not significantly enhance performance, suggesting that the limitations are more related to information than network size, leading to a reusable framework applicable across various games.
Lightweight agents can achieve competitive performance against expert opponents without direct training on them, revealing critical strategies for success in reinforcement learning.
Reinforcement learning agents for imperfect-information card games are only as strong as the opponents they train against, and they are hard to grade, since they beat a random opponent over 99 percent of the time and only tie copies of themselves. So we build a strong, fixed, rule-based expert for Gin Rummy and use it only as a yardstick, never for training. It beats every agent we trained 70 to 99 percent of the time. Across more than a hundred runs, we isolate what makes a lightweight agent stronger. Trust region updates, a well-aimed reward, a curriculum of tougher opponents, warm starting, and keeping the best checkpoint all help, and stacking them lifts a self-play champion from about 30 to 36 percent against the expert. Several ideas did not pay off. Short-term and longer-term reward shaping, learned state embeddings, imitation and DAgger, and a live large language model opponent were each unhelpful, too slow, or too heavy to train at scale. Comparing MLP, convolutional, set-based, attention, and recurrent encoders shows that extra capacity does little to break the ceiling, suggesting the limit is information rather than network size. We add standard baselines (neural fictitious self-play and information set Monte Carlo search) and confirm the approach carries over to Leduc Hold'em, where the optimum is computable. The result is a lightweight, game-agnostic recipe that trains competitive agents without training on the expert, for any game a small model can handle, reported with robust statistics and released as a reusable package.
