Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces Q-Learning Lab, an innovative browser-based tool designed to enhance the teaching of tabular Q-learning through learner-generated trace analysis. By allowing students to actively engage with the learning process—running their own agents, exporting detailed transition traces, and analyzing their performance—the tool transforms passive observation into an interactive learning experience. Validation through correctness checks, hyperparameter sweeps, and reward-editing studies confirms the pedagogical efficacy of the tool, making it a valuable resource for teaching reinforcement learning concepts effectively.
Learners can now generate and analyze their own Q-learning data, turning abstract reinforcement learning concepts into tangible insights.
Reinforcement learning is usually introduced through the Bellman update, yet the equation often remains abstract to undergraduates: they watch policy arrows converge but rarely observe how each value is computed or why an action is chosen. We present Q-Learning Lab, a single-file, browser-based, bilingual (Thai/English) tool for teaching tabular Q-learning that requires no installation. Beyond the usual gridworld visualization - color-coded Q-values and policy arrows on a $5 \times 5$ world - the tool exposes a live Bellman-substitution panel showing the numeric update at every step, and logs each transition, including the full pre-action Q-row, the greedy-versus-random decision under $\varepsilon$-greedy exploration, and wall-collision events, into an exportable trace. The central contribution is a learn-export-analyze loop: learners run their own agent, export the complete trace as CSV, and analyze it themselves, producing learning curves, value heatmaps, and visitation maps, turning a passive demonstration into a source of learner-generated data for reflective inquiry. We validate the tool without human-subject data through three complementary evaluations: (i) correctness of the learned values and policy against a value-iteration ground truth on the identical MDP; (ii) hyperparameter sweeps over $α$, $γ$, and $\varepsilon$ showing that every pedagogical claim the tool makes is reproducible; and (iii) a reward-editing study that uses the ground-truth optimal policy to separate two behaviorally identical but diagnostically opposite failure modes - an exploration failure versus genuine reward misspecification - that a single edited reward can produce. We also compare the tool against existing gridworld visualizers, describe its grounding in learning-by-doing pedagogy, and include a 50-minute lesson plan. The tool and all experiment code are openly available.