Watch Algorithms Come to Life
Not just tutorials โ living, breathing simulations that show you exactly how algorithms learn. Train a neural network in your browser. Watch Q-Learning explore a maze. See PPO clip its own updates. All 100% client-side, no sign-up, no server.
๐ง Reinforcement Learning Series
A complete journey from tabular methods to deep RL โ 12 interactive tools, each teaching one core concept.
Tabular RL
Zero dependencies, pure JavaScript. Learn the foundations โ Bellman equations, TD learning, on-policy vs off-policy, Monte Carlo, and model-based planning.
RL Grid World โ Q-Learning Visualizer
Watch AI learn in real-time! This interactive Reinforcement Learning demo shows a Q-Learning agent mastering a grid world โ from random exploration to optimal policy. Adjust hyperparameters, visualize Q-values and policy arrows, and see the learning curve. 100% browser-side, no sign-up. Perfect for learning RL.
Start Learning โSARSA vs Q-Learning โ Cliff Walking
Watch SARSA and Q-Learning learn side-by-side on the classic Cliff Walking benchmark! See on-policy vs off-policy RL diverge in real-time โ SARSA learns safe paths, Q-Learning learns optimal but risky paths. Adjust hyperparameters, compare learning curves, and understand the core difference in RL. 100% browser-side.
Start Learning โDouble Q-Learning โ Overestimation Bias Visualizer
See why Q-Learning overestimates Q-values and how Double Q-Learning fixes it! Watch Q-Learning, Double Q-Learning, and the exact optimal Q* (via Value Iteration) learn side-by-side on Cliff Walking. Real-time bias chart shows Q-Learning's systematic positive bias vs Double Q-Learning's unbiased estimates. 100% browser-side.
Start Learning โMonte Carlo Blackjack โ Learn the Optimal Strategy
Watch Monte Carlo control learn the optimal Blackjack strategy from scratch! The agent plays thousands of hands per second, using first-visit MC to estimate Q-values. See the strategy table converge to the famous optimal Blackjack playbook. Compare with the known optimal policy. 100% browser-side, no server needed.
Start Learning โDyna-Q โ Planning vs Direct RL Maze Race
Watch Dyna-Q learn a maze 10x faster than Q-Learning using "imagined" experience! After each real step, Dyna-Q replays N planning steps through a learned model. See the shortcut maze experiment: when the environment changes mid-training, Dyna-Q adapts in episodes, not dozens. 100% browser-side.
Start Learning โDeep RL
Introduce TensorFlow.js. Replace Q-tables with neural networks. Experience replay, target networks, policy gradients, and the algorithms behind ChatGPT.
DQN CartPole โ Neural Network Learns to Balance
Watch a neural network learn to balance a pole on a cart โ right in your browser! This is Deep Q-Network (DQN) with TensorFlow.js: experience replay, target network, and ฮต-greedy exploration. See the AI go from random flailing to 500-step perfect balance in real-time. 100% browser-side, no server.
Start Learning โDouble & Dueling DQN โ Fixing DQN's Blind Spots
Three neural networks race on CartPole! Compare vanilla DQN vs Double DQN (decouples action selection from evaluation to fix overestimation) vs Dueling DQN (separates state value from action advantage). Watch Q-value estimates, training stability, and learning speed differ in real-time. TensorFlow.js, 100% browser-side.
Start Learning โREINFORCE โ Policy Gradient Learns from Full Episodes
Watch a neural network learn a policy directly โ no Q-values, no value function! REINFORCE (Monte Carlo Policy Gradient) is the purest form of policy-based RL. The agent plays full episodes, then updates its strategy using the policy gradient theorem: โJ = E[โlog ฯ(a|s) ยท G]. See action probabilities shift in real-time as the AI learns. TensorFlow.js, 100% browser-side.
Start Learning โA2C โ Actor-Critic with Advantage Baseline
REINFORCE has high variance. A2C fixes it by adding a Critic network that estimates V(s) as a baseline. The Actor now learns from the advantage A = G - V(s) โ "how much better was this action than expected?" Watch two agents train side by side: vanilla REINFORCE vs A2C with baseline. See the variance drop and training stabilize. TensorFlow.js, 100% browser-side.
Start Learning โPPO โ Proximal Policy Optimization (OpenAI's Default)
A2C throws away data after one update. PPO reuses it! By clipping the policy ratio r = ฯ_new/ฯ_old to [1-ฮต, 1+ฮต], PPO safely performs multiple epochs on the same batch โ getting more learning per episode without destabilizing. This is OpenAI's go-to algorithm (and the RLHF engine behind ChatGPT). Watch A2C vs PPO train side by side: PPO learns faster, more stably, and recovers from bad episodes. TensorFlow.js, 100% browser-side.
Start Learning โContinuous Control
Break into continuous action spaces. Real robots need real-valued actions โ motor torque, steering angles, voltages.
SAC โ Soft Actor-Critic for Continuous Control
Everything so far used discrete actions (left/right). Real robots need continuous control โ motor torque, steering angle, voltage. SAC handles this with three innovations: (1) Maximum entropy โ the agent optimizes reward PLUS policy entropy, balancing exploitation and exploration automatically. (2) Stochastic policy โ the Actor outputs a Gaussian distribution (mean + std), not a single action. (3) Twin Q-networks โ take the min of two critics to fight overestimation. Watch a pendulum swing up from scratch, purely in your browser. TensorFlow.js, 100% browser-side.
Start Learning โRL Benchmark โ Compare All Algorithms Side by Side
The grand finale: run DQN, Double DQN, REINFORCE, A2C, and PPO simultaneously on the same CartPole environment and watch their learning curves diverge in real time. Which algorithm converges fastest? Which is most stable? Which solves CartPole in the fewest episodes? See the answer unfold before your eyes. Plus a complete algorithm selection decision tree โ when to use value-based vs policy-based, on-policy vs off-policy, discrete vs continuous. TensorFlow.js, 100% browser-side.
Start Learning โ๐ More Learning Tools
Other interactive educational tools.
๐ก What You'll Learn
Core RL Math
Bellman equations, TD learning, policy gradient theorem, maximum entropy framework โ not just formulas, but visible in action.
Algorithm Intuition
Why does Q-Learning overestimate? Why does REINFORCE have high variance? Why does PPO clip? See the problems before the solutions.
Real Implementation
Experience replay buffers, target networks, reparameterization tricks, clipped surrogate losses โ all in readable JavaScript.
Hyperparameter Feel
Adjust learning rates, epsilon schedules, clip ratios, entropy coefficients. Build intuition for what each knob actually does.
Ready to Watch AI Learn?
Start from Episode 1 โ no math degree required, just curiosity.
Start with Q-Learning โ