RL Grid World — Q-Learning Visualizer
Watch AI learn in real-time! This interactive Reinforcement Learning demo shows a Q-Learning agent mastering a grid world — from random exploration to optimal policy. Adjust hyperparameters, visualize Q-values and policy arrows, and see the learning curve. 100% browser-side, no sign-up. Perfect for learning RL.
⚙️ Hyperparameters
Learning Curve
Steps per episode — fewer is better (agent finds shorter paths)
How Q-Learning Works
1. The World
An 8×8 grid with walls, a start (S), and a goal (★). The agent must find the shortest path from S to ★.
2. Q-Table
For every cell and every action (↑→↓←), the agent keeps a "quality" value — how good is it to take that action here?
3. Exploration
At first, the agent moves randomly (ε = 1.0). Over time, ε decays and it relies more on learned Q-values.
4. Learning
After each move, Q is updated: Q(s,a) ← Q(s,a) + α[r + γ·max Q(s',a') − Q(s,a)]
5. Rewards
Reaching the goal: +10. Each step: −1. Hitting a wall: −5. The agent learns to reach the goal fast.
6. Optimal Policy
Watch the arrows converge — each cell's best action is highlighted. The learning curve drops as paths get shorter.
Frequently Asked Questions
What is Q-Learning?
Q-Learning is a model-free reinforcement learning algorithm. It learns a Q-function that maps state-action pairs to expected future rewards. The agent doesn't need to know how the environment works — it learns purely from trial and error.
Is this really running in my browser?
Yes! The entire Q-Learning algorithm runs in vanilla JavaScript on your device. No server, no API calls, no data upload. You can even disconnect your internet after the page loads.
What am I looking at?
The grid shows an AI agent (stick figure) learning to navigate from the start (S) to the goal (★). The colored cells show Q-values (red = low, green = high). The arrows show the best action from each cell. The chart shows steps-per-episode over time — it drops as the agent learns shorter paths.
Why does the agent move randomly at first?
This is the exploration phase. The agent uses an ε-greedy strategy: with probability ε, it takes a random action to explore. As it learns, ε decreases and it relies more on its Q-values (exploitation). This exploration-exploitation balance is the heart of reinforcement learning.
What do the colors mean?
The heatmap shows the maximum Q-value for each cell. Red means the agent expects low future reward from that cell; green means high expected reward. As training progresses, you'll see a "green path" emerge from start to goal — the learned optimal route.
What are the hyperparameters?
α (learning rate): how much to update Q-values each step. γ (discount factor): how much to value future vs. immediate rewards. ε (epsilon): exploration rate — starts at 1.0 and decays over episodes. Try changing them to see how learning speed and quality are affected!
Can I change the maze?
Yes! Click "New Maze" to generate a random obstacle layout. Each layout is verified to be solvable using BFS pathfinding, so the agent can always reach the goal.
Is this tool free?
Completely free, no sign-up required. It's a pure educational tool running 100% in your browser. Perfect for students, ML practitioners, and anyone curious about reinforcement learning.