01 / THE IDEA
Learning a route,
one decision at a time.
This project explores Q-learning through a small grid-world game. The agent has to reach a goal while avoiding a pit and navigating around a wall. The central idea is to learn how valuable each move is from the state of the board and the reward that follows.
I used a neural network to represent the Q-function. It takes the board as input and estimates four action values, turning the problem into a connection between state representation, reward design and decision-making.
02 / THE ENVIRONMENT
A board the network
can read.
The implemented state is an 8 × 8 × 4 NumPy array. Each channel records a different object: goal, pit, wall or player. Flattening that array gives the network 256 input values.
The supplied code starts the player at row 0, column 1, places the pit at (1, 1), the wall at (2, 2) and the goal at (7, 7). A separate display function converts the internal representation into a readable grid.
Rewards give each move a consequence.
The step penalty encourages shorter routes, while the terminal rewards distinguish success from failure. Those choices make reward design a central part of the learning problem.
03 / THE ENGINEERING
One state.
Four action values.
The Keras model uses three dense hidden layers with ReLU activations and 20% dropout, followed by a linear output layer. The output values can therefore represent both positive and negative expected rewards.
The model is configured with mean squared error and RMSprop. The test routine predicts the four values, selects the largest with argmax, and calls the movement function before checking the new reward. It also includes a move limit to stop an unsuccessful run.
The original figures are retained as published. The architecture described here follows the Python implementation where their dimensions differ.
04 / WHAT THE REPOSITORY SHOWS
The design and
the implementation excerpts.
The archive contains the grid and reward functions, the neural-network definition, and a greedy test routine in Q_NN_2.01.py. A second file loads the saved model and repeats the environment setup.
The main file explicitly identifies itself as a partial source sample. The movement function, full training loop and trained model are not included. The README describes goal-reaching behaviour, but the archive does not provide the runs needed to reproduce or measure shortest-path performance.
05 / LOOKING BACK
Small environments make
the decisions visible.
This project connects three practical parts of reinforcement learning: how a state is encoded, what a reward means, and how the model’s output becomes an action. A simple grid makes those decisions easier to inspect.
A fuller evaluation would compare the learned policy with the shortest valid route, report success rates and path lengths across repeated runs, and test different starting positions. Publishing the complete training setup and saved weights would make those comparisons reproducible.
A network definition explains the model. Repeated runs, a baseline and a complete environment establish whether its decisions achieve the intended goal.