---
tags:
- reinforcement-learning
- q-learning
- gymnasium
- custom-implementation
model-index:
- name: q-FrozenLake-v1-4x4-noSlippery
  results:
  - task:
      type: reinforcement-learning
      name: Reinforcement Learning
    dataset:
      type: FrozenLake-v1-4x4-no_slippery
      name: FrozenLake-v1-4x4-no_slippery
    metrics:
    - type: mean_reward
      name: Mean reward
      value: 1.000000
    - type: success_rate
      name: Success rate
      value: 100.000000
---

# Q-Learning Agent playing FrozenLake-v1-4x4-no_slippery

This repository contains a tabular Q-Learning agent trained on
**FrozenLake-v1-4x4-no_slippery** using Gymnasium.

The agent was implemented as part of the Hugging Face Deep
Reinforcement Learning Course.

## Evaluation results

| Metric | Result |
|---|---:|
| Evaluation episodes | 1000 |
| Mean reward | 1.0000 |
| Reward standard deviation | 0.0000 |
| Success rate | 100.00% |
| Mean episode length | 6.00 |

## Agent replay

![Q-Learning agent replay](replay.gif)

## Repository files

- `q-learning.pkl`: Q-table, environment settings and hyperparameters
- `qtable.npy`: NumPy Q-table
- `results.json`: evaluation summary
- `evaluation_episodes.csv`: individual evaluation episodes
- `training_history.csv`: training history, when available
- `replay.gif`: animated agent replay
- `replay.mp4`: video replay, when available

## Load the trained agent

```python
import pickle
import gymnasium as gym

from huggingface_hub import hf_hub_download

model_path = hf_hub_download(
    repo_id="a1914114315/q-FrozenLake-v1-4x4-noSlippery",
    filename="q-learning.pkl"
)

with open(model_path, "rb") as file:
    model = pickle.load(file)

qtable = model["qtable"]

env = gym.make(
    model["env_id"],
    **model.get("env_kwargs", {})
)

print("Environment:", model["env_id"])
print("Q-table shape:", qtable.shape)
print(
    "Mean reward:",
    model["evaluation"]["mean_reward"]
)
```

## Training configuration

```json
{
  "n_training_episodes": 10000,
  "max_steps": 99,
  "learning_rate": 0.7,
  "gamma": 0.95,
  "max_epsilon": 1.0,
  "min_epsilon": 0.05,
  "decay_rate": 0.0005,
  "seed": 42
}
```