---
license: mit
library_name: custom
language:
  - en
tags:
  - reinforcement-learning
  - dqn
  - self-play
  - fighting-game
  - game-ai
pipeline_tag: reinforcement-learning
---

# Lil Maina Combat - Neural Difficulty Policy

**Model:** DQN policy that powers the **Neural** difficulty tier in *Lil Maina Combat*, a 2D fighting game (Java 26 / Gradle). It learns to fight by self-play against a scripted rule-based opponent and beats every built-in difficulty tier.

## Quick facts

| | |
|---|---|
| Architecture | MLP `10 -> 64 -> 64 -> 6` (ReLU hidden, linear output) |
| Input | 10-dim normalized fight state |
| Output | Q-values over 6 actions (attack, block, approach, retreat, jump, hold) |
| File | `fighter_policy.bin` (~42 KB, custom binary `LFCDQN1`, version 1, double precision) |
| License | MIT |
| Model type | DQN (Deep Q-Network), greedy argmax policy |

## State features (normalized)

1. My X / arena width
2. My Y / ground level
3. My health / max health
4. Opponent X / arena width
5. Opponent Y / ground level
6. Opponent health / max health
7. Horizontal distance / arena width
8. Vertical offset / ground level
9. My facing direction
10. Opponent facing direction

Features are mirrored per-agent so the same policy plays both sides of the fight.

## Training

- **Algorithm:** DQN with target network and experience replay (replay buffer capacity 100k).
- **Opponent:** rule-based controller at the Hard tier, played in a 600-tick time-capped match.
- **Reward:** health-lead shaping each tick, knockout bonus, anti-turtle penalty so holding at range is never free.
- **Exploration:** epsilon annealed from 1.0 to 0.05 across a 30k-step final drain; best checkpoint kept by greedy ladder evaluation during training.

## Evaluation

Greedy (argmax) policy vs. each built-in rule tier, on **two independent 200-game ladder evals (1200 games)**:

| Opponent | Win rate |
|---|---|
| Easy | 100% |
| Medium | 100% |
| Hard | 100% |

0 losses across all 1200 games. The GUI Neural tier plays this exact greedy policy.

## Usage

```java
NeuralNet net = ModelPersistence.loadOrNull("fighter_policy.bin");
```

The game ships the file as `fighter_policy.bin` and embeds it in packaged builds (jar / AppImage / Windows .exe) so the Neural tier works self-contained.

## Limitations

- Single trained policy; tuned against the rule-based opponent family, not against human playstyles or other learned agents.
- Greedy action selection only (no exploration at inference).
- Q-values are not calibrated probabilities; treat as relative preferences.