Lil Maina Combat - Neural Difficulty Policy
Model: DQN policy that powers the Neural difficulty tier in Lil Maina Combat, a 2D fighting game (Java 26 / Gradle). It learns to fight by self-play against a scripted rule-based opponent and beats every built-in difficulty tier.
Quick facts
| Architecture | MLP 10 -> 64 -> 64 -> 6 (ReLU hidden, linear output) |
| Input | 10-dim normalized fight state |
| Output | Q-values over 6 actions (attack, block, approach, retreat, jump, hold) |
| File | fighter_policy.bin (~42 KB, custom binary LFCDQN1, version 1, double precision) |
| License | MIT |
| Model type | DQN (Deep Q-Network), greedy argmax policy |
State features (normalized)
- My X / arena width
- My Y / ground level
- My health / max health
- Opponent X / arena width
- Opponent Y / ground level
- Opponent health / max health
- Horizontal distance / arena width
- Vertical offset / ground level
- My facing direction
- Opponent facing direction
Features are mirrored per-agent so the same policy plays both sides of the fight.
Training
- Algorithm: DQN with target network and experience replay (replay buffer capacity 100k).
- Opponent: rule-based controller at the Hard tier, played in a 600-tick time-capped match.
- Reward: health-lead shaping each tick, knockout bonus, anti-turtle penalty so holding at range is never free.
- Exploration: epsilon annealed from 1.0 to 0.05 across a 30k-step final drain; best checkpoint kept by greedy ladder evaluation during training.
Evaluation
Greedy (argmax) policy vs. each built-in rule tier, on two independent 200-game ladder evals (1200 games):
| Opponent | Win rate |
|---|---|
| Easy | 100% |
| Medium | 100% |
| Hard | 100% |
0 losses across all 1200 games. The GUI Neural tier plays this exact greedy policy.
Usage
NeuralNet net = ModelPersistence.loadOrNull("fighter_policy.bin");
The game ships the file as fighter_policy.bin and embeds it in packaged builds (jar / AppImage / Windows .exe) so the Neural tier works self-contained.
Limitations
- Single trained policy; tuned against the rule-based opponent family, not against human playstyles or other learned agents.
- Greedy action selection only (no exploration at inference).
- Q-values are not calibrated probabilities; treat as relative preferences.