Add Phase 2 Reasoner results: benchmark table, GRPO methodology, updated system prompt

README.md

@@ -16,19 +16,23 @@ tags:
 - economics
 - qwen2.5
 - qlora
+- grpo
+- reinforcement-learning
 - reasoning
 - demo
 models:
+- Alogotron/GameTheory-Reasoner
 - 2reb/GameTheory-Solver
 datasets:
 - 2reb/GameTheory-Bench
 ---
 
-# 🎯 GameTheory-Solver Demo
+# 🎯 GameTheory-Reasoner Demo
 
-**An interactive demo for the [GameTheory-Solver](https://huggingface.co/2reb/GameTheory-Solver) model — a QLoRA fine-tuned Qwen2.5-7B that solves game theory problems with step-by-step mathematical reasoning.**
+**An interactive demo for the [GameTheory-Reasoner](https://huggingface.co/Alogotron/GameTheory-Reasoner) model — a Qwen2.5-7B trained through SFT + GRPO reinforcement learning with verifiable rewards to solve game theory problems with enhanced step-by-step reasoning.**
 
-[![Model](https://img.shields.io/badge/🤗_Model-GameTheory--Solver-orange)](https://huggingface.co/2reb/GameTheory-Solver)
+[![Reasoner](https://img.shields.io/badge/🤗_Reasoner-GameTheory--Reasoner-purple)](https://huggingface.co/Alogotron/GameTheory-Reasoner)
+[![Solver](https://img.shields.io/badge/🤗_Solver-GameTheory--Solver-orange)](https://huggingface.co/2reb/GameTheory-Solver)
 [![Dataset](https://img.shields.io/badge/📊_Dataset-GameTheory--Bench-yellow)](https://huggingface.co/datasets/2reb/GameTheory-Bench)
 [![License](https://img.shields.io/badge/License-Apache_2.0-green.svg)](https://opensource.org/licenses/Apache-2.0)
 
@@ -52,17 +56,21 @@ datasets:
 | **Cooperative Games** | Shapley value, core, nucleolus |
 | **Mechanism Design** | VCG, incentive compatibility, revelation principle |
 
-## 📈 Model Performance
+## 📈 Model Performance: Base → Solver → Reasoner
 
-| Metric | Score |
-|---|:---:|
-| **Overall Accuracy** | **94%** |
-| **Hard Problems** | **94.4%** |
-| Bayesian Games | 100% (from 0% base) |
-| Mechanism Design | 100% (from 60% base) |
+| Metric | Base (82%) | Solver (Phase 1 SFT) | Reasoner (Phase 2 GRPO) |
+|---|:---:|:---:|:---:|
+| **Overall Accuracy** | 82% | **94%** | **94%** |
+| **Hard Problems** | 66.7% | 94.4% | **94.4%** |
+| **Reasoning Quality** | 0.48 | 0.51 | **0.54 (+6%)** |
+| Bayesian Games | 0% | 100% | **100%** |
+| Mechanism Design | 60% | 100% | **100%** |
+
+> Phase 2 uses **GRPO (Group Relative Policy Optimization)** with verifiable rewards, achieving the same 94% accuracy while producing +6% better reasoning quality.
 
 ## 🔗 Links
 
-- 🤖 **Model**: [2reb/GameTheory-Solver](https://huggingface.co/2reb/GameTheory-Solver)
+- 🧠 **Reasoner (Phase 2)**: [Alogotron/GameTheory-Reasoner](https://huggingface.co/Alogotron/GameTheory-Reasoner)
+- 🤖 **Solver (Phase 1)**: [2reb/GameTheory-Solver](https://huggingface.co/2reb/GameTheory-Solver)
 - 📊 **Dataset**: [2reb/GameTheory-Bench](https://huggingface.co/datasets/2reb/GameTheory-Bench) (2,913 verified problems)
 - 🏠 **Base Model**: [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct)

app.py

@@ -39,14 +39,15 @@ DIFFICULTY_COLORS = {
     "hard": "🔴",
 }
 
-SYSTEM_PROMPT = """You are GameTheory-Solver, an expert AI system specialized in game theory analysis. You solve game theory problems with rigorous mathematical reasoning.
+SYSTEM_PROMPT = """You are GameTheory-Reasoner, an expert AI system specialized in game theory analysis. You were trained in two phases — Phase 1 (Solver) used supervised fine-tuning on computationally verified solutions, and Phase 2 (Reasoner) used Group Relative Policy Optimization (GRPO) with verifiable rewards to enhance step-by-step reasoning quality.
 
 For every problem:
-1. Identify the game type and key components (players, strategies, payoffs, information structure)
-2. Apply the appropriate solution concept (Nash Equilibrium, Subgame Perfect Equilibrium, Bayesian Nash Equilibrium, Core, Shapley Value, etc.)
-3. Show complete step-by-step mathematical derivation
-4. Clearly state the final answer
-5. Verify your solution
+1. Think carefully and reason step-by-step through the problem before jumping to conclusions
+2. Identify the game type and key components (players, strategies, payoffs, information structure)
+3. Apply the appropriate solution concept (Nash Equilibrium, Subgame Perfect Equilibrium, Bayesian Nash Equilibrium, Core, Shapley Value, etc.)
+4. Show complete step-by-step mathematical derivation with clear logical transitions between each step
+5. Clearly state the final answer
+6. Verify your solution by checking all equilibrium conditions are satisfied
 
 Be precise with mathematical notation. Show all work. Format payoff matrices clearly using markdown tables when relevant."""
 
@@ -229,7 +230,8 @@ def build_app():
             
 *An AI system trained to solve game theory problems with rigorous step-by-step reasoning*
 
-[![Model](https://img.shields.io/badge/🤗_Model-GameTheory--Solver-blue)](https://huggingface.co/2reb/GameTheory-Solver)
+[![Reasoner](https://img.shields.io/badge/🤗_Reasoner-GameTheory--Reasoner-purple)](https://huggingface.co/Alogotron/GameTheory-Reasoner)
+[![Solver](https://img.shields.io/badge/🤗_Solver-GameTheory--Solver-blue)](https://huggingface.co/2reb/GameTheory-Solver)
 [![Dataset](https://img.shields.io/badge/📊_Dataset-GameTheory--Bench-green)](https://huggingface.co/datasets/2reb/GameTheory-Bench)
 [![Examples](https://img.shields.io/badge/Examples-2,913_Problems-orange)](https://huggingface.co/datasets/2reb/GameTheory-Bench)
 """
@@ -302,7 +304,7 @@ def build_app():
                 gr.Markdown(
                     """Enter any game theory problem and get an AI-generated solution.
                     
-*Powered by Qwen2.5-7B-Instruct via the HuggingFace Inference API with a specialized game theory system prompt.*"""
+*Powered by Qwen2.5-7B-Instruct via the HuggingFace Inference API with the GameTheory-Reasoner system prompt, trained through SFT + GRPO reinforcement learning for enhanced step-by-step reasoning.*"""
                 )
 
                 with gr.Row():
@@ -361,12 +363,50 @@ def build_app():
             with gr.TabItem("ℹ️ About", id="about"):
                 gr.Markdown(
                     """
-## About GameTheory-Solver
+## About GameTheory-Reasoner
 
 ### What is this?
-GameTheory-Solver is an AI system fine-tuned to solve game theory problems with rigorous mathematical reasoning.
+GameTheory-Reasoner is an AI system trained in **two phases** to solve game theory problems with rigorous mathematical reasoning.
 It was trained on the **GameTheory-Bench** dataset — a collection of 2,913 computationally verified game theory problems.
 
+### Training Pipeline
+
+| Phase | Method | Model | Description |
+|-------|--------|-------|-------------|
+| Base | — | Qwen2.5-7B-Instruct | Pre-trained foundation model |
+| Phase 1: **Solver** | Supervised Fine-Tuning (SFT) | [GameTheory-Solver](https://huggingface.co/2reb/GameTheory-Solver) | Fine-tuned on verified solutions with LoRA adapters |
+| Phase 2: **Reasoner** | GRPO (RL) | [GameTheory-Reasoner](https://huggingface.co/Alogotron/GameTheory-Reasoner) | Reinforcement learning with verifiable rewards for reasoning quality |
+
+### Benchmark Results: Base → Solver → Reasoner
+
+| Metric | Base (Qwen2.5-7B) | Solver (Phase 1 SFT) | Reasoner (Phase 2 GRPO) |
+|--------|:------------------:|:---------------------:|:-----------------------:|
+| **Overall Accuracy** | 82% | **94%** | **94%** |
+| **Hard Problems** | 66.7% | 94.4% | **94.4%** |
+| **Reasoning Quality** | 0.48 | 0.51 | **0.54 (+6%)** |
+
+### Per-Category Breakdown
+
+| Category | Base | Solver | Reasoner |
+|----------|:----:|:------:|:--------:|
+| 🎲 Normal Form 2×2 | 100% | 100% | 100% |
+| 🎲 Normal Form 3×3 | 100% | 100% | 100% |
+| 🎲 Normal Form 3×4 | 80% | 80% | 80% |
+| 🎲 Normal Form 4×4 | 80% | 80% | 80% |
+| ⚔️ Zero-Sum Games | 100% | 100% | 100% |
+| 🌳 Sequential Games | 100% | 100% | 100% |
+| 🔨 Auction Theory | 80% | 100% | 100% |
+| 🔮 Bayesian Games | 0% | 100% | 100% |
+| 🤝 Cooperative Games | 80% | 80% | 80% |
+| ⚙️ Mechanism Design | 60% | 100% | 100% |
+
+### Phase 2: GRPO with Verifiable Rewards
+The Reasoner model was trained using **Group Relative Policy Optimization (GRPO)**, a reinforcement learning method that:
+- Generates multiple solution candidates per problem
+- Scores each using **verifiable reward functions** (answer correctness, format compliance, reasoning quality)
+- Updates the policy to favor higher-quality reasoning chains
+- Achieves the same 94% accuracy as the Solver while producing **+6% better reasoning quality** (measured by structured reasoning metrics)
+
 ### Supported Problem Types
 | Category | Description | Examples |
 |----------|------------|----------|
@@ -378,18 +418,13 @@ It was trained on the **GameTheory-Bench** dataset — a collection of 2,913 com
 | 🤝 Cooperative Games | Coalition-based games | Shapley value, core, nucleolus |
 | ⚙️ Mechanism Design | Incentive design problems | VCG, revelation principle, IC constraints |
 
-### Technical Details
-- **Base Model:** [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct)
-- **Training Data:** [2reb/GameTheory-Bench](https://huggingface.co/datasets/2reb/GameTheory-Bench) (2,913 verified problems)
-- **Training Method:** Supervised fine-tuning with LoRA adapters
-- **Solution Verification:** All training solutions computationally verified using nashpy and scipy
-
 ### How It Works
 - **Browse Examples tab:** Shows pre-loaded problems from the dataset with verified solutions
-- **Solve Your Own tab:** Sends your problem to Qwen2.5-7B-Instruct via the HuggingFace Inference API with a specialized game theory system prompt
+- **Solve Your Own tab:** Sends your problem to Qwen2.5-7B-Instruct via the HuggingFace Inference API with the GameTheory-Reasoner system prompt
 
 ### Links
-- 🤗 [Model on HuggingFace](https://huggingface.co/2reb/GameTheory-Solver)
+- 🤖 [Reasoner Model (Phase 2)](https://huggingface.co/Alogotron/GameTheory-Reasoner)
+- 🤗 [Solver Model (Phase 1)](https://huggingface.co/2reb/GameTheory-Solver)
 - 📊 [Dataset on HuggingFace](https://huggingface.co/datasets/2reb/GameTheory-Bench)
 """
                 )