Switch demo to bird-cot CoT model, add sqlite upload + reasoning, honest numbers

README.md

@@ -5,24 +5,27 @@ app_file: app.py
 license: apache-2.0
 colorFrom: blue
 colorTo: green
-pinned: false
+pinned: true
 ---
 
 # LocalSQL
 
 Ask a SQLite database questions in plain English.
 
-LocalSQL is a small, open text-to-SQL tool built around fine-tuned
-Qwen2.5-Coder-7B LoRA adapters. The default adapter reached **78.2% result
-accuracy on Spider V1 dev**, beating the Grok-4 and DeepSeek-V3 baselines used
-to build its preference data.
+LocalSQL is a small, open text-to-SQL tool built around a fine-tuned
+Qwen2.5-Coder-7B model. It reasons step by step over the schema, writes the SQL,
+and runs it read-only against your database. The default model,
+[`jk200201/qwen2.5-coder-7b-bird-cot`](https://huggingface.co/jk200201/qwen2.5-coder-7b-bird-cot),
+scores **52.1%** greedy and **58.5%** with self-consistency (K=8) on the BIRD
+dev split, matching DeepSeek V4-Pro (58.7%, 1.6T params) at roughly 0.4% of the
+size.
 
 The product goal is simple:
 
 - **Local mode:** run the model on your own GPU, point it at your own `.sqlite`
   file, and keep your data on your machine.
-- **Demo mode:** run the same model in a hosted Gradio app against sample
-  data so people can try it without setup.
+- **Demo mode:** run the same model in a hosted Gradio app against sample data,
+  or upload your own `.sqlite` file to try it without any setup.
 
 ```bash
 python3 -m src.text2sql --db chinook.sqlite --q "Which 5 artists have the most albums?"
@@ -48,17 +51,19 @@ Results
 
 Most text-to-SQL tools are cloud wrappers: send schema plus question to a large
 API model, pay per request, and trust the remote provider with your database
-structure. LocalSQL explores the opposite path: a specialized 7B model that can
-run locally and still compete with much larger frontier models on SQL tasks.
+structure. LocalSQL explores the opposite path: a specialized 7B model that runs
+locally and still competes with much larger frontier models on SQL tasks. With
+self-consistency it reaches the accuracy of a 1.6T frontier model while keeping
+your schema and rows on your own machine.
 
 ## Models
 
-| Adapter | Best For | Notes |
+| Model | Best For | Notes |
 |---|---|---|
-| `jk200201/qwen2.5-coder-7b-sql-dpo` | Spider-style/general SQLite questions | Default. 78.2% on Spider V1 dev. |
-| `jk200201/qwen2.5-coder-7b-bird-dpo` | Messier schemas plus domain hints | Better for BIRD-style questions with `--evidence`. |
+| `jk200201/qwen2.5-coder-7b-bird-cot` | General SQLite questions, messy schemas | Default. Merged CoT model. 52.1% greedy / 58.5% self-consistency on BIRD dev. |
+| `jk200201/qwen2.5-coder-7b-bird-cot-GGUF` | Running on a laptop (Ollama / llama.cpp) | Quantized builds (Q4_K_M is 4.4 GB). `ollama run muence/bird-cot`. |
 
-Both adapters sit on top of `Qwen/Qwen2.5-Coder-7B-Instruct`.
+The model was trained on top of `Qwen/Qwen2.5-Coder-7B-Instruct`.
 
 ## Install
 
@@ -79,18 +84,20 @@ python3 -m src.text2sql \
   --db mydata.sqlite \
   --q "top 5 customers by revenue"
 
-# Generate SQL only.
+# Generate SQL only (no execution).
 python3 -m src.text2sql \
   --db mydata.sqlite \
   --q "total revenue per month in 2023" \
   --no-exec
 
-# Use the BIRD adapter with a domain hint.
+# Add a domain hint for questions that need external knowledge.
 python3 -m src.text2sql \
   --db mydata.sqlite \
-  --adapter jk200201/qwen2.5-coder-7b-bird-dpo \
   --evidence "revenue = price * quantity" \
   --q "revenue by month"
+
+# Skip reasoning and use the direct-SQL prompt (faster, usually less accurate).
+python3 -m src.text2sql --db mydata.sqlite --q "top 5 customers" --direct
 ```
 
 As a library:
@@ -98,59 +105,66 @@ As a library:
 ```python
 from src.text2sql import load_model, predict
 
-model, tokenizer = load_model()
+model, tokenizer = load_model()  # loads the merged CoT model
 result = predict("mydata.sqlite", "top 5 customers by revenue", model, tokenizer)
 print(result["sql"])
+print(result["reasoning"])  # the step-by-step chain of thought
 print(result["rows"])
 ```
 
 ## Hosted Demo
 
-This repo includes a minimal Gradio app:
+This repo includes a Gradio app:
 
 ```bash
 python3 app.py
 ```
 
-The public Space runs against a generated sample SQLite database. For private
-databases, run the CLI locally so your schema and rows stay on your machine.
+The public Space runs against a generated sample SQLite database, and you can
+upload your own `.sqlite` file to try it on real data. Uploaded files are used
+only for the current query and are not stored. For fully private use, run the
+CLI locally so your schema and rows never leave your machine.
 
 ## Research Snapshot
 
-The training recipe is the interesting part:
-
-1. Run two frontier models on a SQL benchmark.
-2. Execute their SQL against the benchmark database.
-3. Where one model is right and the other wrong, turn that disagreement into a
-   DPO preference pair.
-4. SFT a 7B model on gold SQL, then DPO on the generated preference pairs.
+The training recipe is the interesting part. The current best model is a
+chain-of-thought SFT: distil step-by-step SQL reasoning from a strong teacher on
+the BIRD train split, keep only the correct chains, and fine-tune the 7B on
+them. It is then evaluated on the held-out BIRD dev split.
 
-Spider V1 dev:
+BIRD dev (execution accuracy):
 
-| Model | Result Accuracy |
-|---|---:|
-| LocalSQL Spider adapter | **78.2%** |
-| Grok-4 baseline | 73.7% |
-| DeepSeek-V3 baseline | 71.8% |
+| Model | Params | Result Accuracy |
+|---|---:|---:|
+| Qwen2.5-Coder-7B base | 7B | 27.0% |
+| LocalSQL, greedy | 7B | 52.1% |
+| LocalSQL, self-consistency (K=8) | 7B | 58.5% |
+| DeepSeek V4-Pro | 1.6T | 58.7% |
+| GLM 5.2 | 744B | 63.0% |
 
 For the broader experiment log, see `docs/research/`.
 
+> **Note on an earlier number.** A previous version of this project reported
+> 78.2% on Spider dev for a DPO model. That evaluation was later found to be
+> train/test contaminated (the DPO preference pairs were built from the same
+> dev split used for scoring), so that number has been retired. The BIRD results
+> above use a clean train/dev split.
+
 ## Roadmap
 
-- Hosted Hugging Face Space with sample SQLite databases.
-- High-accuracy mode using Best-of-N execution self-consistency.
+- Keep the hosted Space warm and let users upload their own SQLite files.
+- High-accuracy mode using Best-of-N execution self-consistency in the demo.
 - Schema search/linking for large databases.
 - Postgres and MySQL connectors.
-- Publish a clean BIRD CoT-SFT adapter if the 58.5% result is fully packaged and
-  reproducible.
+- On-policy RL (GRPO / rejection fine-tuning) to push past 58.5%.
 
 ## Citation
 
 ```bibtex
 @misc{kothari2026localsql,
   author = {Kothari, Jenish},
-  title  = {LocalSQL: A Local Text-to-SQL Model Trained with Frontier-Disagreement DPO},
+  title  = {LocalSQL: A Local Chain-of-Thought Text-to-SQL Model},
   year   = {2026},
-  howpublished = {\url{https://huggingface.co/jk200201/qwen2.5-coder-7b-sql-dpo}},
+  howpublished = {\url{https://huggingface.co/jk200201/qwen2.5-coder-7b-bird-cot}},
 }
 ```