Deploy bf16 LocalSQL Space for ZeroGPU

README.md

@@ -0,0 +1,156 @@
+---
+title: LocalSQL
+sdk: gradio
+app_file: app.py
+license: apache-2.0
+colorFrom: blue
+colorTo: green
+pinned: false
+---
+
+# LocalSQL
+
+Ask a SQLite database questions in plain English.
+
+LocalSQL is a small, open text-to-SQL tool built around fine-tuned
+Qwen2.5-Coder-7B LoRA adapters. The default adapter reached **78.2% result
+accuracy on Spider V1 dev**, beating the Grok-4 and DeepSeek-V3 baselines used
+to build its preference data.
+
+The product goal is simple:
+
+- **Local mode:** run the model on your own GPU, point it at your own `.sqlite`
+  file, and keep your data on your machine.
+- **Demo mode:** run the same model in a hosted Gradio app against sample
+  data so people can try it without setup.
+
+```bash
+python3 -m src.text2sql --db chinook.sqlite --q "Which 5 artists have the most albums?"
+```
+
+```text
+SQL
+SELECT ar.Name, COUNT(al.AlbumId) AS album_count
+FROM Artist ar
+JOIN Album al ON ar.ArtistId = al.ArtistId
+GROUP BY ar.ArtistId
+ORDER BY album_count DESC
+LIMIT 5;
+
+Results
+| Name         | album_count |
+|--------------|-------------|
+| Iron Maiden  | 21          |
+| Led Zeppelin | 14          |
+```
+
+## Why This Exists
+
+Most text-to-SQL tools are cloud wrappers: send schema plus question to a large
+API model, pay per request, and trust the remote provider with your database
+structure. LocalSQL explores the opposite path: a specialized 7B model that can
+run locally and still compete with much larger frontier models on SQL tasks.
+
+## Models
+
+| Adapter | Best For | Notes |
+|---|---|---|
+| `jk200201/qwen2.5-coder-7b-sql-dpo` | Spider-style/general SQLite questions | Default. 78.2% on Spider V1 dev. |
+| `jk200201/qwen2.5-coder-7b-bird-dpo` | Messier schemas plus domain hints | Better for BIRD-style questions with `--evidence`. |
+
+Both adapters sit on top of `Qwen/Qwen2.5-Coder-7B-Instruct`.
+
+## Install
+
+```bash
+git clone https://github.com/jenishk20/finetuning-text-to-sql
+cd finetuning-text-to-sql
+pip install -r requirements.txt
+```
+
+You need a CUDA GPU for 4-bit inference. A hosted demo should use a GPU-backed
+Hugging Face Space or an equivalent GPU service.
+
+## CLI Usage
+
+```bash
+# Generate SQL and execute it read-only against a SQLite database.
+python3 -m src.text2sql \
+  --db mydata.sqlite \
+  --q "top 5 customers by revenue"
+
+# Generate SQL only.
+python3 -m src.text2sql \
+  --db mydata.sqlite \
+  --q "total revenue per month in 2023" \
+  --no-exec
+
+# Use the BIRD adapter with a domain hint.
+python3 -m src.text2sql \
+  --db mydata.sqlite \
+  --adapter jk200201/qwen2.5-coder-7b-bird-dpo \
+  --evidence "revenue = price * quantity" \
+  --q "revenue by month"
+```
+
+As a library:
+
+```python
+from src.text2sql import load_model, predict
+
+model, tokenizer = load_model()
+result = predict("mydata.sqlite", "top 5 customers by revenue", model, tokenizer)
+print(result["sql"])
+print(result["rows"])
+```
+
+## Hosted Demo
+
+This repo includes a minimal Gradio app:
+
+```bash
+python3 app.py
+```
+
+The public Space runs against a generated sample SQLite database. For private
+databases, run the CLI locally so your schema and rows stay on your machine.
+
+## Research Snapshot
+
+The training recipe is the interesting part:
+
+1. Run two frontier models on a SQL benchmark.
+2. Execute their SQL against the benchmark database.
+3. Where one model is right and the other wrong, turn that disagreement into a
+   DPO preference pair.
+4. SFT a 7B model on gold SQL, then DPO on the generated preference pairs.
+
+Spider V1 dev:
+
+| Model | Result Accuracy |
+|---|---:|
+| LocalSQL Spider adapter | **78.2%** |
+| Grok-4 baseline | 73.7% |
+| DeepSeek-V3 baseline | 71.8% |
+
+For the broader experiment log, see `docs/research/`.
+
+## Roadmap
+
+- Hosted Hugging Face Space with sample SQLite databases.
+- High-accuracy mode using Best-of-N execution self-consistency.
+- Schema search/linking for large databases.
+- Postgres and MySQL connectors.
+- Publish a clean BIRD CoT-SFT adapter if the 58.5% result is fully packaged and
+  reproducible.
+
+## Citation
+
+```bibtex
+@misc{kothari2026localsql,
+  author = {Kothari, Jenish},
+  title  = {LocalSQL: A Local Text-to-SQL Model Trained with Frontier-Disagreement DPO},
+  year   = {2026},
+  howpublished = {\url{https://huggingface.co/jk200201/qwen2.5-coder-7b-sql-dpo}},
+}
+```

app.py

@@ -0,0 +1,194 @@
+"""
+Minimal Gradio demo for LocalSQL.
+
+Deploy this on a GPU-backed Space. The default sample database is generated at
+startup so the repo does not need to track a binary SQLite file.
+"""
+from __future__ import annotations
+
+import sqlite3
+import traceback
+import tempfile
+from functools import lru_cache
+from pathlib import Path
+
+import gradio as gr
+import pandas as pd
+
+from src.text2sql import DEFAULT_ADAPTER, DEFAULT_BASE, load_model, predict
+
+try:
+    import spaces
+except ImportError:
+    class _SpacesShim:
+        @staticmethod
+        def GPU(*args, **kwargs):
+            def decorator(fn):
+                return fn
+            return decorator
+
+    spaces = _SpacesShim()
+
+SAMPLE_ROOT = Path(tempfile.gettempdir()) / "localsql_samples"
+SAMPLE_DB = SAMPLE_ROOT / "music_store.sqlite"
+
+
+def ensure_sample_db() -> Path:
+    SAMPLE_ROOT.mkdir(parents=True, exist_ok=True)
+    if SAMPLE_DB.exists():
+        return SAMPLE_DB
+
+    conn = sqlite3.connect(SAMPLE_DB)
+    cur = conn.cursor()
+    cur.executescript(
+        """
+        CREATE TABLE Artist (
+            ArtistId INTEGER PRIMARY KEY,
+            Name TEXT NOT NULL
+        );
+        CREATE TABLE Album (
+            AlbumId INTEGER PRIMARY KEY,
+            Title TEXT NOT NULL,
+            ArtistId INTEGER NOT NULL,
+            FOREIGN KEY (ArtistId) REFERENCES Artist(ArtistId)
+        );
+        CREATE TABLE Track (
+            TrackId INTEGER PRIMARY KEY,
+            Name TEXT NOT NULL,
+            AlbumId INTEGER NOT NULL,
+            Milliseconds INTEGER,
+            UnitPrice REAL NOT NULL,
+            FOREIGN KEY (AlbumId) REFERENCES Album(AlbumId)
+        );
+        CREATE TABLE Customer (
+            CustomerId INTEGER PRIMARY KEY,
+            FirstName TEXT NOT NULL,
+            LastName TEXT NOT NULL,
+            Country TEXT NOT NULL
+        );
+        CREATE TABLE Invoice (
+            InvoiceId INTEGER PRIMARY KEY,
+            CustomerId INTEGER NOT NULL,
+            InvoiceDate TEXT NOT NULL,
+            Total REAL NOT NULL,
+            FOREIGN KEY (CustomerId) REFERENCES Customer(CustomerId)
+        );
+
+        INSERT INTO Artist VALUES
+            (1, 'Iron Maiden'),
+            (2, 'Led Zeppelin'),
+            (3, 'Miles Davis'),
+            (4, 'Nina Simone');
+        INSERT INTO Album VALUES
+            (1, 'Powerslave', 1),
+            (2, 'Seventh Son of a Seventh Son', 1),
+            (3, 'Led Zeppelin IV', 2),
+            (4, 'Kind of Blue', 3),
+            (5, 'I Put a Spell on You', 4);
+        INSERT INTO Track VALUES
+            (1, 'Aces High', 1, 271000, 0.99),
+            (2, '2 Minutes to Midnight', 1, 360000, 0.99),
+            (3, 'Stairway to Heaven', 3, 482000, 0.99),
+            (4, 'So What', 4, 545000, 1.29),
+            (5, 'Feeling Good', 5, 178000, 1.29);
+        INSERT INTO Customer VALUES
+            (1, 'Ada', 'Lovelace', 'United Kingdom'),
+            (2, 'Grace', 'Hopper', 'United States'),
+            (3, 'Katherine', 'Johnson', 'United States');
+        INSERT INTO Invoice VALUES
+            (1, 1, '2024-01-15', 12.87),
+            (2, 2, '2024-02-20', 22.74),
+            (3, 2, '2024-03-12', 8.91),
+            (4, 3, '2024-03-30', 14.85);
+        """
+    )
+    conn.commit()
+    conn.close()
+    return SAMPLE_DB
+
+
+@lru_cache(maxsize=1)
+def get_model():
+    print("Loading LocalSQL model in bf16 for ZeroGPU...", flush=True)
+    return load_model(DEFAULT_BASE, DEFAULT_ADAPTER, use_4bit=False)
+
+
+@spaces.GPU(duration=600)
+def answer(
+    question: str,
+    evidence: str,
+    run_sql: bool,
+):
+    if not question.strip():
+        return "", pd.DataFrame(), "Ask a question first.", ""
+
+    db_path = ensure_sample_db()
+    try:
+        model, tokenizer = get_model()
+        result = predict(
+            str(db_path),
+            question.strip(),
+            model,
+            tokenizer,
+            evidence=evidence.strip(),
+            execute=run_sql,
+        )
+    except Exception as exc:
+        traceback.print_exc()
+        message = f"{type(exc).__name__}: {exc!s}" if str(exc) else repr(exc)
+        return "", pd.DataFrame(), f"Model error: {message}", ""
+
+    frame = pd.DataFrame(result["rows"], columns=result["columns"]) if result["rows"] else pd.DataFrame()
+    status = result["error"] or f"{result['row_count']} rows"
+    return result["sql"], frame, status, result["schema"]
+
+
+with gr.Blocks(title="LocalSQL") as demo:
+    gr.Markdown(
+        "# LocalSQL\n"
+        "Ask a sample SQLite database questions in plain English. "
+        "This hosted demo uses sample data; run LocalSQL locally for private databases."
+    )
+    gr.Markdown(
+        "On ZeroGPU, the GPU is allocated only while a query is running. "
+        "The first query can take longer while the model downloads and loads."
+    )
+
+    question = gr.Textbox(
+        label="Question",
+        value="Which artists have the most albums?",
+        lines=2,
+    )
+
+    evidence = gr.Textbox(
+        label="Optional domain hint",
+        placeholder="Example: revenue = unit price * quantity",
+        lines=2,
+    )
+    run_sql = gr.Checkbox(label="Execute generated SQL", value=True)
+    submit = gr.Button("Ask LocalSQL", variant="primary")
+
+    sql = gr.Code(label="Generated SQL", language="sql")
+    rows = gr.Dataframe(label="Results", interactive=False)
+    status = gr.Textbox(label="Status")
+    schema = gr.Code(label="Schema", language="sql")
+
+    submit.click(
+        answer,
+        inputs=[question, evidence, run_sql],
+        outputs=[sql, rows, status, schema],
+    )
+
+    gr.Examples(
+        examples=[
+            ["Which artists have the most albums?", "", True],
+            ["List customers by total invoice amount.", "", True],
+            ["What is the longest track?", "", True],
+        ],
+        inputs=[question, evidence, run_sql],
+    )
+
+
+if __name__ == "__main__":
+    ensure_sample_db()
+    demo.launch()

docs/research/README.md

@@ -0,0 +1,33 @@
+# Research Notes
+
+This repository started as a text-to-SQL fine-tuning research project and is now
+being shaped into LocalSQL, a usable local/private SQL assistant.
+
+## Core Result
+
+The first strong result was Spider V1:
+
+- Fine-tuned Qwen2.5-Coder-7B: **78.2%** result accuracy.
+- Grok-4 baseline: 73.7%.
+- DeepSeek-V3 baseline: 71.8%.
+
+The method was frontier-disagreement DPO: run two strong models, execute their
+SQL, and turn correctness disagreements into DPO preference pairs.
+
+## BIRD Findings
+
+BIRD is harder because schemas are larger, train/dev databases are disjoint, and
+questions include external evidence. The important lessons so far:
+
+- Clear correctness pairs helped the 7B model.
+- Judge-resolved "both correct but stylistically different" pairs hurt BIRD,
+  because BIRD rewards execution correctness, not SQL style.
+- Reasoning distillation plus execution-based Best-of-N is the strongest story
+  to package next, pending final artifact verification.
+
+## Branches
+
+- `main`: public product/reproducible baseline branch.
+- `feat/localsql-product`: current cleanup branch for the LocalSQL product.
+- `feat/bird-7b-clean`: GRPO/execution-reward research branch.
+- `feat/excot-onpolicy`: reasoning-distillation and Best-of-N research branch.

model_cards/qwen2.5-coder-7b-bird-dpo.md

@@ -0,0 +1,148 @@
+---
+license: apache-2.0
+base_model: Qwen/Qwen2.5-Coder-7B-Instruct
+library_name: peft
+tags:
+- text-to-sql
+- sql
+- code-generation
+- bird
+- bird-sql
+- spider
+- dpo
+- qwen
+- qwen2.5
+- text-generation
+language:
+- en
+pipeline_tag: text-generation
+datasets:
+- birdsql/bird_mini_dev
+---
+
+# Qwen2.5-Coder-7B BIRD-DPO
+
+A LoRA adapter for **Qwen2.5-Coder-7B-Instruct** fine-tuned for real-world Text-to-SQL with **DPO using frontier model disagreements**. Achieves **50.3% on the BIRD dev set** — a +23.3pp improvement over the base model on real-world database queries with messy schemas and domain-specific evidence.
+
+## Results
+
+| Benchmark | Metric | Score |
+|---|---|---|
+| **BIRD dev** (1,534 Q) | Result accuracy | **50.3%** |
+| **BIRD mini-dev** (500 Q) | Result accuracy | **44.4%** |
+| BIRD mini-dev — Simple | Result accuracy | 62.8% |
+| BIRD mini-dev — Moderate | Result accuracy | 38.8% |
+| BIRD mini-dev — Challenging | Result accuracy | 31.4% |
+| **Spider V1 dev** (1,034 Q, cross-eval) | Result accuracy | **75.9%** |
+
+**Cross-domain transfer**: trained purely on BIRD, this adapter scores **75.9% on Spider** — only 2.3pp below a Spider-specific adapter (78.2%). Strong evidence that DPO from frontier disagreements generalizes across SQL benchmarks.
+
+| Model | BIRD dev |
+|---|---|
+| Qwen2.5-Coder-7B base | 27.0% |
+| Qwen2.5-Coder-7B + SFT only | ~35% |
+| **Qwen2.5-Coder-7B + SFT + BIRD-DPO (this model)** | **50.3%** |
+
+## Quick Start
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
+from peft import PeftModel
+import torch
+
+BASE_MODEL = "Qwen/Qwen2.5-Coder-7B-Instruct"
+ADAPTER    = "jk200201/qwen2.5-coder-7b-bird-dpo"
+
+tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL, trust_remote_code=True)
+
+bnb = BitsAndBytesConfig(
+    load_in_4bit=True, bnb_4bit_quant_type="nf4",
+    bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True,
+)
+model = AutoModelForCausalLM.from_pretrained(
+    BASE_MODEL, quantization_config=bnb, device_map="auto", trust_remote_code=True
+)
+model = PeftModel.from_pretrained(model, ADAPTER)
+model.eval()
+
+schema   = "CREATE TABLE users (id INT, name TEXT, country TEXT);"
+question = "How many users are from Japan?"
+evidence = ""  # optional domain hint for BIRD-style queries
+
+prompt = f"""Convert the following natural language question into a valid SQL query.
+
+Database Schema:
+{schema}
+
+{f'External Knowledge:{chr(10)}{evidence}{chr(10)}{chr(10)}' if evidence.strip() else ''}Question: {question}
+
+Return only the SQL query with no explanation."""
+
+inputs = tokenizer.apply_chat_template(
+    [{"role": "user", "content": prompt}],
+    return_tensors="pt", add_generation_prompt=True
+).to(model.device)
+
+out = model.generate(inputs, max_new_tokens=256, do_sample=False, pad_token_id=tokenizer.eos_token_id)
+sql = tokenizer.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True).strip()
+print(sql)
+```
+
+## Training Details
+
+**The novel idea**: rather than human-annotated preferences, this model uses **automatically generated preference pairs from frontier model disagreements** — total cost: ~$25 of OpenRouter API calls.
+
+### Pipeline
+
+1. Run **Grok-4.1-fast** and **DeepSeek-V3.2** on BIRD train (9,428 questions). Both score ~53%.
+2. Compare results question-by-question. Where one model is right and the other wrong → preference pair (1,219 pairs from BIRD).
+3. SFT Qwen2.5-Coder-7B on BIRD train gold SQL (QLoRA r=32, α=64, NF4 4-bit, 3 epochs).
+4. DPO on the 1,219 clear-preference pairs on top of SFT (β=0.05, 1 epoch, cutoff=8192).
+
+### Hyperparameters
+
+| Stage | Setting |
+|---|---|
+| Quantization | 4-bit NF4 (QLoRA) |
+| LoRA rank | 32 |
+| LoRA alpha | 64 |
+| LoRA dropout | 0.05 |
+| Target modules | q/k/v/o_proj, gate/up/down_proj |
+| SFT epochs | 3, LR 2e-4 cosine |
+| DPO epochs | 1, LR 5e-5 cosine, β=0.05 |
+| Cutoff length | 8,192 tokens (H200 80GB) |
+
+### Hardware
+
+Northeastern Discovery HPC — single NVIDIA H200 80GB. Training time: ~1h 15min.
+
+## Key Finding — What Doesn't Work
+
+Initially trained with 4,677 pairs (1,219 clear-preference + 3,458 judge-resolved style pairs). This **regressed to 40.7%** (-9.6pp).
+
+**Lesson**: judge-resolved pairs where both models are correct but write different SQL carry zero correctness signal for BIRD's result-accuracy metric. They dilute training and hurt performance. Only use pairs where one model is right and one is wrong.
+
+## Limitations
+
+- BIRD test set was not used (hidden); evaluation is on the public dev set
+- Real-world databases with very long schemas (>8K tokens) get truncated
+- Optimized for SQLite syntax (BIRD format); MySQL/PostgreSQL outputs may need adaptation
+- Trained only on English questions
+
+## Related Models
+
+- **Spider version** of this approach: [`jk200201/qwen2.5-coder-7b-sql-dpo`](https://huggingface.co/jk200201/qwen2.5-coder-7b-sql-dpo) — 78.2% on Spider V1, also beats Grok-4 and DeepSeek V3
+
+## Citation
+
+If you use this model, please cite:
+
+```bibtex
+@misc{kothari2026qwenbirddpo,
+  author       = {Kothari, Jenish},
+  title        = {Qwen2.5-Coder-7B BIRD-DPO: Frontier-Disagreement DPO for Real-World Text-to-SQL},
+  year         = {2026},
+  publisher    = {Hugging Face},
+  howpublished = {\url{https://huggingface.co/jk200201/qwen2.5-coder-7b-bird-dpo}},
+}
+```

model_cards/qwen2.5-coder-7b-sql-dpo.md

@@ -0,0 +1,134 @@
+---
+license: apache-2.0
+base_model: Qwen/Qwen2.5-Coder-7B-Instruct
+library_name: peft
+tags:
+- text-to-sql
+- sql
+- code-generation
+- spider
+- dpo
+- qwen
+- qwen2.5
+- text-generation
+language:
+- en
+pipeline_tag: text-generation
+datasets:
+- jk200201/spider-dpo-1040
+---
+
+# Qwen2.5-Coder-7B Spider-DPO
+
+A LoRA adapter for **Qwen2.5-Coder-7B-Instruct** fine-tuned with DPO that achieves **78.2% on Spider V1 dev** — outperforming Grok-4 (73.7%) and DeepSeek V3 (71.8%) despite being a 7B model.
+
+## Results
+
+| Model | Spider V1 dev | Parameters |
+|---|---|---|
+| **Qwen2.5-Coder-7B + Spider-DPO (this model)** | **78.2%** | 7B |
+| Grok-4 (frontier baseline) | 73.7% | unknown (very large) |
+| DeepSeek-V3 (frontier baseline) | 71.8% | 671B (37B active MoE) |
+| Qwen2.5-Coder-7B base | ~50% | 7B |
+
+### Cross-benchmark transfer
+
+| Benchmark | Score |
+|---|---|
+| Spider V1 dev (in-domain) | **78.2%** |
+
+For real-world database queries (BIRD-style schemas with evidence), use the companion model: [`jk200201/qwen2.5-coder-7b-bird-dpo`](https://huggingface.co/jk200201/qwen2.5-coder-7b-bird-dpo).
+
+## Quick Start
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
+from peft import PeftModel
+import torch
+
+BASE_MODEL = "Qwen/Qwen2.5-Coder-7B-Instruct"
+ADAPTER    = "jk200201/qwen2.5-coder-7b-sql-dpo"
+
+tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL, trust_remote_code=True)
+
+bnb = BitsAndBytesConfig(
+    load_in_4bit=True, bnb_4bit_quant_type="nf4",
+    bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True,
+)
+model = AutoModelForCausalLM.from_pretrained(
+    BASE_MODEL, quantization_config=bnb, device_map="auto", trust_remote_code=True
+)
+model = PeftModel.from_pretrained(model, ADAPTER)
+model.eval()
+
+schema   = "CREATE TABLE users (id INT, name TEXT, country TEXT);"
+question = "How many users are from Japan?"
+
+prompt = f"""Convert the following natural language question into a valid SQL query.
+
+Database Schema:
+{schema}
+
+Question: {question}
+
+Return only the SQL query with no explanation."""
+
+inputs = tokenizer.apply_chat_template(
+    [{"role": "user", "content": prompt}],
+    return_tensors="pt", add_generation_prompt=True
+).to(model.device)
+
+out = model.generate(inputs, max_new_tokens=256, do_sample=False, pad_token_id=tokenizer.eos_token_id)
+sql = tokenizer.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True).strip()
+print(sql)
+```
+
+## Training Details
+
+**The novel idea**: rather than human-annotated preferences, this model uses **automatically generated preference pairs from frontier model disagreements** — total cost: ~$25 of OpenRouter API calls.
+
+### Pipeline
+
+1. Run **Grok-4** and **DeepSeek-V3** on Spider dev set (1,034 questions).
+2. Compare against gold SQL question-by-question. Where one frontier model is right and the other wrong → preference pair (the correct SQL is "chosen", the wrong one "rejected").
+3. SFT Qwen2.5-Coder-7B on Spider train gold SQL (QLoRA r=32, α=64, NF4 4-bit, 3 epochs).
+4. DPO on **1,040 clear-preference pairs** on top of SFT (β=0.1, 2 epochs).
+
+### Hyperparameters
+
+| Stage | Setting |
+|---|---|
+| Quantization | 4-bit NF4 (QLoRA) |
+| LoRA rank | 32 |
+| LoRA alpha | 64 |
+| LoRA dropout | 0.05 |
+| Target modules | q/k/v/o_proj, gate/up/down_proj |
+| SFT epochs | 3, LR 2e-4 cosine |
+| DPO epochs | 2, LR 5e-5 cosine, β=0.1 |
+
+### Training data
+
+[`jk200201/spider-dpo-1040`](https://huggingface.co/datasets/jk200201/spider-dpo-1040) — 1,040 preference pairs built from Grok-4 vs DeepSeek-V3 disagreements on Spider dev.
+
+### Hardware
+
+AWS EC2 g5.xlarge (NVIDIA A10G 24GB VRAM). Training time: ~3h total.
+
+## Limitations
+
+- Designed for **Spider-style** queries: academic-style English, clean schemas, single SQLite dialect
+- For real-world messy databases with domain knowledge ("BIRD-style"), use [`jk200201/qwen2.5-coder-7b-bird-dpo`](https://huggingface.co/jk200201/qwen2.5-coder-7b-bird-dpo)
+- 4-bit quantized — for highest accuracy use bf16 base model
+- Trained only on English questions
+
+## Citation
+
+```bibtex
+@misc{kothari2026qwenspiderdpo,
+  author       = {Kothari, Jenish},
+  title        = {Qwen2.5-Coder-7B Spider-DPO: A 7B Model that Beats Frontier Models on Spider via Frontier-Disagreement DPO},
+  year         = {2026},
+  publisher    = {Hugging Face},
+  howpublished = {\url{https://huggingface.co/jk200201/qwen2.5-coder-7b-sql-dpo}},
+}
+```

requirements.txt

@@ -0,0 +1,9 @@
+pandas>=2.0.0
+tabulate>=0.9.0
+gradio>=4.44.0
+spaces
+
+torch>=2.4.0
+transformers>=4.48.0
+peft>=0.14.0
+accelerate>=1.0.0

src/__init__.py

@@ -0,0 +1 @@
+

src/bird/inference.py

@@ -0,0 +1,186 @@
+"""
+Shared inference utilities for BIRD pipelines.
+
+Provides:
+- build_instruction(): the canonical prompt format (must match training)
+- BIRDvLLMEngine: vLLM-backed batched generation with optional Best-of-N
+- _flash_attn_available(): helper used by trainer scripts
+
+This module is imported by:
+  - src.bird.eval_finetuned         (build_instruction)
+  - src.bird.build_sft_data         (build_instruction)
+  - src.bird.build_pairs            (build_instruction)
+  - src.bird.sft_train              (_flash_attn_available)
+  - src.bird.dpo_train              (_flash_attn_available)
+  - src.bird.eval_best_of_n         (BIRDvLLMEngine)
+  - src.bird.build_self_sampling_pairs (BIRDvLLMEngine)
+"""
+from __future__ import annotations
+
+import re
+from typing import Iterable
+
+
+# ─────────────────────────────────────────────────────────────────────────────
+# PROMPT — must match exactly what was used during DPO training
+# Single source of truth across the codebase
+# ─────────────────────────────────────────────────────────────────────────────
+
+def build_instruction(question: str, schema: str, evidence: str = "") -> str:
+    """Canonical BIRD prompt format. Used by training data builders + eval."""
+    evidence_block = f"External Knowledge:\n{evidence}\n\n" if evidence.strip() else ""
+    return (
+        "Convert the following natural language question into a valid SQL query.\n\n"
+        f"Database Schema:\n{schema}\n\n"
+        f"{evidence_block}"
+        f"Question: {question}\n\n"
+        "Return only the SQL query with no explanation."
+    )
+
+
+SYSTEM_PROMPT = (
+    "You are an expert SQL query generator. "
+    "Given a database schema and a natural language question, "
+    "generate a single valid SQL query that answers the question. "
+    "Output ONLY the SQL query, nothing else."
+)
+
+
+# ─────────────────────────────────────────────────────────────────────────────
+# OUTPUT POSTPROCESSING — strip markdown fences, add trailing semicolon
+# ─────────────────────────────────────────────────────────────────────────────
+
+_FENCE_RE = re.compile(r"```(?:sql)?\s*\n?(.*?)```", re.DOTALL)
+
+
+def extract_sql(text: str) -> str:
+    """Strip markdown fences, return SQL with trailing semicolon."""
+    text  = text.strip()
+    match = _FENCE_RE.search(text)
+    if match:
+        text = match.group(1).strip()
+    return text if text.endswith(";") else text + ";"
+
+
+# ─────────────────────────────────────────────────────────────────────────────
+# FLASH-ATTENTION HELPER — used by SFT + DPO trainers
+# ─────────────────────────────────────────────────────────────────────────────
+
+def _flash_attn_available() -> bool:
+    try:
+        import flash_attn  # noqa: F401
+        return True
+    except ImportError:
+        return False
+
+
+# ─────────────────────────────────────────────────────────────────────────────
+# vLLM BATCHED INFERENCE ENGINE
+# ─────────────────────────────────────────────────────────────────────────────
+
+class BIRDvLLMEngine:
+    """
+    vLLM-backed batched SQL generation. Supports K-sample generation for
+    Best-of-N inference and self-sampling preference pair generation.
+
+    Loads base model + (optional) LoRA adapter once. All subsequent
+    generate() calls reuse the loaded engine.
+
+    Usage:
+        engine = BIRDvLLMEngine(
+            base_model="Qwen/Qwen2.5-Coder-14B-Instruct",
+            adapter_path="/scratch/.../bird_sft_adapter_14b/checkpoint-3100",
+        )
+        # Each item is a tuple (question, schema, evidence)
+        prompts = [(q, s, e) for q, s, e in items]
+        # Returns: list of (list of K candidate SQL strings)
+        candidates_per_question = engine.generate(prompts, k=4, temperature=0.8)
+    """
+
+    def __init__(
+        self,
+        base_model: str,
+        adapter_path: str | None = None,
+        max_model_len: int = 8192,
+        gpu_memory_utilization: float = 0.90,
+        dtype: str = "bfloat16",
+    ):
+        from vllm import LLM
+        from vllm.lora.request import LoRARequest
+        from transformers import AutoTokenizer
+
+        self.tokenizer = AutoTokenizer.from_pretrained(base_model, trust_remote_code=True)
+        if self.tokenizer.pad_token is None:
+            self.tokenizer.pad_token = self.tokenizer.eos_token
+
+        self.llm = LLM(
+            model=base_model,
+            dtype=dtype,
+            max_model_len=max_model_len,
+            gpu_memory_utilization=gpu_memory_utilization,
+            enable_lora=adapter_path is not None,
+            max_lora_rank=64,  # safe upper bound; covers r=32 with margin
+            trust_remote_code=True,
+        )
+
+        self.lora_request = None
+        if adapter_path:
+            self.lora_request = LoRARequest(
+                lora_name="bird_adapter",
+                lora_int_id=1,
+                lora_path=adapter_path,
+            )
+            print(f"vLLM engine loaded with adapter: {adapter_path}")
+        else:
+            print(f"vLLM engine loaded (base model only, no adapter)")
+
+    def _build_chat_prompt(self, question: str, schema: str, evidence: str) -> str:
+        messages = [
+            {"role": "system", "content": SYSTEM_PROMPT},
+            {"role": "user",   "content": build_instruction(question, schema, evidence)},
+        ]
+        return self.tokenizer.apply_chat_template(
+            messages, tokenize=False, add_generation_prompt=True
+        )
+
+    def generate(
+        self,
+        items: Iterable[tuple[str, str, str]],
+        k: int = 1,
+        temperature: float = 0.0,
+        top_p: float = 1.0,
+        max_tokens: int = 512,
+    ) -> list[list[str]]:
+        """
+        Args:
+            items: iterable of (question, schema, evidence) tuples
+            k: number of candidates per question (1 = greedy, K>1 = sampling)
+            temperature: 0.0 for greedy, 0.7-1.0 for sampling
+            max_tokens: max generated tokens per candidate
+
+        Returns:
+            list of length len(items); each element is a list of K SQL strings
+        """
+        from vllm import SamplingParams
+
+        items = list(items)
+        prompts = [self._build_chat_prompt(q, s, e) for q, s, e in items]
+
+        sampling_params = SamplingParams(
+            n=k,
+            temperature=temperature if k > 1 else 0.0,
+            top_p=top_p,
+            max_tokens=max_tokens,
+        )
+
+        outputs = self.llm.generate(
+            prompts,
+            sampling_params=sampling_params,
+            lora_request=self.lora_request,
+        )
+
+        results: list[list[str]] = []
+        for out in outputs:
+            cands = [extract_sql(o.text) for o in out.outputs]
+            results.append(cands)
+        return results

src/shared/schema_loader.py

@@ -0,0 +1,45 @@
+"""
+Schema loader for Spider databases.
+
+Extracts CREATE TABLE DDL from SQLite database files for use in prompts.
+"""
+
+import sqlite3
+from pathlib import Path
+
+
+def get_schema_from_sqlite(db_path: str) -> str:
+    """
+    Extract CREATE TABLE statements directly from a SQLite database file.
+    This is the most reliable way since it matches the actual DB structure.
+    """
+    conn = sqlite3.connect(db_path)
+    cursor = conn.cursor()
+
+    cursor.execute(
+        "SELECT sql FROM sqlite_master WHERE type='table' AND sql IS NOT NULL"
+    )
+    tables = cursor.fetchall()
+    conn.close()
+
+    ddl_statements = []
+    for (sql,) in tables:
+        if sql and "sqlite_sequence" not in sql.lower():
+            ddl_statements.append(sql.strip() + ";")
+
+    return "\n\n".join(ddl_statements)
+
+
+def get_db_path(spider_data_dir: str, db_id: str) -> str:
+    """Resolve the full path to a Spider SQLite database file."""
+    base = Path(spider_data_dir) / "database" / db_id / f"{db_id}.sqlite"
+    if base.exists():
+        return str(base)
+
+    # Fallback: search for any .sqlite file in the directory
+    db_dir = Path(spider_data_dir) / "database" / db_id
+    if db_dir.exists():
+        for f in db_dir.glob("*.sqlite"):
+            return str(f)
+
+    raise FileNotFoundError(f"No SQLite database found for db_id={db_id} in {spider_data_dir}")

src/shared/sqlite_executor.py

@@ -0,0 +1,73 @@
+"""
+SQL executor for running queries against Spider's SQLite databases.
+"""
+
+import sqlite3
+import threading
+import time
+
+
+def execute_sqlite_query(sql: str, db_path: str, timeout: int = 30) -> dict:
+    """
+    Execute a SQL query against a SQLite database file.
+
+    Returns a dict with:
+      - "success": bool
+      - "columns": list of column names
+      - "rows": list of tuples
+      - "row_count": number of rows
+      - "execution_time_ms": time taken
+      - "error": error message (if failure)
+    """
+    conn = None
+    try:
+        conn = sqlite3.connect(db_path)
+        conn.execute("PRAGMA foreign_keys = ON")
+        conn.text_factory = str
+        cursor = conn.cursor()
+
+        # Enforce hard timeout via conn.interrupt() — the timeout parameter
+        # in sqlite3.connect() only covers lock acquisition, not query execution.
+        timer = threading.Timer(timeout, conn.interrupt)
+        timer.start()
+
+        start = time.time()
+
+        statements = [s.strip() for s in sql.split(";") if s.strip()]
+
+        columns = []
+        rows = []
+
+        try:
+            for stmt in statements:
+                cursor.execute(stmt)
+                if cursor.description:
+                    columns = [desc[0] for desc in cursor.description]
+                    rows = cursor.fetchall()
+        finally:
+            timer.cancel()
+
+        elapsed_ms = round((time.time() - start) * 1000, 2)
+
+        return {
+            "success": True,
+            "columns": columns,
+            "rows": rows,
+            "row_count": len(rows),
+            "execution_time_ms": elapsed_ms,
+            "error": None,
+        }
+
+    except Exception as e:
+        return {
+            "success": False,
+            "columns": [],
+            "rows": [],
+            "row_count": 0,
+            "execution_time_ms": 0,
+            "error": str(e),
+        }
+
+    finally:
+        if conn:
+            conn.close()

src/text2sql.py

@@ -0,0 +1,230 @@
+"""
+Local text-to-SQL over SQLite databases.
+
+The public product path is:
+  - load a Qwen2.5-Coder-7B base model plus a SQL LoRA adapter
+  - introspect a SQLite schema
+  - generate SQL from a natural-language question
+  - optionally execute only read-only SQL against the database
+"""
+from __future__ import annotations
+
+import argparse
+import re
+import sys
+from pathlib import Path
+from typing import Any
+
+import torch
+from peft import PeftModel
+from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
+
+from src.bird.inference import build_instruction, extract_sql
+from src.shared.schema_loader import get_schema_from_sqlite
+from src.shared.sqlite_executor import execute_sqlite_query
+
+DEFAULT_BASE = "Qwen/Qwen2.5-Coder-7B-Instruct"
+DEFAULT_ADAPTER = "jk200201/qwen2.5-coder-7b-sql-dpo"
+
+ADAPTERS = {
+    "spider": "jk200201/qwen2.5-coder-7b-sql-dpo",
+    "bird": "jk200201/qwen2.5-coder-7b-bird-dpo",
+    "base": "",
+}
+
+
+def resolve_adapter(adapter: str | None) -> str | None:
+    """Accept a shortcut, HF repo id, local path, empty string, or None."""
+    if adapter is None:
+        return DEFAULT_ADAPTER
+    if adapter in ADAPTERS:
+        return ADAPTERS[adapter] or None
+    return adapter or None
+
+
+def load_model(
+    base_model: str = DEFAULT_BASE,
+    adapter: str | None = DEFAULT_ADAPTER,
+    use_4bit: bool = True,
+):
+    """Load the base model plus an optional LoRA adapter."""
+    adapter = resolve_adapter(adapter)
+
+    if use_4bit and not torch.cuda.is_available():
+        raise RuntimeError(
+            "4-bit inference needs a CUDA GPU. On Hugging Face Spaces, open "
+            "Settings -> Hardware and choose a GPU such as Nvidia L4 before "
+            "running the demo."
+        )
+
+    print(
+        f"Loading {base_model}" + (f" + {adapter}" if adapter else " (base only)"),
+        file=sys.stderr,
+    )
+    tokenizer = AutoTokenizer.from_pretrained(base_model, trust_remote_code=True)
+    if tokenizer.pad_token is None:
+        tokenizer.pad_token = tokenizer.eos_token
+
+    kwargs: dict[str, Any] = {"trust_remote_code": True, "device_map": "auto"}
+    if use_4bit:
+        kwargs["quantization_config"] = BitsAndBytesConfig(
+            load_in_4bit=True,
+            bnb_4bit_quant_type="nf4",
+            bnb_4bit_compute_dtype=torch.bfloat16,
+            bnb_4bit_use_double_quant=True,
+        )
+    else:
+        kwargs["torch_dtype"] = torch.bfloat16
+
+    model = AutoModelForCausalLM.from_pretrained(base_model, **kwargs)
+    if adapter:
+        model = PeftModel.from_pretrained(model, adapter)
+    model.eval()
+    return model, tokenizer
+
+
+def generate_sql(
+    model,
+    tokenizer,
+    schema: str,
+    question: str,
+    evidence: str = "",
+    max_new_tokens: int = 256,
+) -> str:
+    """Build the training-time prompt, greedily decode, and return SQL."""
+    instruction = build_instruction(question, schema, evidence)
+    inputs = tokenizer.apply_chat_template(
+        [{"role": "user", "content": instruction}],
+        return_tensors="pt",
+        add_generation_prompt=True,
+    ).to(model.device)
+
+    with torch.no_grad():
+        outputs = model.generate(
+            inputs,
+            max_new_tokens=max_new_tokens,
+            do_sample=False,
+            pad_token_id=tokenizer.eos_token_id,
+        )
+    raw = tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True)
+    return extract_sql(raw)
+
+
+def is_read_only_sql(sql: str) -> bool:
+    """
+    Conservative guard for product execution.
+
+    We allow SELECT/WITH queries only and reject obvious mutating statements.
+    This keeps the CLI/demo from modifying a user's database by accident.
+    """
+    cleaned = re.sub(r"--.*?$|/\*.*?\*/", "", sql, flags=re.MULTILINE | re.DOTALL).strip()
+    if not cleaned:
+        return False
+    first_token = cleaned.split(None, 1)[0].lower().rstrip(";")
+    if first_token not in {"select", "with"}:
+        return False
+    blocked = re.search(
+        r"\b(insert|update|delete|drop|alter|create|replace|attach|detach|vacuum|pragma)\b",
+        cleaned,
+        flags=re.IGNORECASE,
+    )
+    return blocked is None
+
+
+def predict(
+    db_path: str,
+    question: str,
+    model,
+    tokenizer,
+    evidence: str = "",
+    execute: bool = True,
+    max_new_tokens: int = 256,
+) -> dict:
+    """Introspect schema, generate SQL, and optionally run it read-only."""
+    schema = get_schema_from_sqlite(db_path)
+    sql = generate_sql(model, tokenizer, schema, question, evidence, max_new_tokens)
+
+    result = {
+        "sql": sql,
+        "columns": [],
+        "rows": [],
+        "row_count": 0,
+        "error": None,
+        "schema": schema,
+    }
+
+    if execute:
+        if not is_read_only_sql(sql):
+            result["error"] = "Refusing to execute non-read-only SQL. Use --no-exec to inspect it."
+            return result
+        exec_out = execute_sqlite_query(sql, db_path)
+        result.update(
+            columns=exec_out["columns"],
+            rows=exec_out["rows"],
+            row_count=exec_out["row_count"],
+            error=exec_out["error"],
+        )
+    return result
+
+
+def _print_result(result: dict) -> None:
+    print("\nSQL")
+    print(result["sql"])
+
+    if result["error"]:
+        print("\nError")
+        print(result["error"])
+        return
+
+    print(f"\nResults ({result['row_count']} rows)")
+    try:
+        from tabulate import tabulate
+
+        print(tabulate(result["rows"][:50], headers=result["columns"], tablefmt="github"))
+    except ImportError:
+        print(result["columns"])
+        for row in result["rows"][:50]:
+            print(row)
+    if result["row_count"] > 50:
+        print(f"... ({result['row_count'] - 50} more rows)")
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description="Local text-to-SQL over a SQLite database")
+    parser.add_argument("--db", required=True, help="Path to a .sqlite database file")
+    parser.add_argument("--q", "--question", dest="question", required=True)
+    parser.add_argument(
+        "--adapter",
+        default=DEFAULT_ADAPTER,
+        help="LoRA adapter, local path, or shortcut: spider, bird, base",
+    )
+    parser.add_argument("--base-model", default=DEFAULT_BASE)
+    parser.add_argument("--evidence", default="", help="Optional BIRD-style domain hint")
+    parser.add_argument("--bf16", action="store_true", help="Load in bf16 instead of 4-bit")
+    parser.add_argument("--no-exec", action="store_true", help="Generate SQL without running it")
+    parser.add_argument("--max-new-tokens", type=int, default=256)
+    args = parser.parse_args()
+
+    db_path = Path(args.db)
+    if not db_path.exists():
+        sys.exit(f"Database not found: {db_path}")
+
+    model, tokenizer = load_model(
+        base_model=args.base_model,
+        adapter=args.adapter,
+        use_4bit=not args.bf16,
+    )
+    result = predict(
+        str(db_path),
+        args.question,
+        model,
+        tokenizer,
+        evidence=args.evidence,
+        execute=not args.no_exec,
+        max_new_tokens=args.max_new_tokens,
+    )
+    _print_result(result)
+
+
+if __name__ == "__main__":
+    main()