Update README.md

README.md

@@ -1,21 +1,113 @@
 ---
-base_model: unsloth/qwen2.5-3b-instruct-unsloth-bnb-4bit
-tags:
-- text-generation-inference
-- transformers
-- unsloth
-- qwen2
+base_model: unsloth/Qwen2.5-3B-Instruct
+library_name: transformers
+pipeline_tag: text-generation
 license: apache-2.0
 language:
 - en
+tags:
+- text-to-sql
+- sql
+- qwen2
+- lora
+- qlora
+- unsloth
+- trl
+datasets:
+- b-mc2/sql-create-context
 ---
 
-# Uploaded finetuned  model
+# Qwen2.5-3B Text-to-SQL
+
+A fine-tuned version of **Qwen2.5-3B-Instruct** that converts natural-language
+questions into SQL queries, given a database schema. Trained with LoRA/QLoRA on a
+single free GPU.
 
 - **Developed by:** ashishsahu2008
+- **Base model:** [unsloth/Qwen2.5-3B-Instruct](https://huggingface.co/unsloth/Qwen2.5-3B-Instruct)
+- **Fine-tuned on:** [b-mc2/sql-create-context](https://huggingface.co/datasets/b-mc2/sql-create-context)
 - **License:** apache-2.0
-- **Finetuned from model :** unsloth/qwen2.5-3b-instruct-unsloth-bnb-4bit
+- **Live demo:** https://huggingface.co/spaces/ashishsahu2008/text2sql-demo
+- **Code / write-up:** https://github.com/ashishsahu2008/fine-tuning
+
+## What it does
+
+Give it a `CREATE TABLE` schema and a plain-English question, and it returns only
+the SQL query that answers the question — no explanations or markdown, just runnable SQL.
+
+## Results
+
+Evaluated on a held-out test split (n=200) the model never saw during training.
+Both the base and fine-tuned models were scored with identical string normalization
+(lowercase, strip `;`, canonicalize quotes), so the comparison is apples-to-apples.
+
+| Model                          | Exact-match accuracy |
+|--------------------------------|:--------------------:|
+| Qwen2.5-3B-Instruct (base)     |        41.5%         |
+| **This model (fine-tuned)**    |      **72.5%**       |
+
+Only 0.96% of parameters (~30M of 3.1B) were trained via LoRA. Much of the gain
+comes from the base model learning to emit clean, executable SQL instead of wrapping
+answers in prose or markdown fences.
+
+> **Note on the metric:** exact-match undercounts semantically-correct queries that
+> are written differently (column order, aliases, whitespace). A stronger evaluation
+> would use AST comparison (`sqlglot`) or execution accuracy on a benchmark with
+> populated databases (Spider / BIRD); `sql-create-context` ships schemas only.
+
+## Usage
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "ashishsahu2008/qwen2.5-3b-text2sql"
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16)
+
+schema = "CREATE TABLE employees (name VARCHAR, salary INTEGER, department VARCHAR)"
+question = "List the names of employees in Sales earning over 50000."
+
+messages = [
+    {"role": "system", "content": "You are a SQL expert. Given a database schema "
+                                   "and a question, output only the SQL query that answers it."},
+    {"role": "user", "content": f"Schema:\n{schema}\n\nQuestion: {question}"},
+]
+inputs = tokenizer.apply_chat_template(
+    messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
+).to(model.device)
+out = model.generate(**inputs, max_new_tokens=128, do_sample=False)
+print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
+```
+
+## Prompt format
+
+The model expects the schema and question in a single user turn, with this system prompt:
+
+```
+System:  You are a SQL expert. Given a database schema and a question, output only the SQL query that answers it.
+User:    Schema:
+         <CREATE TABLE ...>
+
+         Question: <your question>
+```
+
+## Training details
+
+- **Method:** supervised fine-tuning (SFT) with LoRA on a 4-bit quantized base (QLoRA)
+- **LoRA config:** rank 16, alpha 16, applied to attention + MLP projections
+- **Data:** 3,000-row subset of `b-mc2/sql-create-context` (2,700 train / 300 test)
+- **Hyperparameters:** 2 epochs, learning rate 2e-4, effective batch size 8
+- **Hardware:** single Colab T4, ~25 minutes
+- **Frameworks:** [Unsloth](https://github.com/unslothai/unsloth) + TRL `SFTTrainer`
+
+## Limitations
 
-This qwen2 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
+- Trained on **single-table** `CREATE TABLE` schemas; complex multi-join databases
+  are out of distribution.
+- Assumes the schema is provided in the prompt — it does not know any tables you
+  don't give it.
+- SQL dialect follows the training data (broadly SQLite-compatible); it does not
+  target a specific engine's extensions.
 
-[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
+Fine-tuned with [Unsloth](https://github.com/unslothai/unsloth) and Hugging Face's TRL library.