Upload SmolLM2-1.7B QLoRA fine-tuned on Clinical NER (Chapter 7)

README.md

@@ -0,0 +1,146 @@
+---
+language:
+- en
+tags:
+- qlora
+- clinical-ner
+- smollm2
+- rearchitecting-llms
+- fine-tuning
+- medical
+- educational
+license: apache-2.0
+base_model: HuggingFaceTB/SmolLM2-1.7B-Instruct
+datasets:
+- oopere/clinical-ner-qdora
+metrics:
+- accuracy
+---
+
+# SmolLM2-1.7B-ClinicalNER
+
+## Model Description
+
+QLoRA fine-tuned version of **HuggingFaceTB/SmolLM2-1.7B-Instruct** for clinical named entity
+recognition (NER). Created as part of **Chapter 7** of *Rearchitecting LLMs*.
+
+* **Book:** [Rearchitecting LLMs](https://hubs.la/Q04k2VyY0)
+* **Technique:** QLoRA (Quantized Low-Rank Adaptation)
+* **Task:** Clinical NER — structured JSON extraction from clinical notes
+* **Chapter:** Chapter 7 — Specialization Tuning
+
+[![Rearchitecting LLMs](https://cdn-uploads.huggingface.co/production/uploads/640f7924f2d7c41a1e9eced1/sa4ivCbm8kk6C9NAPmb-x.jpeg)](https://hubs.la/Q040tvsK0)
+
+---
+
+## What This Model Does
+
+Given a free-text clinical note, the model extracts structured clinical entities
+into a strict JSON schema using only the two-word prompt `Extract:`.
+
+Before fine-tuning: a 15-line system prompt was required.
+After QLoRA training: the model responds correctly to `Extract:` alone.
+
+---
+
+## Schema Compliance Results
+
+Results from **CH07_NB02_L4_QLoRA_QDoRA** on the `oopere/clinical-ner-qdora` test set (40 samples, 5 categories).
+
+| Model | Prompt | Schema Compliance |
+|:---|:---|:---:|
+| SmolLM2-1.7B baseline | Strict (15-line prompt) | 87.5% |
+| SmolLM2-1.7B baseline | Minimal (`Extract:`) | 0.0% |
+| **SmolLM2-1.7B QLoRA (this model)** | **Minimal (`Extract:`)** | **95.0%** |
+
+Fine-tuning permanently absorbed the 15-line prompt into the model weights.
+
+---
+
+## Training Details
+
+### Dataset
+
+* **Source:** [oopere/clinical-ner-qdora](https://huggingface.co/datasets/oopere/clinical-ner-qdora)
+* **Train samples:** 200 (40 per category)
+* **Test samples:** 40 (8 per category)
+* **Categories:** clean, abbreviations, implicit, typos, irrelevant
+
+### QLoRA Hyperparameters
+
+| Parameter | Value |
+|:---|:---|
+| Base model | `HuggingFaceTB/SmolLM2-1.7B-Instruct` |
+| Quantization | NF4 4-bit + double quantization |
+| LoRA rank (r) | 8 |
+| LoRA alpha | 16 |
+| LoRA dropout | 0.05 |
+| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
+| Epochs | 3 |
+| Batch size | 8 |
+| Learning rate | 2e-4 |
+| LR scheduler | cosine |
+| Max sequence length | 512 |
+| Compute dtype | bfloat16 |
+
+### Hardware
+* **GPU:** NVIDIA L4 (Google Colab)
+
+---
+
+## How to Use
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+model_id = 'oopere/SmolLM2-1.7B-ClinicalNER'
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(model_id)
+model.eval()
+
+note = 'Patient 45yo male, fever and dry cough for 3 days. Temp 38.5C, HR 98, BP 120/80.'
+
+messages = [{'role': 'system', 'content': 'Extract:'}, {'role': 'user', 'content': note}]
+text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+inputs = tokenizer([text], return_tensors='pt').to(model.device)
+
+with torch.inference_mode():
+    output_ids = model.generate(**inputs, max_new_tokens=256, do_sample=False)
+
+new_tokens = output_ids[0][len(inputs.input_ids[0]):]
+print(tokenizer.decode(new_tokens, skip_special_tokens=True))
+```
+
+---
+
+## Limitations & Intended Use
+
+**Educational model** from *Rearchitecting LLMs* Chapter 7.
+Demonstrates QLoRA fine-tuning and the workflow: Train → Merge → Upload → Verify.
+
+**Not intended for clinical or production use.** Training data is synthetic.
+
+---
+
+## Citation
+
+```bibtex
+@book{martra2026rearchitecting,
+  author    = {Pere Martra},
+  title     = {Rearchitecting LLMs: Structural techniques for efficient models},
+  publisher = {Manning Publications},
+  year      = {2026},
+  url       = {https://hubs.la/Q040tvtp0}
+}
+```
+
+---
+
+## Acknowledgments
+
+Created following *Rearchitecting LLMs* (Manning, 2026).
+Challenge: can you push schema compliance above 95%?
+Try: higher LoRA rank, more epochs, or QDoRA instead.
+Share your results: [discussion forum](https://hubs.la/Q04k2VyY0)
+____

chat_template.jinja

@@ -0,0 +1,6 @@
+{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system
+You are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>
+' }}{% endif %}{{'<|im_start|>' + message['role'] + '
+' + message['content'] + '<|im_end|>' + '
+'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant
+' }}{% endif %}

config.json

@@ -0,0 +1,58 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "float32",
+  "eos_token_id": 2,
+  "head_dim": 64,
+  "hidden_act": "silu",
+  "hidden_size": 2048,
+  "initializer_range": 0.02,
+  "intermediate_size": 8192,
+  "max_position_embeddings": 8192,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 24,
+  "num_key_value_heads": 32,
+  "pad_token_id": 2,
+  "pretraining_tp": 1,
+  "quantization_config": {
+    "_load_in_4bit": true,
+    "_load_in_8bit": false,
+    "bnb_4bit_compute_dtype": "bfloat16",
+    "bnb_4bit_quant_storage": "uint8",
+    "bnb_4bit_quant_type": "nf4",
+    "bnb_4bit_use_double_quant": true,
+    "llm_int8_enable_fp32_cpu_offload": false,
+    "llm_int8_has_fp16_weight": false,
+    "llm_int8_skip_modules": null,
+    "llm_int8_threshold": 6.0,
+    "load_in_4bit": true,
+    "load_in_8bit": false,
+    "quant_method": "bitsandbytes"
+  },
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 130000,
+    "rope_type": "default"
+  },
+  "tie_word_embeddings": true,
+  "transformers.js_config": {
+    "dtype": "q4",
+    "kv_cache_dtype": {
+      "fp16": "float16",
+      "q4f16": "float16"
+    },
+    "use_external_data_format": {
+      "model.onnx": true,
+      "model_fp16.onnx": true
+    }
+  },
+  "transformers_version": "5.0.0",
+  "use_cache": false,
+  "vocab_size": 49152
+}

generation_config.json

@@ -0,0 +1,9 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": [
+    2
+  ],
+  "pad_token_id": 2,
+  "transformers_version": "5.0.0"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e31f67669af0dd06d9bfb1232d8b2617bb4def04101758950b6e032a741e172c
+size 1234259317

tokenizer_config.json

@@ -0,0 +1,17 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": "<|im_start|>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "is_local": false,
+  "model_max_length": 8192,
+  "pad_token": "<|im_end|>",
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": "<|endoftext|>",
+  "vocab_size": 49152
+}