---
license: apache-2.0
base_model: openai/gpt-oss-20b
tags:
- peft
- lora
- medical
- azerbaijani
- reasoning
- healthcare
library_name: peft
language:
- az
pipeline_tag: text-generation
datasets:
- LocalDoc/medical-o1-reasoning-SFT-azerbaijani
---

# GPT-OSS 20B Medical Reasoning (Azerbaijani)

This is a fine-tuned version of the OpenAI GPT-OSS 20B model specialized for medical reasoning and diagnostic tasks in the Azerbaijani language.
Article about FT - https://medium.com/@vrashad/gpt-oss-20b-modelinin-az%C9%99rbaycan-tibbi-dataseti-il%C9%99-fine-tuning-edilm%C9%99si-c9abf4819ede

## Model Details

- **Base Model**: openai/gpt-oss-20b
- **Adapter Type**: LoRA (Low-Rank Adaptation)
- **Language**: Azerbaijani
- **Domain**: Medical reasoning and diagnosis
- **Training Dataset**: Medical Q&A reasoning dataset (19,000 examples)

## Usage

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

# Load base model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("openai/gpt-oss-20b")
base_model = AutoModelForCausalLM.from_pretrained(
    "openai/gpt-oss-20b",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Load PEFT adapter
model = PeftModel.from_pretrained(base_model, "vrashad/gpt-oss-20b-medical-az")
model = model.merge_and_unload()

# Generate response
messages = [
    {"role": "system", "content": "reasoning language: Azerbaijani"},
    {"role": "user", "content": "Ürək ağrısının səbəbləri nə ola bilər?"},
]

input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

with torch.no_grad():
    output_ids = model.generate(
        input_ids,
        max_new_tokens=2048,
        do_sample=True,
        temperature=0.6,
        top_p=0.9,
    )

response = tokenizer.decode(
    output_ids[0][input_ids.shape[1]:], 
    skip_special_tokens=True
)
print(response)
```

## Training Details

- **Training Method**: LoRA fine-tuning
- **Rank (r)**: 16
- **Alpha**: 32
- **Target Modules**: Attention and MLP layers
- **Training Epochs**: 1
- **Learning Rate**: 1e-4
- **Batch Size**: 16 (effective)

## Dataset

The training dataset consists of medical questions and answers with reasoning. Each sample follows this format:
- System prompt: "reasoning language: Azerbaijani"  
- User question: Medical question in Azerbaijani
- Assistant response: `<thinking>reasoning process</thinking>final answer`


## Limitations

- Optimized specifically for Azerbaijani language
- Intended for educational purposes only, not for medical advice
- Professional medical consultation is required for real medical decisions