Rumiii/LiquiMedThink1.2B

Fine Tuned LiquidAI/LFM2.5-1.2B-Thinking on a medical reasoning dataset to adapt to medical domain while retaining its thinking

LiquiMedThink 1.2B

LiquiMedThink 1.2B is a fine-tuned version of LiquidAI/LFM2.5-1.2B-Thinking, adapted for medical chain-of-thought reasoning. The model retains the base model's explicit reasoning traces — producing visible step-by-step thinking before arriving at a final answer — while developing clinical domain knowledge through supervised fine-tuning on a purpose-built medical reasoning dataset.

This is the third and final iteration of a structured fine-tuning research project. Two prior runs established that dataset format is the primary determinant of whether thinking ability is retained or lost during domain-specific SFT. This version uses a dataset with genuinely separate reasoning traces and final answers, which successfully preserved the model's thinking behaviour.

---

Model Details

Property	Value
Base model	LiquidAI/LFM2.5-1.2B-Thinking
Parameters	1.17B
Architecture	Hybrid LIV convolution + GQA (16 layers)
Fine-tuning method	QLoRA (4-bit) via Unsloth
Trainable parameters	9,142,272 (0.78% of total)
LoRA rank	16
LoRA alpha	16
Max sequence length	4096 tokens
Training dataset	FreedomIntelligence/medical-o1-reasoning-SFT (English, 19,704 samples)
Training epochs	1
Effective batch size	16 (2 per device x 8 gradient accumulation)
Learning rate	2e-4 (cosine schedule)
Optimizer	AdamW 8-bit
Hardware	Kaggle Tesla T4 (15.6 GB VRAM)
Training time	Approximately 3 hours
Final training loss	1.718

---

Training Data

The model was trained on the English split of FreedomIntelligence/medical-o1-reasoning-SFT, a dataset containing 19,704 clinical questions paired with two distinct components per sample:

• Complex_CoT — a genuine internal reasoning trace written as exploratory, step-by-step clinical thinking, including uncertainty, self-correction, and differential reasoning
• Response — a clean, concise final answer separate in both content and style from the reasoning trace

Each training sample was structured as follows:

User:      [clinical question]
Assistant: <think>
           [Complex_CoT — the full reasoning trace]
           </think>

           [Response — the final answer]

This structure is the critical difference from prior fine-tuning attempts on this model. Using a dataset where the reasoning trace and final answer are genuinely different content — rather than artificially reformatted versions of the same text — is what preserved the base model's thinking behaviour through fine-tuning.

No truncation was applied in preprocessing. The 4096 token sequence length was chosen to accommodate the full length of Complex_CoT entries, which are substantially longer than standard question-answer pairs.

---

Intended Use

This model is intended for:

• Research into small language model capabilities in healthcare and clinical reasoning
• Medical education and clinical reasoning demonstration
• Prototyping and evaluation of medical AI pipelines
• Study of how fine-tuning data format affects reasoning behaviour in instruction-tuned models

This model is not intended for:

• Clinical decision support in real patient care
• Diagnostic or treatment decisions in any clinical setting
• Replacement of licensed medical professionals
• Any high-stakes medical application without expert oversight

---

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
import torch, re

model_id = "Rumiii/LiquiMedThink1.2B"

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    quantization_config=BitsAndBytesConfig(
        load_in_4bit=True,
        bnb_4bit_compute_dtype=torch.float16,
        bnb_4bit_use_double_quant=True,
        bnb_4bit_quant_type="nf4",
    ),
    device_map="auto",
)
model.eval()

question = "A 45-year-old male presents with crushing chest pain radiating to the left arm, diaphoresis, and shortness of breath for 30 minutes. What is the most likely diagnosis and immediate management?"

inputs = tokenizer.apply_chat_template(
    [{"role": "user", "content": question}],
    add_generation_prompt=True,
    tokenize=True,
    return_tensors="pt",
    return_dict=True,
).to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=2048,
        temperature=0.05,
        top_k=50,
        repetition_penalty=1.05,
        do_sample=True,
        pad_token_id=tokenizer.eos_token_id,
    )

raw = tokenizer.decode(
    outputs[0][inputs["input_ids"].shape[-1]:],
    skip_special_tokens=False,
)

# Parse thinking trace and final answer
match = re.search(r"<think>(.*?)</think>(.*)", raw, re.DOTALL)
if match:
    thinking = match.group(1).strip()
    answer   = re.sub(r"<[^>]+>", "", match.group(2)).strip()
    print("THINKING:\n", thinking)
    print("\nANSWER:\n", answer)
else:
    print(re.sub(r"<[^>]+>", "", raw).strip())

---

Sample Output

Question: A 45-year-old male presents with crushing chest pain radiating to the left arm, diaphoresis, and shortness of breath for 30 minutes. What is the most likely diagnosis and immediate management?

Thinking trace (abridged): The presentation of crushing chest pain radiating to the left arm, combined with diaphoresis and shortness of breath, strongly points toward an acute coronary event. The duration of 30 minutes is significant — this exceeds the threshold for transient ischemia and raises concern for myocardial infarction. I need to consider STEMI versus NSTEMI and think through the immediate priorities...

Final answer: The most likely diagnosis is acute myocardial infarction, specifically STEMI given the classic presentation. Immediate management includes activating emergency services, obtaining a 12-lead ECG, administering aspirin 300mg, establishing IV access, and arranging urgent percutaneous coronary intervention within 90 minutes of first medical contact...

---

Research Background

This model is the result of a three-run fine-tuning research project investigating the relationship between training data format and reasoning trace retention in small language models.

Run 1 used the OctoMed/II-Medical-SFT dataset (40,000 samples) with raw question-answer pairs and no thinking tags. The model lost its thinking ability entirely, answering directly without producing reasoning traces.

Run 2 used the same dataset but artificially wrapped answers inside thinking tags. Thinking tags appeared in the output, but the thinking trace and final answer were identical in content — the model learned the structural pattern without learning the semantic distinction between reasoning and conclusion.

Run 3 (this model) used FreedomIntelligence/medical-o1-reasoning-SFT, where Complex_CoT and Response are genuinely different content written for different purposes. This successfully preserved the base model's thinking behaviour while adding medical domain knowledge.

The primary finding is that thinking ability retention during SFT is determined by dataset format, not model capacity. The same 1.2B model produced three qualitatively different reasoning behaviours across three runs that were otherwise identical in architecture, training configuration, and hardware.

---

Limitations

Clinical accuracy is not guaranteed. As a 1.2B parameter model trained for one epoch on approximately 20,000 samples, this model can produce plausible-sounding but clinically incorrect responses, particularly for complex, rare, or atypical presentations. All outputs must be verified by a qualified medical professional before any practical use.

Thinking traces are exploratory, not authoritative. The model's internal reasoning reflects the style of the training data — it thinks out loud, entertains multiple hypotheses, and sometimes reaches incorrect conclusions. The thinking trace should be read as the model's reasoning process, not as verified clinical logic.

Vague or underspecified questions reduce accuracy. The model reasons better when given structured clinical context (patient age, specific symptoms, duration, relevant history) than when given open-ended or conversational symptom descriptions.

Single-epoch training. One epoch over approximately 20,000 samples limits how thoroughly medical reasoning patterns are consolidated. Performance on rare conditions, highly specialised clinical topics, or complex multi-system presentations will be weaker than on common presentations.

Knowledge cutoff. The base model's knowledge cutoff is mid-2024. Clinical guidelines, drug approvals, or research published after this date are not reflected in the model's responses.

English only. Fine-tuning was conducted exclusively on English-language data.

---

Training Infrastructure

Fine-tuning was conducted on Kaggle using the Tesla T4 GPU (15.6 GB VRAM). The Unsloth library was used for memory-efficient 4-bit QLoRA training. Sequence packing was enabled to improve GPU utilisation on variable-length samples without modifying sample content. Gradient checkpointing and 8-bit AdamW were used to maintain VRAM stability at the 4096 token sequence length.

---

Author

Rumi Iqbal Sufi Graduate Trainee, Excelra Knowledge Solutions, Hyderabad HuggingFace: Rumiii GitHub: sufirumii

---

License

Apache 2.0, consistent with the base model license.