README.md

---
base_model: Qwen/Qwen3.5-0.8B
license: apache-2.0
language:
- en
library_name: peft
pipeline_tag: text-generation
datasets:
- hetanshwaghela/autoscientist-healthcare-reasoning
tags:
- healthcare
- medical
- clinical-reasoning
- chain-of-thought
- grounded-generation
- hallucination-mitigation
- safety
- adaptive-data
- autoscientist
- lora
- qwen
- fine-tuned
---

# 🧪 AutoScientist — Healthcare Clinical-Reasoning (LoRA) + Experiment Log

> **Built with Adaptive Data by Adaption.**
> A `Qwen/Qwen3.5-0.8B` LoRA adapter trained on the adapted clinical-reasoning dataset —
> released together with the *honest, reproducible experiment* that produced it.

## TL;DR — read this first (integrity over hype)

This is a **research artifact**, not a leaderboard-beating model. On a held-out
LLM-judged win-rate (Gemini 3.1 pro, 200 samples) the **base model wins, 58/42**.
We report that plainly. The **contribution of this project is the dataset** (+30%
quality, Grade B→A — see the
[dataset card](https://huggingface.co/datasets/hetanshwaghela/autoscientist-healthcare-reasoning))
and the **rigorous finding** documented below.

## 📓 The AutoScientist's notebook

**Hypothesis.** Adapting a high-quality medical chain-of-thought dataset (+30% platform
quality) and SFT-ing a small model on it should beat the base model on held-out medical
reasoning.

**Experiment 1** — 20k rows, LoRA r=64, 3 epochs.
→ Base **58** / adapted **42**. Medical category: base **55** / adapted **46**.
→ Diagnostic: eval-loss plateaued after ~epoch 1 (1.559→1.525) while train-loss kept
falling (2.27→1.19) — **overfitting**.

**Experiment 2** — controlled follow-up: **60k rows**, **+40% general-purpose data**
(to counter catastrophic forgetting, per platform guidance), tuned recipe, more steps.
→ Base **62** / adapted **38**. It got **worse** — higher peak LR + more steps moved the
model *further* from a strong base.

**Conclusion (reproducible across two runs).**
> Supervised fine-tuning an *already-instruction-tuned* 0.8B model on long (~780-word)
> chain-of-thought makes it **more verbose**, and the judge prefers the base's crisper
> answers. **Dataset quality and small-model win-rate are different axes.** No
> hyperparameter or data-mix change flipped it — the effect is structural, not a bug.

This is the result the challenge is designed to surface: a clean, honest, reproducible
negative — the dataset is the win; the model is the documented experiment.

## Model description
- **Base model:** `Qwen/Qwen3.5-0.8B` — the exact model AutoScientist trained from
  (served via `togethercomputer/Qwen3.5-0.8B`). All numbers are relative to this base,
  identical prompts and settings.
- **Method:** AutoScientist, LoRA SFT, `train_on_inputs=false`.
- **Released recipe (Experiment 1, the stronger of the two):** r=64, α=128, dropout 0.05,
  target `q,k,v,o_proj`, 3 epochs, LR ~1.1e-4 (cosine, warmup 0.05), weight-decay 0.01,
  grad-clip 2. Final train-loss ≈ 1.19, eval-loss ≈ 1.53 (168 steps).
- **Weight format:** **LoRA adapter** (`adapter_model.safetensors` + `adapter_config.json`),
  base repointed to `Qwen/Qwen3.5-0.8B` for portable PEFT loading.
- **Language:** English. **Size:** ~0.8B base params.

## 📊 Evaluation (base vs. adapted) — honest
Judge = **Gemini 3.1 pro**, 200 held-out samples, identical prompts/settings.

| Win-rate (head-to-head) | Base `Qwen3.5-0.8B` | Adapted (this model) |
|---|---|---|
| On the dataset task | **58** | 42 |
| Medical category (all tasks) | **55** | 46 |

**Dataset quality (Adaptive Data, platform-measured):** +30% overall, Grade B→A,
completion quality +37.9%, message quality +17.6%, percentile 15.3→33.0.

## 🩺 What it's designed to do (safety blueprint)
Trained to *reason then answer*, stay grounded in evidence, hedge, recommend clinician
confirmation, escalate red-flag/emergency symptoms, refuse when uncertain, and preserve
numeric values exactly. (Design goals of the dataset; not a claim of clinical accuracy.)

## How to use
```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = "Qwen/Qwen3.5-0.8B"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base)
model = PeftModel.from_pretrained(model, "hetanshwaghela/autoscientist-healthcare-reasoning")
```

## ⚠️ Limitations & safety
- **Underperforms its base on the held-out judge** — do not treat as an improvement over
  `Qwen/Qwen3.5-0.8B`. Tends to be verbose.
- **Educational decision-support, not a medical device.** Not for individual diagnosis,
  treatment, or dosing without a qualified clinician. Escalate emergencies; preserve
  numeric values exactly.
- Inherits base-model and machine-generated-data limitations; English, exam-style skew.

## 🔁 Reproducibility
- Base: `Qwen/Qwen3.5-0.8B` (served as `togethercomputer/Qwen3.5-0.8B`)
- AutoScientist experiment id: `0eac225d-a25c-43b3-ae85-0549d5d08d8e`
- Adapted dataset id: `26048b57-f164-46d5-810b-12d498a76660` (20,000 rows)
- Dataset: https://huggingface.co/datasets/hetanshwaghela/autoscientist-healthcare-reasoning
- Kaggle model: https://www.kaggle.com/models/hetanshwaghela1/autoscientist-healthcare-reasoning
- 🧪 Live demo (HF Space): https://huggingface.co/spaces/hetanshwaghela/autoscientist-healthcare-demo

## Credit
**Built with Adaptive Data by Adaption.** Base: `Qwen/Qwen3.5-0.8B`. Foundation data:
`FreedomIntelligence/medical-o1-reasoning-SFT` (Apache-2.0). Public-health blend: CDC
public-domain text (*Source: Centers for Disease Control and Prevention; no CDC
endorsement is implied*).