---
license: apache-2.0
base_model: Qwen/Qwen2.5-0.5B-Instruct
tags:
  - qwen2.5
  - rag
  - knowledge-graph
  - json-extraction
  - summarization
  - fine-tuned
  - fp16
model_name: raging1.0-0.5B-fp16
---

# saji bsrs/raging1.0-0.5B-fp16

A lightweight 0.5B (494M) parameter model fine-tuned specifically for RAG ingestion pipelines. This is the full-precision FP16 fused version, suitable for conversion to GGUF, further quantization, or continued fine-tuning.

## Purpose

`raging1.0-0.5B-fp16` is a formatting specialist designed to sit at the front door of your RAG pipeline. It handles three core ingestion tasks:

1. Knowledge Graph Extraction - Entity-relation triples as `[subject, relation, object]` JSON arrays
2. Structured JSON Formatting - Strict schema adherence with correct data types
3. Concise Summarization - Factual compression in 1-3 sentences

For Apple Silicon users, a pre-quantized 4-bit MLX version is available at [sajibsrs/raging1.0-0.5B-MLX-4bit](https://huggingface.co/sajibsrs/raging1.0-0.5B-MLX-4bit).

## Benchmarks vs Base (Qwen2.5-0.5B-Instruct)

| Metric | Base | raging1.0-0.5B | Delta |
|--------|------|----------------|-------|
| KG Triple Precision | 0.33 | 0.67 | +103% |
| Throughput (tok/s) | 64 | 122 | +90% |
| TTFT (ms) | 680 | 338 | -50% |
| Output Length Stability | 22.1 | 17.1 | -23% |
| JSON Schema Accuracy | 100% | 100% | - |
| Summarization Compression | 0.69 | 0.68 | - |

Benchmarks run on Apple M4 using the 4-bit MLX quantization, 10-sample subset, averaged over 3 independent runs. FP16 quality is identical; speed metrics will vary by hardware.

## Quick Start

```python
from mlx_lm import load, generate

model, tokenizer = load("sajibsrs/raging1.0-0.5B-fp16")

messages = [
    {"role": "system", "content": "Extract entity-relation triples as JSON array of [subject, relation, object]. Keep elements 1-4 words. Output ONLY JSON array."},
    {"role": "user", "content": "Microsoft was founded by Bill Gates and Paul Allen in Albuquerque."}
]

prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
response = generate(model, tokenizer, prompt=prompt, max_tokens=256, verbose=False)
print(response)
```

## Training Details

- Base Model: Qwen/Qwen2.5-0.5B-Instruct
- Method: QLoRA (4-bit quantized base, LoRA rank 16, all 24 layers)
- Trainable Parameters: 2.93M (0.59%)
- Dataset: ~19.5K curated examples (summarization + structured JSON + KG extraction)
- Iterations: 1,000 (batch size 2, LR 1e-5)
- Hardware: Apple M4 Mac Mini (16GB unified memory)
- Framework: mlx-lm
- Post-training: LoRA adapters fused into base weights, saved as FP16

## Known Limitations

- KG Boundary Detection: May produce compound objects (e.g., "Bill Gates and Paul Allen in Albuquerque" instead of separate triples). Recommend post-processing with regex splitting.
- Pronoun Resolution: May output pronouns ("he", "the company") as subjects instead of resolved entity names.
- Not a Chat Model: Fine-tuned for single-turn structured extraction. Multi-turn conversational quality is degraded compared to base.
- JSON Complexity: Handles flat and simple nested schemas reliably. Deeply nested or conditional schemas may require fallback to a larger model.

## Recommended Pipeline Integration

Use `raging1.0-0.5B-fp16` as a fast first-pass processor. Validate outputs with Pydantic/jsonschema and fall back to a 7B+ model on validation failure:

```python
import json
from pydantic import BaseModel, ValidationError

def safe_extract(raw_output, schema):
    try:
        return schema.model_validate(json.loads(raw_output))
    except (json.JSONDecodeError, ValidationError):
        return None  # Fallback to larger model
```

## License

Apache 2.0 (inherited from Qwen2.5-0.5B-Instruct)

## Author

[sajibsrs](https://huggingface.co/sajibsrs)