Instructions to use abhishekai/slm-500m-legal-sft with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use abhishekai/slm-500m-legal-sft with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="abhishekai/slm-500m-legal-sft")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("abhishekai/slm-500m-legal-sft")
model = AutoModelForCausalLM.from_pretrained("abhishekai/slm-500m-legal-sft", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use abhishekai/slm-500m-legal-sft with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "abhishekai/slm-500m-legal-sft"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "abhishekai/slm-500m-legal-sft",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/abhishekai/slm-500m-legal-sft

SGLang

How to use abhishekai/slm-500m-legal-sft with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "abhishekai/slm-500m-legal-sft" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "abhishekai/slm-500m-legal-sft",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "abhishekai/slm-500m-legal-sft" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "abhishekai/slm-500m-legal-sft",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use abhishekai/slm-500m-legal-sft with Docker Model Runner:
```
docker model run hf.co/abhishekai/slm-500m-legal-sft
```

slm-500m-legal-sft

A 517.8M-parameter Llama-style model fine-tuned for grounded (RAFT-style) question answering over US legal and financial text.

It answers from a passage you supply. It is not a closed-book model.

Provenance — read this first

This is a fine-tune of someone else's pretrained base, not a from-scratch model:


base model	thesreedath/slm-500m-base (Apache-2.0)
base pretraining	2.08B tokens, 5 epochs, 8×B200, val ppl 7.91, bits-per-byte 0.588
base pretraining cost	$115.69, paid by the base's author — not by this fine-tune
what we added	Q&A SFT on 9,178 curated grounded pairs, 3 epochs, 1×H100
our cost	$0.91

The base was pretrained on a legal/financial corpus (~40% US case law, ~40% SEC filings, ~20% educational web text), MinHash-deduplicated and decontaminated against CaseHOLD/LexGLUE.

Why this exists

The base author already publishes a Q&A fine-tune, thesreedath/slm-500m-qa. On our prompt distribution it did not answer questions — it continued the source passage. Measured, not assumed:

check	`slm-500m-qa`	this model
answers a grounded question	no — continues the passage	yes
fails at greedy decoding too	yes	—
fails under 4 different prompt formats	yes	—
usable preference pairs mined from 4 samples/prompt	54 / 1175 (4.6%)	1048 / 1380 (76%)

That is not a criticism of the base or of their fine-tune — their reported QA-SFT perplexity of 5.41 is a fine number on their own Q&A data. Perplexity does not measure instruction-following on a different task mix. Ours is heavier on summarize / rewrite / list / extract over 700–3,200 character passages.

Training data

9,178 train / 799 val curated grounded Q&A pairs, generated with Gemini 2.5 Flash and filtered by: embedding dedup, teacher grounding self-check, length/format gates, task+difficulty balancing, and n-gram + embedding decontamination against the eval split. Loss is masked to the answer tokens only.

Results

metric	value
answer-token validation perplexity	3.43
training	3 epochs, 3,441 steps, 13.8 min, 1×H100
tokens / epoch	6.82M (389K supervised)

For reference, a 125M model trained on the identical data reaches 3.36 — but on a 16K vocab versus this model's 32K. A larger vocab packs more text per token and makes each prediction harder, so 3.43 here is the better result in tokenizer-invariant terms. Compare bits-per-byte, not perplexity, across different tokenizers.

Usage

The user turn must contain the passage. Use greedy or low temperature.

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

m = "abhishekai/slm-500m-legal-sft"
tok = AutoTokenizer.from_pretrained(m)
model = AutoModelForCausalLM.from_pretrained(m).eval()

question = "What standard of proof applies to the plaintiff?"
context = ("In a civil negligence action the plaintiff must prove duty, breach, "
           "causation, and damages by a preponderance of the evidence.")
messages = [
    {"role": "system", "content": "You are a precise legal and financial assistant. "
     "Answer using only the provided context. If the context does not contain the "
     "answer, say you cannot answer from the context."},
    {"role": "user", "content": f"{question}\n\nContext:\n{context}"},
]
text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
ids = tok(text, add_special_tokens=False, return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=160, do_sample=False,
                     eos_token_id=tok.convert_tokens_to_ids("<|eos|>"),
                     pad_token_id=tok.convert_tokens_to_ids("<|pad|>"))
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))

Limitations

Grounded only. Bare closed-book questions are out of distribution.
No arithmetic reliability. At this scale it will state a figure and a derived percentage that do not follow from each other. Do not trust generated numbers.
Synthetic training data. Q&A pairs were written by Gemini 2.5 Flash and inherit its biases and errors, filtered but not eliminated.
Not legal or financial advice. Outputs are not a substitute for a qualified professional.
1,024-token context.

License

Apache-2.0, inherited from thesreedath/slm-500m-base.

Downloads last month: 17

Safetensors

Model size

0.5B params

Tensor type

F32

Model tree for abhishekai/slm-500m-legal-sft

Base model

thesreedath/slm-500m-base

Finetuned

(3)

this model

Finetunes

3 models