legal-slm-500m-sft-onnx

ONNX export of DeependraVerma/legal-slm-500m-sft, dynamically quantized to int8, built to run entirely inside a web browser via transformers.js (@huggingface/transformers v3) — no server, no API, no cost. This is the scaled-up sibling of DeependraVerma/legal-slm-125m-sft-onnx, which powers the original demo's in-browser mode.

Same weights, same architecture, same fine-tuning as the PyTorch SFT model — this repo just packages them for WebAssembly inference. See DeependraVerma/legal-slm-500m-sft for the full model card (training data, procedure, the full evaluation comparison against the 125M model, and the documented limitations); this page covers the ONNX-specific details.

• Repo / full build: github.com/DeependraVerma/legal-slm-125M
• Author: Deependra Verma — Generative AI Researcher / AI Engineer (Hugging Face)
• Source (PyTorch) model: DeependraVerma/legal-slm-500m-sft
• 125M sibling: DeependraVerma/legal-slm-125m-sft-onnx

Read the limitations section below (and on the source model card) before using this for anything real. Open-book contract clause extraction and classification (given an excerpt in the prompt) is the reliable mode. Closed-book general legal/financial Q&A (no excerpt given) is meaningfully better than the 125M model but still not reliable — see the source model card's documented regressions. Never use its output as legal, financial, or factual advice.

What's in this repo

File	What
onnx/model.onnx + onnx/model.onnx_data	fp32 ONNX export, split across two files — at 528.5M parameters the fp32 weights (~2.1GB) exceed ONNX's single-protobuf-file limit (2GB), so the exporter split the weights into a separate .onnx_data file alongside the graph
onnx/model_quantized.onnx	dynamically quantized int8, ~524MB, self-contained (no external data file) — this is the one the browser demo actually loads
tokenizer.json, tokenizer_config.json, special_tokens_map.json	the project's own 16,384-vocab byte-level BPE tokenizer (identical to the 125M build's — deliberately shared, not retrained)
config.json	the 500M LlamaConfig (24L/1152d/18h, RoPE θ=10,000, SwiGLU 4,608, 1,024 ctx, 16,384 vocab)

Why this exists

Same reasoning as the 125M ONNX export: Hugging Face's free serverless Inference API doesn't host arbitrary custom models, so shipping weights to the browser is the only genuinely free, zero-infrastructure way to let the public run this model. Worth knowing plainly: 528.5M parameters is a noticeably heavier in-browser payload than the 125M model — expect a larger one-time download (~524MB vs. the 125M model's ~133MB) and slower per-token generation on the visitor's own device, since WebAssembly has no GPU acceleration here.

Verified end-to-end with a real Node.js transformers.js test, not just a smoke test of the export step: given a contract excerpt about a 30-day termination notice and asked about the termination terms, it correctly extracted the relevant text from the excerpt.

How to use (transformers.js, in the browser)

import { AutoTokenizer, AutoModelForCausalLM, TextStreamer, env } from "@huggingface/transformers";

const ONNX_REPO = "DeependraVerma/legal-slm-500m-sft-onnx";
const SYSTEM_PROMPT =
  "You are a knowledgeable legal and financial assistant. Answer accurately and concisely.";

env.allowLocalModels = false;

const tokenizer = await AutoTokenizer.from_pretrained(ONNX_REPO);
const model = await AutoModelForCausalLM.from_pretrained(ONNX_REPO, {
  dtype: "q8",       // loads onnx/model_quantized.onnx
  device: "wasm",
  progress_callback: (p) => {
    if (p?.status === "progress" && p?.total) {
      console.log(`Loading: ${Math.round((p.loaded / p.total) * 100)}%`);
    }
  },
});

// Open-book (recommended): fold a contract excerpt into the user turn
const excerpt = "This Agreement may be terminated by either party upon 30 days written notice...";
const message = `${excerpt}\n\nQuestion: What are the termination terms of this agreement?`;
const prompt = `<|bos|><|system|>${SYSTEM_PROMPT}<|user|>${message}<|assistant|>`;
const inputs = tokenizer(prompt, { add_special_tokens: false });
const eos = tokenizer.model.tokens_to_ids.get("<|eos|>");

const streamer = new TextStreamer(tokenizer, {
  skip_prompt: true,
  skip_special_tokens: true,
  callback_function: (token) => process.stdout.write(token),
});

await model.generate({
  ...inputs,
  max_new_tokens: 128,   // browser WASM is single-threaded — keep replies snappy
  do_sample: true,
  temperature: 0.7,
  top_k: 50,
  top_p: 0.9,
  eos_token_id: eos,
  streamer,
});

Known limitations — read before using

Inherited directly from DeependraVerma/legal-slm-500m-sft (quantization changes weights, not behavior at this level of precision loss). Full held-out evaluation, and a direct paired comparison against the 125M model on the identical 1,059 validation questions, found:

• Open-book contract tasks work well — better than the 125M model on both CUAD extraction (F1 0.761 vs 0.711) and LEDGAR classification (77.4% vs 74.1%).
• Closed-book general Q&A improved meaningfully but is still not reliable. Better than the 125M model in every category (case law 42.2% vs 35.6%, SEC 46.9% vs 43.1%, general web 30.3% vs 18.2%), but genuine regressions exist alongside the fixes — bigger and better on net does not mean strictly better on every question. A 528.5M-parameter model can store at most ~2 bits of knowledge per parameter (Allen-Zhu & Li, "Physics of Language Models: Knowledge Capacity Scaling Laws"), a hard capacity ceiling, not a training defect.

Never use this model's output as legal, financial, or factual advice. Always treat specific claims as unverified until checked against a primary source — especially in closed-book use (no excerpt provided).

Citation

@misc{verma2026legalslm500msft,
  author = {Deependra Verma},
  title  = {legal-slm-500m-sft: A Fine-Tuned Q\&A Assistant on a From-Scratch 528.5M Legal/Financial Language Model},
  year   = {2026},
  url    = {https://huggingface.co/DeependraVerma/legal-slm-500m-sft},
  note   = {Code: https://github.com/DeependraVerma/legal-slm-125M}
}

Author

Deependra Verma — Generative AI Researcher / AI Engineer. GitHub · Hugging Face

License

MIT — see LICENSE in the source repo. This is a research artifact, not a source of legal or financial advice.