Abogatic/qwen35-9b-legal-sft-es-v1

Qwen3.5-9B Legal SFT ES v1

LoRA adapter for Qwen/Qwen3.5-9B tuned for Spanish legal assistant behavior over statutes and case law, with emphasis on prudence, grounding, and faithful handling of visible legal text.

This is an adapter only. It must be loaded on top of the base model Qwen/Qwen3.5-9B.

Intended behavior

The fine-tuning objective was behavioral, not encyclopedic. The target model should:

• cite only what is visible in the provided material;
• avoid converting an old judicial citation into a statement of current positive law;
• explicitly recognize when the available text is partial or insufficient;
• write in a more prudent, structured, professional Spanish legal style.

The adapter was designed for RAG-style systems over Spanish legal sources, especially where the retrieved context may include:

• anonymized Spanish Supreme Court judgments;
• partial extracts of judgments;
• visible statutory fragments;
• mixed legal bundles where caution matters.

What was trained

The training corpus was built from anonymized Spanish Supreme Court judgments and transformed into chat-format SFT examples focused on:

• sentence analysis;
• prudent legal analysis;
• legal drafting style;
• autocontention and explicit limits;
• separation between facts, allegations, prior rulings, legal reasoning, and operative part;
• temporal normative caution when a judgment cites legal provisions.

The dataset was designed to reduce these failure modes:

• inventing articles, dates, ECLI, ROJ, or holdings not visible in the prompt;
• treating a provision cited inside an old judgment as if it were necessarily the current wording in force;
• overstating what can be concluded from partial context.

Training details

• Base model: Qwen/Qwen3.5-9B
• Training method: supervised fine-tuning with LoRA
• Training stack: Unsloth + TRL + PEFT
• Training profile: gold
• Train samples: 2045
• Validation samples: 258
• Runtime: 7477.95 seconds
• Final train loss: 2.4783
• Thinking mode during training: disabled

LoRA configuration

• Rank r: 16
• Alpha: 16
• Dropout: 0.0
• Bias: none
• Target modules:
  • q_proj
  • k_proj
  • v_proj
  • o_proj
  • gate_proj
  • up_proj
  • down_proj

Evaluation snapshot

An internal 60-case benchmark compared the base model against this adapter on Spanish legal behavior. The benchmark measured:

• fidelity to visible text;
• separation of legal levels;
• prudence / autocontention;
• legal analysis quality;
• writing quality;
• temporal normative hallucination risk.

Summary:

• Mean automatic score: 5.317 -> 5.500
• Fidelity to visible text: 0.917 -> 1.000
• Prudence / autocontention: 0.767 -> 0.883
• Temporal normative risk: 0.950 -> 1.000

Observed improvements in this benchmark:

• hidden holding hallucinations dropped from 1 to 0;
• non-visible article citations dropped from 4 to 0;
• improper current-validity language in jurisprudence-only contexts dropped from 3 to 0.

Benchmark artifacts included in this repository:

• benchmark_summary.json
• BENCHMARK.md

Known limitation:

• the adapter did not improve fine-grained separation between doctrine and antecedents; that behavior should still be reinforced at retrieval, prompt, and application-flow level.

Recommended use

Best fit:

• Spanish legal RAG over statutes and case law;
• systems that pass visible source excerpts to the model;
• structured legal summarization with explicit limits.

Recommended guardrails:

• instruct the model to answer only from retrieved material;
• distinguish between "norm cited by the judgment" and "currently visible statutory text";
• require explicit statements like no consta when the context is partial;
• prefer deterministic decoding for legal production workflows.

Not a full legal authority

This adapter is not a substitute for:

• legal source validation;
• current-law verification;
• professional legal review.

It should not be used to assert current legal validity unless the current statutory text is explicitly present in the retrieved context.

Usage with Transformers

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

base_model = "Qwen/Qwen3.5-9B"
adapter_path = "PATH/TO/ADAPTER"

tokenizer = AutoTokenizer.from_pretrained(base_model, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter_path)
model.eval()

Usage with vLLM

vllm serve Qwen/Qwen3.5-9B \
  --trust-remote-code \
  --enable-lora \
  --max-lora-rank 16 \
  --lora-modules '{"name":"legal-v1","path":"PATH/TO/ADAPTER","base_model_name":"Qwen/Qwen3.5-9B"}' \
  --generation-config vllm \
  --default-chat-template-kwargs '{"enable_thinking": false}'

Languages

• Spanish

Source and licensing notes

The adapter was trained from anonymized judicial material transformed into behavioral SFT examples. Review the legal and organizational requirements applicable to the source corpus before redistribution or downstream commercial use.

The adapter itself is released under the same practical serving assumptions as the base repository, but use remains subject to the base model license and the legal constraints of the training corpus.