Question about your legal DPO work on AMALIA
Hi, I came across your legal-domain DPO LoRAs on AMALIA while researching how people run preference training on open-weight models. I'm researching tooling for exactly this kind of work, so to be upfront, I'm not after anything competitive, just trying to learn from teams doing it hands-on. I'm curious how you build preference pairs for legal quality, and how you figure out whether a run actually changed the model the way you wanted before you ship it.
Would you be open to a quick chat? Happy to do email if easier, I'm at austin@aureliusaligned.ai.
Thanks either way,
Austin
Hi Austin,
Thanks for reaching out, happy to share what I can.
One correction first, these are SFT LoRAs, not DPO. The "DPO" in the repo name (AMALIA-9B-0626-DPO) refers to the base checkpoint I built on top of (AMALIA team's own preference), not something I did myself on the legal side.
On your two questions:
- Building the training pairs, since it's SFT rather than preference pairs, the shape is a bit different. Positive examples take a question and real source excerpts as input; the target is an answer that cites the exact article, built directly from Portuguese Diário da República's own amendment records. The questions come from real amending laws' official summaries, the answers are templated citations of the real target text. No LLM generation involved anywhere in that pipeline. Negative examples pair a real question with deliberately unrelated excerpts, with a fixed refusal as the target. Both are gold-by-construction rather than model graded or human ranked.
- Knowing whether a run actually worked, I rely on deterministic, code-based checkers rather than an LLM judge, using a logic of does the response contain a citation tag pointing at an excerpt that was actually shown, does it correctly decline when the excerpts don't cover the question. I always compare against an untrained baseline and at least one general purpose reference model on the same benchmark before trusting a result. Most recently I started testing through the live retrieval service instead of a fixed offline eval set, which surfaced a failure mode (refusal calibration on off-topic questions) the static benchmark hadn't caught.
Most of this is written up publicly if it's useful. The model cards for legal-v2 and legal-v1 on HF have the full methodology and results.
Any additional questions let me know.
Best,
Filipe
Hi Filipe, thanks for the correction and the detail, this is helpful. Rather than going back and forth in a thread, would you be up for a quick 15 minute call? I'm less interested in any one technique and more in getting a feel for how you work end to end, and I have a couple of questions about building on top of AMALIA's preference-tuned base that are easier live. Shoot me an email at austin@aureliusaligned.ai and we'll find a time.
Best,
Austin
Hi Austin,
Email sent.
Best,
Filipe