---
license: llama3.3
base_model: meta-llama/Llama-3.3-70B-Instruct
datasets:
- flamiinngo/personal-finance-advice-qa
tags:
- personal-finance
- finance
- lora
- peft
- adapter
- autoscientist
- adaption
language:
- en
pipeline_tag: text-generation
library_name: peft
---

# Personal Finance Advice — Llama-3.3-70B LoRA

A LoRA adapter for **Llama-3.3-70B-Instruct**, fine-tuned to answer practical personal
finance questions: debt and credit, retirement, investing, budgeting, tax, insurance,
savings, and estate planning.

Trained with **Adaption Labs' AutoScientist** for the AutoScientist Challenge
(Personal Finance category).

## Result

Head-to-head win rate against the base model:

| Evaluation | Base | Adapted |
|---|---|---|
| Personal Finance category | 25 | **76** |
| In-distribution test set | 57 | 43 |

These are wins in a paired comparison, not accuracy percentages.

The two rows disagree, and that is worth explaining rather than hiding. The training
answers are terse extracted action-blocks that read as fragments, so on the narrow
in-distribution set a judge often prefers the base model's fluent prose. Across the
wider category the adapted model wins decisively — the substance transferred even
though the presentation did not.

## Usage

The adapter is stored unpacked, so it loads directly.

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE = "meta-llama/Llama-3.3-70B-Instruct"
ADAPTER = "flamiinngo/adaption_personal_finance_advice_qa"

tokenizer = AutoTokenizer.from_pretrained(BASE)
model = AutoModelForCausalLM.from_pretrained(
    BASE, torch_dtype=torch.bfloat16, device_map="auto"
)
model = PeftModel.from_pretrained(model, ADAPTER)
model.eval()

messages = [{"role": "user", "content":
    "I have $8,000 in credit card debt at 22% APR and $5,000 in savings "
    "earning 4%. Should I pay off the card?"}]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt").to(model.device)

out = model.generate(inputs, max_new_tokens=300, do_sample=False)
print(tokenizer.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))
```

**Hardware:** the 70B base needs roughly 140 GB in bf16, or about 40 GB with 4-bit
quantisation. The adapter itself is 3.3 GB.

**Note on the base model name.** `adapter_config.json` records
`togethercomputer/Meta-Llama-3.3-70B-Instruct-Reference`, which is how the base was
served during training. It is the same architecture — load against
`meta-llama/Llama-3.3-70B-Instruct` as shown above.

## Training

| Parameter | Value |
|---|---|
| Base | `meta-llama/Llama-3.3-70B-Instruct` |
| Rank (`r`) | 64 |
| `lora_alpha` | 128 |
| `lora_dropout` | 0 |
| Target modules | all-linear |
| Peak learning rate | 1e-4 |
| Schedule | cosine, warmup 0.05, weight decay 0.05 |
| Epochs | 4 |

Trained on 11,639 rows after AutoScientist augmentation — 90% personal-finance by
its own domain classification, with the remainder legal and career-adjacent.

## Dataset

[**flamiinngo/personal-finance-advice-qa**](https://huggingface.co/datasets/flamiinngo/personal-finance-advice-qa)
— 3,805 real questions from r/personalfinance and r/FinancialPlanning, with answers cut
to their actionable core (median 77 words). Derived from
[Akhil-Theerthala/Personal-Finance-Queries](https://huggingface.co/datasets/Akhil-Theerthala/Personal-Finance-Queries) (MIT).

Also on Kaggle:
[model](https://www.kaggle.com/models/flamiinngo/adaption_pers-ad2ce402-880c-42c9-976d-4511b6e11f0d) ·
[dataset](https://www.kaggle.com/datasets/flamiinngo/personal-finance-advice-qa)

## Limitations

- **Not financial advice.** This model produces plausible guidance, not professional
  advice. Do not use it to make financial decisions.
- **US-centric.** IRS forms, 401(k) rules, FHA loans, US credit scoring. Little of it
  transfers to other jurisdictions.
- **Time-sensitive.** Contribution limits, tax thresholds and rates change; some
  training content reflects conditions that have since moved.
- **The training answers were not written by financial professionals.** The questions
  are real Reddit posts. The upstream dataset describes its contents as "Reddit posts
  and top comments" but the answer text reads as model-generated; the source does not
  document which. Either way, no verified professional wrote them.
- **It can state figures confidently and be wrong.** Treat any number it gives as
  unverified.
- **Win rate is not accuracy.** It measures preference against one base model on one
  evaluation.
- **English only.**

## License

The adapter is a derivative of Llama-3.3-70B-Instruct and is subject to the
**Llama 3.3 Community License**. The training data is MIT.

## Acknowledgements

- **Adaption Labs** — AutoScientist platform and the challenge
- **Akhil Theerthala** — upstream personal finance dataset
- **Meta** — Llama 3.3 base model