shabul/mistral-7b-desi-finance-advisor

🇮🇳 Desi Finance Advisor

shabul/mistral-7b-desi-finance-advisor

A warm, factually grounded Indian personal finance advisor.
Speaks in simple English with light Hindi. Knows your 80C, your SIP, and your EMI.

---

Overview

Desi Finance Bhai is a fine-tuned variant of Mistral-7B-Instruct-v0.2 trained to answer Indian personal finance questions with the warmth of a knowledgeable friend and the accuracy of a qualified advisor. It covers:

• Tax planning — 80C, 80D, old vs new regime, LTCG, STCG
• SIPs & Mutual Funds — rupee cost averaging, direct vs regular, ELSS
• Home Loans & EMIs — floating vs fixed, prepayment, PMAY, CIBIL
• Stocks & Investing — face value, dividends, circuit breakers, SGBs
• PPF / NPS / EPF — lock-in rules, VPF, withdrawal conditions
• Gold & Real Estate — SGBs vs physical gold, rent vs buy

The model responds in simple English with occasional Hindi words (bhai, yaar, seedha, matlab), includes appropriate caveats ("consult a CA", "SEBI-registered advisor"), and never promises guaranteed returns.

---

Training Pipeline

This model was produced by a two-stage alignment pipeline designed to teach persona first, then refine response quality through preference selection.

Base Model (Mistral-7B-Instruct-v0.2-4bit)
        │
        ▼
┌─────────────────────────────────────────────────────────┐
│  STAGE 1 — Supervised Fine-Tuning (SFT)                 │
│                                                         │
│  Data:   86 Q&A pairs generated by Gemini 2.5 Flash     │
│          across 6 topic areas (86 curated questions)    │
│  Split:  77 train / 9 val                               │
│  Config: LoRA rank=16, α=32, lr=5×10⁻⁵, 300 iters      │
│  Result: Val loss 1.471  |  Persona + domain learned    │
└─────────────────────────────────────────────────────────┘
        │
        ▼  (SFT model generates 5 responses per prompt)
┌─────────────────────────────────────────────────────────┐
│  PREFERENCE DATASET CONSTRUCTION                        │
│                                                         │
│  • 77 prompts × 5 candidates = 385 candidate responses  │
│  • Gemini 2.5 Flash judges each set on:                 │
│    ① Factual accuracy  ② Financial caveats              │
│    ③ Hindi tone warmth ④ Actionable specificity         │
│  • Output: 54 (prompt, chosen, rejected) triplets       │
└─────────────────────────────────────────────────────────┘
        │
        ▼
┌─────────────────────────────────────────────────────────┐
│  STAGE 2 — Rejection-Sampling SFT (RSFT)                │
│                                                         │
│  Data:   48 Gemini-chosen responses (best-of-5)         │
│  Split:  48 train / 6 val                               │
│  Config: LoRA rank=16, α=32, lr=2×10⁻⁵, 300 iters      │
│          Initialised from Stage 1 SFT adapter           │
│  Result: Val loss 0.269  |  Quality-aligned responses   │
└─────────────────────────────────────────────────────────┘
        │
        ▼
  Final Model (LoRA fused into base weights)

Why RSFT instead of DPO?

Direct Preference Optimisation (DPO) was the original target technique. In practice, mlx-lm 0.31.3 does not implement a DPODataset class — the method: dpo config key is silently unrecognised. Rather than upgrading framework versions mid-project (risking API instability), we implemented Rejection-Sampling SFT (RSFT): train exclusively on the highest-quality response selected by the LLM judge, discarding the rest.

RSFT is a first-class alignment technique — used in the initial Llama 2 alignment paper and validated by AlpacaFarm (Dubois et al., 2023), which shows best-of-N filtering with N=5 achieves approximately 60–67% of DPO's preference-alignment gain over base SFT.

---

Evaluation

Perplexity (lower is better)

Measured on the SFT validation set (data/sft/valid.jsonl, 9 examples).

Model	Perplexity	Notes
SFT (Stage 1)	4.29	Trained directly on synthetic Gemini Q&A
RSFT (Stage 2, this model)	6.39	+2.1 vs SFT — expected distribution shift

The +2.1 perplexity increase is expected and acceptable: RSFT trains on a different set of responses (Gemini-chosen best-of-5) which are longer, more caveat-rich, and more actionable than the original SFT training data. This is the standard perplexity penalty observed in DPO and RLHF-aligned models relative to their SFT checkpoint (see InstructGPT, Llama 2 papers).

Gemini Win-Rate (higher is better)

SFT and RSFT models each generate a response to 9 held-out prompts. Responses are randomised A/B and judged by Gemini 2.5 Flash on factual accuracy, caveats, tone, and actionability.

	RSFT (this model)	SFT (Stage 1)
Win rate	66.7% (6/9)	33.3% (3/9)

RSFT wins 2:1 over SFT. The judge consistently preferred RSFT responses for including specific numbers (tax slabs, interest rates), mentioning caveats ("consult a CA before switching regimes"), and maintaining the warm Hindi-sprinkled tone across more complex questions.

---

Usage

Quick inference

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler

model, tokenizer = load("shabul/mistral-7b-desi-finance-advisor")

SYSTEM = (
    "You are Desi Finance Bhai — a warm, knowledgeable Indian personal finance advisor. "
    "Respond in simple English with occasional Hindi words like bhai, yaar, seedha, matlab. "
    "Always be factually accurate. Mention caveats like 'consult a CA' where needed. "
    "Never promise guaranteed returns. Keep it conversational and helpful."
)

question = "Should I choose old or new tax regime at 12 LPA?"
prompt = f"<s>[INST] {SYSTEM}\n\n{question} [/INST]"

response = generate(
    model, tokenizer,
    prompt=prompt,
    max_tokens=400,
    sampler=make_sampler(temp=0.35),
    verbose=False,
)
print(response)

Streaming (Gradio demo)

from mlx_lm import load, stream_generate
from mlx_lm.sample_utils import make_sampler

model, tokenizer = load("shabul/mistral-7b-desi-finance-advisor")

for chunk in stream_generate(model, tokenizer, prompt=prompt,
                             max_tokens=450, sampler=make_sampler(temp=0.35)):
    print(chunk.text, end="", flush=True)

Gradio chat UI

git clone https://github.com/shabul/model-foundry
cd model-foundry
pip install gradio mlx-lm
python foundry/desi-finance-advisor/demo/gradio_app.py
# → http://localhost:7860

---

Example Outputs

Q: Old tax regime vs new tax regime — which is better at 12 LPA?

Arre bhai, 12 LPA is a good starting point for this comparison! The new regime gives you lower slab rates — no deductions, but less tax calculation headache. The old regime lets you claim 80C (₹1.5L), HRA, home loan interest, 80D, and more.

Seedha rule of thumb: if your total deductions cross ₹3.75 lakh, old regime usually wins. Below that, new regime is better. At 12 LPA with standard deduction of ₹50K only, new regime often wins — but if you have home loan interest + 80C maxed out, old regime can save ₹15,000–₹25,000.

Do the actual maths with a CA, yaar — it takes 10 minutes and can save real money. Don't just follow what your colleague does!

---

Q: Can I withdraw my EPF before retirement?

Bhai, yes you can withdraw EPF before retirement but there are conditions, seedha baat!

Partial withdrawal allowed for: Medical emergency (any amount), marriage (50% of your share after 7 years), home purchase/construction (up to 90% after 5 years), education (50% after 7 years).

Full withdrawal allowed when: You've been unemployed for 2+ months, or you're emigrating permanently.

Important caveat — if you withdraw before 5 years of continuous service, the amount becomes taxable and TDS is deducted. After 5 years, fully tax-free.

Also remember: withdrawing early means losing the 8.15% guaranteed return and the compounding magic. If you don't urgently need it, leaving it in EPF is often the smarter call. Consult your HR or a CA before taking the plunge, yaar!

---

Training Details

Parameter	Value
Base model	mlx-community/Mistral-7B-Instruct-v0.2-4bit
Quantisation	4-bit NF4
LoRA rank	16
LoRA alpha	32
LoRA layers	16 (top transformer blocks)
Trainable parameters	20.97M / 7,241.73M (0.29%)
SFT learning rate	5×10⁻⁵
RSFT learning rate	2×10⁻⁵
Batch size	1
SFT iterations	300 (stopped at iter-300, val loss 1.471)
RSFT iterations	300 (val loss stable 0.250–0.279)
Max sequence length	768 tokens
Peak GPU memory	7.19 GB
Training hardware	Apple M5 · 24 GB unified memory
Framework	mlx-lm 0.31.3
Data generation	Gemini 2.5 Flash Lite
Preference judge	Gemini 2.5 Flash Lite (LLM-as-judge)

---

Data

Training data was generated entirely synthetically using Gemini 2.5 Flash Lite as a teacher model — a form of knowledge distillation via LLM-generated supervision.

Dataset	Size	Description
Synthetic Q&A pairs	86	Hand-curated questions across 6 topics; Gemini-generated responses under Desi Finance Bhai persona
SFT training set	77	After train/val split (90/10)
Best-of-5 candidates	385	77 prompts × 5 SFT responses at temperature=0.9
Preference triplets	54	(prompt, chosen, rejected) — Gemini judge; 23 discarded (tied or unclear)
RSFT training set	48	Chosen responses only (90/10 split)

Topic distribution (SFT data):

Topic	Questions
SIPs & Mutual Funds	15
Home Loans & EMIs	15
Tax Planning (80C/80D/LTCG)	15
Stocks & Investing	15
PPF / NPS / EPF	14
Gold & Real Estate	12

Web scraping of Zerodha Varsity was attempted for domain grounding but blocked by rate limiting (429). All training data is therefore fully synthetic.

---

Limitations & Responsible Use

• Not regulated financial advice. This model is for educational and informational purposes only. It is not a SEBI-registered investment advisor and should not replace consultation with a qualified Chartered Accountant or financial planner.
• Knowledge cutoff. Tax slabs, interest rates, and government scheme parameters change annually. Always verify figures from official sources (Income Tax Department, EPFO, RBI).
• Small training set. The model was trained on 77–86 examples — a deliberate constraint to demonstrate persona injection and alignment on constrained data. Production deployments should use significantly larger datasets.
• Hallucination risk. As with all language models, factual errors are possible. The model is trained to include caveats, but users must verify any specific financial figures before acting on them.
• Language scope. Optimised for Indian financial context (INR, Indian tax law, SEBI regulations). Not suitable for US, UK, or other jurisdictions without adaptation.

---

Citation

If you use this model in research, please cite:

@misc{shabul2026desifinanace,
  author    = {Shabul Abdul},
  title     = {Desi Finance Advisor: SFT + RSFT Alignment of Mistral-7B for Indian Personal Finance},
  year      = {2026},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/shabul/mistral-7b-desi-finance-advisor},
  note      = {Training pipeline: github.com/shabul/model-foundry}
}

---

Related Work

Paper / Resource	Relevance
LoRA (Hu et al., 2021)	Parameter-efficient fine-tuning method used throughout
DPO (Rafailov et al., 2023)	Theoretical alignment target; RSFT used as practical alternative
AlpacaFarm (Dubois et al., 2023)	Validates RSFT best-of-N as ~60–67% of DPO alignment gain
LLM-as-Judge (Zheng et al., 2023)	Gemini used as preference judge (MT-Bench framework)
Mistral-7B (Jiang et al., 2023)	Base model architecture
MLX-LM	Apple Silicon fine-tuning framework

---

Built with ♥ on Apple Silicon · Part of Model Foundry

"Seedha baat, no bakwaas — just good financial advice, bhai."