---
license: mit
base_model: microsoft/Phi-3-mini-4k-instruct
tags:
- phi3
- finance
- financial-qa
- fintech
- fine-tuned
- lora
- 4bit
language:
- en
library_name: transformers
pipeline_tag: text-generation
datasets:
- FinGPT/fingpt-fiqa_qa
---

# Phi-3-Mini FinSight Financial Q&A Assistant

This model is a fine-tuned version of [microsoft/Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct) specialized for financial question answering. It serves as the core reasoning engine for the FinSight 360 financial intelligence system.

## Model Details

### Model Description

A 3.8B parameter language model fine-tuned using LoRA (Low-Rank Adaptation) on financial Q&A data. The model is optimized to answer questions about investments, banking, personal finance, and corporate finance topics.

- **Developed by:** sweatSmile
- **Model type:** Causal Language Model (Decoder-only Transformer)
- **Language(s):** English
- **License:** MIT
- **Finetuned from model:** microsoft/Phi-3-mini-4k-instruct

### Model Sources

- **Repository:** [More Information Needed]
- **Base Model:** [microsoft/Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct)

## Uses

### Direct Use

This model can be used directly for:
- Answering financial and investment questions
- Explaining financial concepts and terminology
- Providing guidance on personal finance topics
- Educational purposes for financial literacy

### Downstream Use

The model is designed as a component of the FinSight 360 system, which includes:
- Real-time financial data retrieval (RAG architecture)
- Risk assessment and sentiment analysis
- Entity extraction from financial documents
- Interactive financial dashboard

### Out-of-Scope Use

- **Not financial advice:** This model is for educational and informational purposes only
- **Not for trading decisions:** Should not be used as sole basis for investment decisions
- **Not licensed advice:** Does not replace consultation with qualified financial advisors
- **Not for emergency financial situations:** Cannot provide real-time crisis management

## Bias, Risks, and Limitations

- Trained on only 500 samples - limited coverage of specialized financial topics
- May not reflect the most current market conditions or regulations
- Potential bias toward certain financial instruments or strategies present in training data
- Cannot access real-time market data or perform live calculations
- May generate plausible-sounding but incorrect information (hallucinations)

### Recommendations

Users should:
- Verify all financial information with qualified professionals
- Not use for actual investment or financial decisions without expert consultation
- Be aware of the model's training data limitations
- Cross-reference answers with authoritative financial sources

## How to Get Started with the Model

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "sweatSmile/Phi3-Mini-FinSight-FinancialQA"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True)

prompt = """<|system|>
You are FinSight, an expert financial advisor.<|end|>
<|user|>
What's the difference between a Roth IRA and a Traditional IRA?<|end|>
<|assistant|>
"""

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7, do_sample=True)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
```

## Training Details

### Training Data

Fine-tuned on 500 samples from [FinGPT/fingpt-fiqa_qa](https://huggingface.co/datasets/FinGPT/fingpt-fiqa_qa), which contains financial questions and expert answers covering:
- Investment strategies
- Banking and credit
- Personal finance management
- Corporate finance concepts
- Market analysis

### Training Procedure

#### Training Hyperparameters

- **Training regime:** bf16 mixed precision
- **Epochs:** 3
- **Learning rate:** 2e-5
- **Batch size:** 4 per device
- **Max sequence length:** 1024 tokens
- **Optimizer:** AdamW with cosine learning rate schedule
- **Warmup ratio:** 0.1
- **Weight decay:** 0.01
- **Gradient clipping:** 1.0

#### LoRA Configuration

- **Rank (r):** 8
- **Alpha:** 16
- **Dropout:** 0.1
- **Target modules:** All linear layers
- **Quantization:** 4-bit (nf4)

#### Speeds, Sizes, Times

- **Training time:** ~30 minutes on single T4 GPU
- **Model size (merged):** ~7GB
- **Hardware:** Google Colab T4 GPU (16GB VRAM)

## Evaluation

Due to the small training set (500 samples), formal evaluation metrics were not computed. The model is intended as a proof-of-concept and component for the larger FinSight 360 system.

### Example Outputs

**Question:** "How do interest rates affect stock prices?"  
**Response:** [Model provides explanation of inverse relationship between rates and equity valuations]

**Question:** "What is diversification in investing?"  
**Response:** [Model explains portfolio risk management through asset allocation]

## Technical Specifications

### Model Architecture and Objective

- **Architecture:** Phi-3 (Dense transformer decoder)
- **Parameters:** 3.8 billion (base model)
- **Trainable parameters:** ~4.2 million via LoRA (0.11% of base)
- **Context window:** 4,096 tokens
- **Objective:** Causal language modeling with cross-entropy loss

### Compute Infrastructure

#### Hardware

- GPU: NVIDIA T4 (16GB VRAM)
- Platform: Google Colab

#### Software

- Transformers: 4.x
- TRL (Transformer Reinforcement Learning)
- PEFT (Parameter-Efficient Fine-Tuning)
- PyTorch: 2.x
- bitsandbytes (4-bit quantization)

## Citation

**BibTeX:**

```bibtex
@misc{phi3-finsight-2025,
  author = {sweatSmile},
  title = {Phi-3-Mini FinSight Financial Q&A Assistant},
  year = {2025},
  publisher = {HuggingFace},
  journal = {HuggingFace Model Hub},
  howpublished = {\url{https://huggingface.co/sweatSmile/Phi3-Mini-FinSight-FinancialQA}}
}
```

## Model Card Authors

sweatSmile

## Model Card Contact

Available via HuggingFace profile