---
title: Multiplication LoRA Demo
emoji: 🌖
colorFrom: blue
colorTo: green
sdk: gradio
sdk_version: 6.2.0
app_file: app.py
pinned: false
license: mit
short_description: LoRa fine-tuning a tiny model on a simple arithmetic task
---

# Multiplication LoRA Demo

This demo showcases a tiny model (Qwen2.5-0.5B) fine-tuned with LoRA to multiply 6-digit numbers by 7.

## Performance

| Model | Exact Match Accuracy |
|-------|---------------------|
| Base Qwen2.5-0.5B | ~3% |
| With LoRA adapter | ~94% |

## How to Use

1. Enter a 6-digit number (100000-999999)
2. Toggle the "Use LoRA adapter" checkbox to compare base vs fine-tuned performance
3. Click "Send" to see the prediction

## Training Details

- **Base Model**: Qwen/Qwen2.5-0.5B-Instruct
- **LoRA Adapter**: [nlac/multiplication-lora-demo-adapter](https://huggingface.co/nlac/multiplication-lora-demo-adapter)
- **LoRA Rank**: 16
- **LoRA Alpha**: 32
- **Target Modules**: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- **Training Samples**: 20,000
- **Epochs**: 3

## Local Development

### Setup

```bash
# Clone the repository
git clone https://huggingface.co/spaces/nlac/multiplication-lora-demo
cd multiplication-lora-demo

# Create virtual environment (optional but recommended)
python -m venv venv
source venv/bin/activate  # Linux/Mac
# or: venv\Scripts\activate  # Windows

# Install dependencies
pip install -r requirements-local.txt
```

### Training

To train your own LoRA adapter:

```bash
python scripts/train_multiply.py
```

This will:
- Generate 20,000 training examples (configurable in `config.py`)
- Fine-tune the model for 3 epochs
- Save the adapter to `output/lora-multiplicator/final/`

Training takes about an hour on a consumer GPU.

### Testing

Interactive testing with the trained model:

```bash
# With LoRA adapter (default)
python scripts/test_multiply.py

# Without LoRA (base model only)
python scripts/test_multiply.py --no-lora
```

Commands in the interactive session:
- Enter a multiplication query like `123456 * 7`
- Type `eval` or `eval 500` to run accuracy evaluation
- Type `quit` to exit

### Running the Gradio UI locally

```bash
python app.py
```

Opens at http://localhost:7860

## Author

[nlac](https://github.com/nlac)

## License

MIT