---
library_name: peft
tags:
- summarization
- qlora
- bitsandbytes
base_model: google/flan-t5-large
datasets:
- billsum
---

# Flan-T5 Large — BillSum QLoRA

A [Flan-T5 Large](https://huggingface.co/google/flan-t5-large) model fine-tuned with QLoRA (8-bit quantization) for summarizing US Congressional bills using the [BillSum](https://huggingface.co/datasets/billsum) dataset.

> **Video walkthrough**: [Parameter-efficient fine-tuning with QLoRA and Hugging Face](https://youtube.com/watch?v=ebQ2wyn8RGM)

## Model Details

| Detail | Value |
|--------|-------|
| **Base model** | [google/flan-t5-large](https://huggingface.co/google/flan-t5-large) |
| **Task** | Text summarization |
| **Dataset** | [BillSum](https://huggingface.co/datasets/billsum) (US Congressional bills) |
| **Method** | QLoRA with 8-bit quantization (bitsandbytes) |
| **Framework** | PEFT 0.5.0 |

## Quantization Config

| Parameter | Value |
|-----------|-------|
| `quant_method` | bitsandbytes |
| `load_in_8bit` | True |
| `llm_int8_threshold` | 6.0 |

## Usage

```python
from peft import PeftModel
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

base_model = AutoModelForSeq2SeqLM.from_pretrained("google/flan-t5-large", load_in_8bit=True)
model = PeftModel.from_pretrained(base_model, "juliensimon/flan-t5-large-billsum-qlora")
tokenizer = AutoTokenizer.from_pretrained("google/flan-t5-large")

text = "Summarize: " + bill_text
inputs = tokenizer(text, return_tensors="pt", max_length=512, truncation=True)
outputs = model.generate(**inputs, max_new_tokens=150)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```