File size: 1,104 Bytes
33468f7 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 | ---
library_name: diffusers
tags:
- fp8
- safetensors
- lora
- low-rank
- diffusion
- converted-by-gradio
---
# FP8 Model with Low-Rank LoRA
- **Source**: `https://huggingface.co/LifuWang/DistillT5`
- **File**: `model.safetensors`
- **FP8 Format**: `E5M2`
- **LoRA Rank**: 64
- **LoRA File**: `model-lora-r64.safetensors`
## Usage (Inference)
```python
from safetensors.torch import load_file
import torch
# Load FP8 model
fp8_state = load_file("model-fp8-e5m2.safetensors")
lora_state = load_file("model-lora-r64.safetensors")
# Reconstruct approximate original weights
reconstructed = {}
for key in fp8_state:
if f"lora_A.{key}" in lora_state and f"lora_B.{key}" in lora_state:
A = lora_state[f"lora_A.{key}"].to(torch.float32)
B = lora_state[f"lora_B.{key}"].to(torch.float32)
lora_weight = B @ A # (rank, out) @ (in, rank) -> (out, in)
fp8_weight = fp8_state[key].to(torch.float32)
reconstructed[key] = fp8_weight + lora_weight
else:
reconstructed[key] = fp8_state[key].to(torch.float32)
```
> Requires PyTorch ≥ 2.1 for FP8 support.
|