This is the official release of the model for paper **[SYNTHIA: Novel Concept Design with Affordance Composition](https://arxiv.org/abs/2502.17793)**.
.

Hugging Face Paper: [https://huggingface.co/papers/2502.17793](https://huggingface.co/papers/2502.17793) Project page: [https://synthia-uiuc.github.io/synthia.github.io/](https://synthia-uiuc.github.io/synthia.github.io/)



# 🪄 Affordance-based Novel Concept Generator (Kandinsky-3 Fine-Tuned)

This is a fine-tuned version of the **Kandinsky-3** text-to-image pipeline, designed to generate **novel object and furniture concepts** by combining affordance-driven functionalities (e.g., "sofa + bed + cargo + bicycle").

---
## 🚀 How to Use
```python
import os
import sys
import torch
from kandinsky3 import get_T2I_pipeline, get_T2I_Flash_pipeline

# Add kandinsky3 to Python path
sys.path.append('..')

# Set device and dtype maps
device_map = torch.device('cuda:0')
dtype_map = {
    'unet': torch.float32,
    'text_encoder': torch.float32,
    'movq': torch.float32,
}

# Load the Flash text-to-image pipeline
t2i_pipe = get_T2I_Flash_pipeline(
    device_map=device_map,
    dtype_map=dtype_map,
    cache_dir="./cache/"
)

# Load fine-tuned UNet weights
t2i_pipe.unet.load_state_dict(torch.load(
    "unet_model_checkpoint.pt",
    map_location=device_map
))

# Generate image from prompt
res = t2i_pipe(
    text="a new furniture design that has functions from sofa, bed, cargo, bicycle",
    steps=50
)[0]

# Save the result
res.save("generated_image.jpg")

```