---
license: apache-2.0
base_model: google/siglip-base-patch16-224
tags:
- vision
- image-text-similarity
- aesthetics
- rich-captions
- synthetic-dataset
- vlm-trained
pipeline_tag: zero-shot-image-classification
---

# SigLIP: Fine-Tuned for Detailed Image Retrieval

This repository contains a fine-tuned version of Google's [SigLIP](https://huggingface.co/google/siglip-base-patch16-224), optimized for **high-precision semantic search** and **detailed image description alignment**. The model is specifically engineered to understand complex visual compositions, premium aesthetics, and intricate details that standard CLIP/SigLIP models often miss.

[github repo](https://github.com/rollenso/siglip-synthetic-hq-retrieval-v1)

## 🚀 Model Details

- **Base Model:** `google/siglip-base-patch16-224`
- **Architecture:** Contrastive Image-Text Encoder
- **Parameter Count:** ~200M (bfloat16)
- **Primary Use Case:** High-fidelity image retrieval, aesthetic-aware search engines, and vector databases (e.g., using `pgvector` or Milvus).

## 🧠 Training: The Power of Rich Captions

Unlike standard models trained on brief web alt-texts, this model was fine-tuned using a **High-Density Synthetic Dataset**. 

- **Data Source:** ~32,000 image-text pairs refined by a Vision Language Model (VLM).
- **Descriptive Richness:** Instead of simple tags, the model learned from multi-sentence descriptions generated by Qwen-VL. These captions cover lighting, texture (e.g., Glassmorphism, Liquid Metal), professional composition, and specific object relationships.
- **Optimization:** Fine-tuned via LoRA (Low-Rank Adaptation) to preserve base knowledge while injecting deep domain expertise in aesthetics.

## ⚠️ Search Strategy: Detailed Queries vs. Tags

Due to its training on VLM-generated descriptions, this model excels when provided with **descriptive, natural language queries**. It is designed to match the "richness" of the visual data.

For optimal results, use detailed prompts that describe the scene:

**❌ Basic Tag (Standard Performance):**
`texts = ["modern office"]`

**✅ Rich Description (Superior Performance):**
`texts = ["A minimalist modern office with natural sunlight, wooden furniture, and a clean glass desk against a neutral wall."]`

*Pro Tip:* For production environments, we recommend using a small LLM proxy (like Qwen 1.5B or Llama 3) to expand simple user keywords into descriptive search sentences before generating embeddings.

## 💻 Usage (Inference)

```python
import torch
import torch.nn.functional as F
from transformers import AutoProcessor, AutoModel
from PIL import Image

device = torch.device("mps" if torch.backends.mps.is_available() else ("cuda" if torch.cuda.is_available() else "cpu"))
model_path = "rollenso/siglip-synthetic-hq-retrieval-v1"

processor = AutoProcessor.from_pretrained(model_path)
model = AutoModel.from_pretrained(model_path, dtype=torch.bfloat16).to(device)
model.eval()

texts = [
    "A product photography shot of a black mirrorless camera on a blue metallic tripod.",
    "A glowing liquid glass user interface dashboard."
]
image = Image.open("path_to_your_image.jpg").convert("RGB")

inputs = processor(
    text=texts, 
    images=image, 
    padding="max_length", 
    max_length=64, 
    truncation=True,
    return_tensors="pt"
).to(device)

with torch.no_grad():
    outputs = model(**inputs)
    image_embeds = F.normalize(outputs.image_embeds, p=2, dim=-1)
    text_embeds = F.normalize(outputs.text_embeds, p=2, dim=-1)
    similarity = torch.matmul(image_embeds, text_embeds.t())

scores = similarity[0].cpu().tolist()
for text, score in zip(texts, scores):
    print(f"[{score:.4f}] {text}")
```