Instructions to use Marqo/marqo-fashionSigLIP with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- OpenCLIP
How to use Marqo/marqo-fashionSigLIP with OpenCLIP:
import open_clip model, preprocess_train, preprocess_val = open_clip.create_model_and_transforms('hf-hub:Marqo/marqo-fashionSigLIP') tokenizer = open_clip.get_tokenizer('hf-hub:Marqo/marqo-fashionSigLIP') - Transformers
How to use Marqo/marqo-fashionSigLIP with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("zero-shot-image-classification", model="Marqo/marqo-fashionSigLIP", trust_remote_code=True) pipe( "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/parrots.png", candidate_labels=["animals", "humans", "landscape"], )# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Marqo/marqo-fashionSigLIP", trust_remote_code=True, device_map="auto") - Transformers.js
How to use Marqo/marqo-fashionSigLIP with Transformers.js:
// npm i @huggingface/transformers import { pipeline } from '@huggingface/transformers'; // Allocate pipeline const pipe = await pipeline('zero-shot-image-classification', 'Marqo/marqo-fashionSigLIP'); - Notebooks
- Google Colab
- Kaggle
load model from local dir
Ist there a way to load the model from a local directory rather than the hub? I have been struggling to achieve this as open_clip always wants to load from the hub due to the hf-hub: prefix.
struggle for this too, is there any solution?
i think you can do it three ways
1 - you can use a local version specified in a similar way to the hub https://github.com/mlfoundations/open_clip/pull/1069
2 - you can use pretrained and pass the local weights, note this will not load the correct image preprocessor though and will need to be loaded explicitly, see here https://github.com/mlfoundations/open_clip/issues/920
3 - you can load the pretrained base model (pretrained=webli) and then load the state dict of fashion siglip into that model (model.load_state_dict)