README.md

metadata

title: Multimodal RAG with Nemotron Embed and Rerank VL
emoji: 👁️📑
colorFrom: green
colorTo: green
sdk: gradio
python_version: '3.12'
app_file: app.py
pinned: false
license: apache-2.0
sdk_version: 6.8.0

Multimodal RAG with Nemotron Embed and Rerank VL Models

Search for recipes using input queries of text or images.

Dataset is 10,000 recipes from Food.com turned into images and markdown and embedded into the same feature space.

Options:

• Default is to retrieve most similar recipes via visual or text search
• Optional: Rerank initial results with reranker model (note: this increases latency)
• Optional: Generate a summary of the returned recipes (note: this increases latency)

Details:

• Dataset used: https://huggingface.co/datasets/mrdbourke/recipe-synthetic-images-10k
• Embedding model used: https://huggingface.co/nvidia/llama-nemotron-embed-vl-1b-v2
  • Note: By default we use the image + text embeddings as we have access to image and text pairs in our dataset, and according to the launch blog post, these work the best.
• Rerank model used: https://huggingface.co/nvidia/llama-nemotron-rerank-vl-1b-v2
• Generation model used: https://huggingface.co/Qwen/Qwen3-VL-2B-Instruct (note: you could use a larger model such as Nemotron v3, however, this will require more compute resources)

Note: This README.md was authored in a live tutorial recorded for YouTube.