---
title: NASA Space Image Recommender
emoji: ๐ญ
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 4.44.1
app_file: app.py
python_version: "3.10"
pinned: false
---
๐ฅ Project Presentation Video
Click the image above to watch the full presentation on YouTube.
---
# ๐ญ NASA Space Image Recommender
A semantic image recommendation system that lets you search **5,000 curated NASA, ESO & Hubble space images** using natural language *or* by uploading your own image. Built with **CLIP embeddings**, **Sentence Transformers**, and **K-Means clustering**, deployed on **HuggingFace Spaces** with a **Gradio** interface.
> **Try it:** type a description like *"spiral galaxy"* or *"rings of saturn"*, or upload any space photo and let CLIP find visually similar NASA images.
---
## ๐ Table of Contents
1. [Project Overview](#-project-overview)
2. [The Dataset](#-the-dataset)
3. [Part 2 โ Exploratory Data Analysis](#-part-2--exploratory-data-analysis-eda)
4. [Part 3 โ Embeddings & Clustering](#-part-3--embeddings--clustering)
5. [Part 4 โ Recommendation Pipeline](#-part-4--recommendation-pipeline)
6. [Part 5 โ The HuggingFace App](#-part-5--the-huggingface-app)
7. [Challenges & Engineering Decisions](#-challenges--engineering-decisions)
8. [Tech Stack](#-tech-stack)
9. [How to Run Locally](#-how-to-run-locally)
---
## ๐ฏ Project Overview
This project builds a complete, end-to-end **data science pipeline** for a visual recommendation system:
**Problem โ Data โ EDA โ Embeddings โ Clustering โ Recommendation โ Deployment**
The goal is to let a user find relevant space imagery through two complementary search modes:
| Mode | How it works | Powered by |
|------|-------------|-----------|
| ๐ **Search by Text** | Type a description; the system matches it against image captions | Sentence Transformers + cosine similarity |
| ๐ผ๏ธ **Search by Image** | Upload a photo; the system finds visually similar NASA images | CLIP visual embeddings + cosine similarity |
Both modes return the **Top 3** matches, each tagged with a **similarity score** and an automatically-assigned **cluster category** (e.g. *Galaxies & Deep Space*, *Nebulae & Gas Clouds*).
---
## ๐ The Dataset
**Name:** `AstroLLaVA_convos` โ [UniverseTBD on HuggingFace](https://huggingface.co/datasets/UniverseTBD/AstroLLaVA_convos)
A large collection of NASA, ESA & Hubble space images, each paired with a detailed caption written by professional astronomers. The dataset starts at **29,783 images** and, after filtering out non-astronomical content, retains **15,892 pure space images**.
**Source breakdown:**
| Source | Images | Share |
|--------|--------|-------|
| ๐ญ ESO (European Southern Observatory) | 14,612 | 49% |
| ๐ฐ๏ธ APOD (Astronomy Picture of the Day) | 9,967 | 33% |
| ๐ Hubble Space Telescope | 5,204 | 17% |
**Features (7 columns):** `image` ยท `id` ยท `caption` ยท `conversation` ยท `url` ยท `corpus` ยท `credit`
> **Why this dataset?** Space imagery is visually stunning and scientifically rich, and the professional captions make it ideal for **semantic search** โ exactly the natural-language โ image matching that embeddings excel at.
---
## ๐ฌ Part 2 โ Exploratory Data Analysis (EDA)
Before modeling, I performed a thorough EDA to understand the data's structure, quality, and visual characteristics. A key step was **filtering**: starting from 29,783 images, I applied an astronomical keyword filter plus a strict blocklist (removing portraits, logos, conference photos, diagrams, etc.) to retain **15,892 pure space images**.
### Distribution by Source
The dataset is dominated by **ESO (49%, 14,612 images)**, followed by **APOD/NASA (33%, 9,967 images)** and **Hubble (17%, 5,204 images)**.
### Most Common Words in Captions
A word cloud of the caption corpus reveals the vocabulary of astronomy โ *star*, *galaxy*, *nebula*, *image*, *light* dominate.
### Most Mentioned Astronomical Objects
**Stars** lead with over 11,369 mentions โ nearly double the second-place *Planet* (6,065).
### Color Distribution
The **Red channel** slightly dominates (mean โ 66), reflecting the warm tones of nebulae and stellar explosions.
### Brightness Analysis
Space images are predominantly **dark** (average pixel brightness โ 60/255), with APOD images slightly brighter than ESO and Hubble.
### Resolution Distribution
High-resolution imagery throughout โ average **1,327 ร 1,028 px**, with some professional observatory images exceeding 22,000 px wide.
### Sample Images
A sample grid of the filtered dataset confirms the blocklist worked โ only genuine space imagery remains.
> **EDA Conclusion:** The dataset is clean (no missing values), visually diverse, and scientifically rich โ perfectly suited for a CLIP-based semantic recommender.
---
## ๐ง Part 3 โ Embeddings & Clustering
### Model: CLIP ViT-B/32
I used **`openai/clip-vit-base-patch32`** โ a vision-language model that maps both images and text into a shared **512-dimensional** embedding space. This shared space is what makes both text-search and image-search possible from a single model.
**Embedding generation:**
- Sampled **5,000 images** with a fixed seed (`SEED = 42`) for reproducibility
- Generated image embeddings in batches โ final shape **(5,000 ร 512)**
### Dimensionality Reduction & Clustering
- **PCA** (50 components) to speed up t-SNE
- **K-Means** (k=6) to group the embeddings
- **t-SNE** to project the 512-D embeddings into 2D for visualization
### t-SNE Cluster Map
### Cluster Sizes
The six discovered clusters:
| Cluster | Description | Images |
|---------|-------------|--------|
| ๐ | Galaxies & Deep Space | 992 |
| ๐ช | Solar System & Planets | 969 |
| โญ | Stars & Star Clusters | 873 |
| ๐ญ | Earth & Observatories | 786 |
| โ๏ธ | Nebulae & Gas Clouds | 712 |
| ๐ | Mixed Astronomical | 667 |
### Sample Images per Cluster
> **Clustering Conclusion:** CLIP successfully grouped similar astronomical images **without any labels** โ the clusters are well-balanced and visually coherent, proving the embeddings capture meaningful structure.
### ๐ Bonus โ Validating k=6
To choose the number of clusters rigorously (rather than guessing), I ran two standard validation techniques.
**Elbow Method**
**Silhouette Score**
Both methods confirmed **k=6** as the optimal choice.
**Interactive t-SNE (Plotly)**
An interactive Plotly version lets you hover over any point to read its caption and cluster.
---
## โ๏ธ Part 4 โ Recommendation Pipeline
The text-search recommender works through a four-step semantic-matching pipeline:
1. **Caption embeddings** โ all 5,000 captions encoded with `sentence-transformers/all-MiniLM-L6-v2` โ shape **(5,000 ร 384)**
2. **User input** โ the query text is converted into a 384-D embedding
3. **Similarity scores** โ cosine similarity between the query and every caption embedding
4. **Top 3** โ the three highest-scoring images are returned with their similarity scores and cluster labels
**Sample test results (real output from the notebook):**
| Query | Top Match | Similarity |
|-------|-----------|-----------|
| ๐ "spiral galaxy" | The Whirlpool Galaxy (M51a) | **0.757** |
| โ๏ธ "colorful nebula" | The Flaming Star Nebula | **0.724** |
| ๐ช "rings of saturn" | Saturn with moons Tethys & Dione | **0.760** |
> **Why two embedding models?** Caption-based matching (Sentence Transformers) works best for **text** queries because it compares meaning-to-meaning. CLIP image embeddings power the **image-to-image** search, where visual similarity is what matters.
---
## ๐ Part 5 โ The HuggingFace App
The app is built with **Gradio Blocks** and features a custom dark "space" theme.
**Features:**
- ๐ **Search by Text** โ natural-language query box with 15 clickable examples
- ๐ผ๏ธ **Search by Image** โ upload any space photo to find visually similar NASA images
- ๐ฒ **Surprise Me** โ random query button for exploration
- ๐ข๐ก๐ด **Color-coded similarity bars** โ instant visual feedback on match quality
- ๐ท๏ธ **Cluster labels** โ every result is tagged with its astronomical category
- โน๏ธ **"How it works"** โ an expandable explainer panel
---
## ๐ ๏ธ Challenges & Engineering Decisions
Building a production app on the free CPU tier surfaced several real engineering trade-offs worth documenting:
**1. Date-search feature (Colab only).**
The Colab notebook includes a third search mode โ **Search by Date** โ that retrieves the NASA Astronomy Picture of the Day (APOD) for any date between 1995โ2023. This feature works in Colab, where the full dataset is held in memory and images load instantly. On the HuggingFace **free CPU tier**, however, it caused instability: the APOD lookup loads images on demand from external URLs, and combined with the limited RAM this led to freezes and timeouts. **Decision:** keep the date feature documented in the notebook (where it runs reliably) but ship only the two robust tabs โ text and image search โ in the deployed app. This is a deliberate scope decision to guarantee a stable demo, not a missing feature.
**2. Free-tier RAM limits.**
Running CLIP inference on a CPU-only Space is memory-constrained. I added explicit garbage collection (`gc.collect()`) after each inference call and resized uploaded images before encoding, keeping the app within the free-tier memory budget.
**3. Reproducibility across restarts.**
A subtle issue with HuggingFace Spaces: each restart re-runs the whole pipeline from scratch, and the random sample of 3,000 images can differ from the Colab run (different data-loading order). I pinned `SEED = 42` everywhere to keep results as reproducible as possible.
**4. Dead image URLs.**
Some images in the dataset point to URLs that no longer resolve. I returned the in-memory dataset images directly to the Gradio components rather than relying on external URLs, eliminating broken-image results.
**5. Emoji-aware query handling.**
The example buttons include emojis (๐ spiral galaxy) for a friendly UI, but emojis pollute the embedding. I strip non-ASCII characters from queries before encoding, so the emoji is shown to the user but ignored by the model.
---
> *"To view the complete data analysis and modeling architecture in the official IPYNB file, click the button below:"*
[](https://colab.research.google.com/drive/1Ylqnk9yoOGEZ5Vg6--5fY07pBgFq8Nfv?usp=sharing)
## โถ๏ธ How to Run Locally
```bash
pip install -r requirements.txt
python app.py
```
The app loads the dataset, generates embeddings, runs clustering, and launches a local Gradio server. First startup takes a few minutes while embeddings are computed.
---
## ๐ค Author
**Omer Inbar** โ Adelson School of Entrepreneurship, Reichman University
*Introduction to Data Science โ Assignment 3*
*Built with CLIP, Sentence Transformers & Gradio ยท Dataset: AstroLLaVA (UniverseTBD)*