---
license: apache-2.0
language:
- en
base_model:
- google/vit-base-patch16-224
pipeline_tag: image-classification
tags:
- OpenPlants
- plant-identification
- species-classification
- image-classification
---

<div align="center">
  <img src="openplants.png" alt="OpenPlants logo" width="220" />
  
  <h1>OpenPlants Plant Identification Model</h1>
  <p>Species-level plant classifier for <a href="https://github.com/domai-tb/OpenPlants">OpenPlants</a>.</p>
</div>

This model is used in **OpenPlants** ([`domai-tb/OpenPlants`](https://github.com/domai-tb/OpenPlants)), an open-source and privacy-friendly companion app for plant care.
Inside the app, it powers **local image classification** so users can identify plants directly on-device from a photo.

Built on `google/vit-base-patch16-224`, the model was fine-tuned for species-level plant identification on the GBIF / iNaturalist plant image dataset.

## Highlights

| Property | Value |
| --- | --- |
| Base model | `google/vit-base-patch16-224` |
| Parameters | ~97.2M |
| Training samples | 2,000,000 curated plant occurrences |
| Species coverage | ~14,000 unique species |
| Source data | GBIF / iNaturalist |
| Training method | End-to-end supervised fine-tuning |
| Primary use | Fast plant species classification from a single image |

## OpenPlants

OpenPlants is the companion app this model was built for.

- Runs plant identification locally
- Keeps inference on-device for a privacy-friendly experience
- Helps users identify plants in a fast, offline-friendly workflow

## Example Usage

```python
from transformers import AutoImageProcessor, AutoModelForImageClassification
from PIL import Image
import requests
import torch

model_id = "domai-tb/OpenPlants-Identification-ViT-Base-Patch16-224"

processor = AutoImageProcessor.from_pretrained(model_id)
model = AutoModelForImageClassification.from_pretrained(model_id)

url = "https://example.com/plant.jpg"
image = Image.open(requests.get(url, stream=True).raw)

inputs = processor(images=image, return_tensors="pt")
with torch.no_grad():
    logits = model(**inputs).logits

probs = logits.softmax(dim=-1)[0]
topk = torch.topk(probs, k=5)

for prob, idx in zip(topk.values, topk.indices):
    label = model.config.id2label[idx.item()]
    print(f"{label}: {prob.item():.4f}")
```

## Intended Applications

- Plant identification in mobile apps
- Ecological surveys
- Nursery and horticulture tools
- Restoration and revegetation workflows
- Field research and biodiversity monitoring
- Citizen science and educational platforms
- Image-based species tagging pipelines

## Data and Training Details

### Dataset Construction

- Sourced from GBIF / iNaturalist occurrences with valid `species` and image metadata
- Cleaned and deduplicated before training
- Species filtered to those with at least 20 images
- Maximum cap of 1,000 images per species to reduce class imbalance
- Final training dataset: 2,000,000 images across roughly 14,000 species

### Training

- ViT-Base fine-tuned for 1 epoch over 2M samples
- AdamW optimizer with standard ViT augmentations
- Mixed-precision training on GPU

### Limitations

- Some species are visually indistinguishable without context such as location or reproductive structures
- Performance varies for rare, morphologically similar, or poorly photographed species
- The model is purely image-based and does not use location metadata

## Labels

Species names follow canonical GBIF taxonomy (`species_name`).
Each class maps directly to one species.

You can inspect all labels with:

```python
from transformers import AutoConfig

cfg = AutoConfig.from_pretrained("domai-tb/OpenPlants-Identification-ViT-Base-Patch16-224")
labels = cfg.id2label
```

## Fine-Tuning and Adapters

You can further specialize the model with LoRA adapters for:

- Regional subsets
- Functional groups
- Threatened species
- Agricultural crops
- Disease classification

The base model is broad enough to support domain-specific adapter tuning with relatively little compute.

## License

Model weights follow the same license as the underlying ViT-Base model.
Users are responsible for complying with GBIF and iNaturalist usage terms for any downstream dataset creation.