| --- |
| thumbnail: >- |
| https://raw.githubusercontent.com/wiki/lhackel-tub/ConfigILM/static/imgs/RSiM_Logo_1.png |
| tags: |
| - convmixer_768_32 |
| - BigEarthNet v2.0 |
| - Remote Sensing |
| - Classification |
| - image-classification |
| - Multispectral |
| library_name: configilm |
| license: mit |
| widget: |
| - src: example.png |
| example_title: Example |
| output: |
| - label: Agro-forestry areas |
| score: 0 |
| - label: Arable land |
| score: 0 |
| - label: Beaches, dunes, sands |
| score: 0 |
| - label: Broad-leaved forest |
| score: 0 |
| - label: Coastal wetlands |
| score: 0 |
| new_version: BIFOLD-BigEarthNetv2-0/convmixer_768_32-s2-v0.2.0 |
| --- |
| |
| [TU Berlin](https://www.tu.berlin/) | [RSiM](https://rsim.berlin/) | [DIMA](https://www.dima.tu-berlin.de/menue/database_systems_and_information_management_group/) | [BigEarth](http://www.bigearth.eu/) | [BIFOLD](https://bifold.berlin/) |
| :---:|:---:|:---:|:---:|:---: |
| <a href="https://www.tu.berlin/"><img src="https://raw.githubusercontent.com/wiki/lhackel-tub/ConfigILM/static/imgs/tu-berlin-logo-long-red.svg" style="font-size: 1rem; height: 2em; width: auto" alt="TU Berlin Logo"/> | <a href="https://rsim.berlin/"><img src="https://raw.githubusercontent.com/wiki/lhackel-tub/ConfigILM/static/imgs/RSiM_Logo_1.png" style="font-size: 1rem; height: 2em; width: auto" alt="RSiM Logo"> | <a href="https://www.dima.tu-berlin.de/menue/database_systems_and_information_management_group/"><img src="https://raw.githubusercontent.com/wiki/lhackel-tub/ConfigILM/static/imgs/DIMA.png" style="font-size: 1rem; height: 2em; width: auto" alt="DIMA Logo"> | <a href="http://www.bigearth.eu/"><img src="https://raw.githubusercontent.com/wiki/lhackel-tub/ConfigILM/static/imgs/BigEarth.png" style="font-size: 1rem; height: 2em; width: auto" alt="BigEarth Logo"> | <a href="https://bifold.berlin/"><img src="https://raw.githubusercontent.com/wiki/lhackel-tub/ConfigILM/static/imgs/BIFOLD_Logo_farbig.png" style="font-size: 1rem; height: 2em; width: auto; margin-right: 1em" alt="BIFOLD Logo"> |
|
|
| # Convmixer_768_32 pretrained on BigEarthNet v2.0 using Sentinel-2 bands |
|
|
| <!-- Optional images --> |
| <!-- |
| [Sentinel-1](https://sentinel.esa.int/web/sentinel/missions/sentinel-1) | [Sentinel-2](https://sentinel.esa.int/web/sentinel/missions/sentinel-2) |
| :---:|:---: |
| <a href="https://sentinel.esa.int/web/sentinel/missions/sentinel-1"><img src="https://raw.githubusercontent.com/wiki/lhackel-tub/ConfigILM/static/imgs/sentinel_2.jpg" style="font-size: 1rem; height: 10em; width: auto; margin-right: 1em" alt="Sentinel-2 Satellite"/> | <a href="https://sentinel.esa.int/web/sentinel/missions/sentinel-2"><img src="https://raw.githubusercontent.com/wiki/lhackel-tub/ConfigILM/static/imgs/sentinel_1.jpg" style="font-size: 1rem; height: 10em; width: auto; margin-right: 1em" alt="Sentinel-1 Satellite"/> |
| --> |
|
|
|
|
| > **_NOTE:_** This version of the model has been trained with a different band order that is not compatible with the newer versions and does not match the order proposed in the technical documentation of Sentinel-2. |
| > |
| > The following bands (in the specified order) were used to train the models with version 0.1.1: |
| > - For models using Sentinel-1 only: Sentinel-1 bands `["VH", "VV"]` |
| > - For models using Sentinel-2 only: Sentinel-2 10m bands and 20m bands `["B02", "B03", "B04", "B08", "B05", "B06", "B07", "B11", "B12", "B8A"]` |
| > - For models using Sentinel-1 and Sentinel-2: Sentinel-2 10m bands and 20m bands and Sentinel-1 bands = `["B02", "B03", "B04", "B08", "B05", "B06", "B07", "B11", "B12", "B8A", "VH", "VV"]` |
| > |
| > Newer models are compatible with the order in the technical documentation of Sentinel-2 and were trained with the following band order: |
| > - For models using Sentinel-1 only: Sentinel-1 bands `["VV", "VH"]` |
| > - For models using Sentinel-2 only: Sentinel-2 10m bands and 20m bands `["B02", "B03", "B04", "B05", "B06", "B07", "B08", "B8A", "B11", "B12"]` |
| > - For models using Sentinel-1 and Sentinel-2: Sentinel-1 bands and Sentinel-2 10m bands and 20m bands `["VV", "VH", "B02", "B03", "B04", "B05", "B06", "B07", "B08", "B8A", "B11", "B12"]` |
|
|
| This model was trained on the BigEarthNet v2.0 (also known as reBEN) dataset using |
| the Sentinel-2 bands. |
| It was trained using the following parameters: |
| - Number of epochs: up to 100 (with early stopping after 5 epochs of no improvement based on validation average |
| precision macro) |
| - Batch size: 512 |
| - Learning rate: 0.001 |
| - Dropout rate: 0.15 |
| - Drop Path rate: 0.15 |
| - Learning rate scheduler: LinearWarmupCosineAnnealing for 1000 warmup steps |
| - Optimizer: AdamW |
| - Seed: 42 |
|
|
| The weights published in this model card were obtained after 28 training epochs. |
| For more information, please visit the [official BigEarthNet v2.0 (reBEN) repository](https://git.tu-berlin.de/rsim/reben-training-scripts), where you can find the training scripts. |
|
|
| ](https://raw.githubusercontent.com/wiki/lhackel-tub/ConfigILM/static/imgs/combined_2000_600_2020_0_wide.jpg) |
|
|
| The model was evaluated on the test set of the BigEarthNet v2.0 dataset with the following results: |
|
|
| | Metric | Macro | Micro | |
| |:------------------|------------------:|------------------:| |
| | Average Precision | 0.696914 | 0.857402 | |
| | F1 Score | 0.638002 | 0.761015 | |
| | Precision | 0.696914 | 0.857402 | |
|
|
| # Example |
| | A Sentinel-2 image (true color representation) | |
| |:---------------------------------------------------:| |
| | ](example.png) | |
|
|
| | Class labels | Predicted scores | |
| |:--------------------------------------------------------------------------|--------------------------------------------------------------------------:| |
| | <p> Agro-forestry areas <br> Arable land <br> Beaches, dunes, sands <br> ... <br> Urban fabric </p> | <p> 0.000000 <br> 0.000000 <br> 0.000000 <br> ... <br> 0.000000 </p> | |
|
|
|
|
| To use the model, download the codes that define the model architecture from the |
| [official BigEarthNet v2.0 (reBEN) repository](https://git.tu-berlin.de/rsim/reben-training-scripts) and load the model using the |
| code below. Note that you have to install [`configilm`](https://pypi.org/project/configilm/) to use the provided code. |
|
|
| ```python |
| from reben_publication.BigEarthNetv2_0_ImageClassifier import BigEarthNetv2_0_ImageClassifier |
| |
| model = BigEarthNetv2_0_ImageClassifier.from_pretrained("path_to/huggingface_model_folder") |
| ``` |
|
|
| e.g. |
|
|
| ```python |
| from reben_publication.BigEarthNetv2_0_ImageClassifier import BigEarthNetv2_0_ImageClassifier |
| |
| model = BigEarthNetv2_0_ImageClassifier.from_pretrained( |
| "BIFOLD-BigEarthNetv2-0/convmixer_768_32-s2-v0.1.1") |
| ``` |
|
|
| If you use this model in your research or the provided code, please cite the following papers: |
| ```bibtex |
| @article{clasen2024refinedbigearthnet, |
| title={reBEN: Refined BigEarthNet Dataset for Remote Sensing Image Analysis}, |
| author={Clasen, Kai Norman and Hackel, Leonard and Burgert, Tom and Sumbul, Gencer and Demir, Beg{\"u}m and Markl, Volker}, |
| year={2024}, |
| eprint={2407.03653}, |
| archivePrefix={arXiv}, |
| primaryClass={cs.CV}, |
| url={https://arxiv.org/abs/2407.03653}, |
| } |
| ``` |
| ```bibtex |
| @article{hackel2024configilm, |
| title={ConfigILM: A general purpose configurable library for combining image and language models for visual question answering}, |
| author={Hackel, Leonard and Clasen, Kai Norman and Demir, Beg{\"u}m}, |
| journal={SoftwareX}, |
| volume={26}, |
| pages={101731}, |
| year={2024}, |
| publisher={Elsevier} |
| } |
| ``` |
