--- language: - en - es license: apache-2.0 base_model: LiquidAI/LFM2.5-VL-450M pipeline_tag: image-text-to-text library_name: transformers tags: - vision-language - lfm2-vl - flood-detection - satellite-imagery - sentinel-2 - humanitarian - colombia datasets: - jpmarindiaz/flood-detection-pair-colombia --- # lfm2-flood Fine-tune of [`LiquidAI/LFM2.5-VL-450M`](https://huggingface.co/LiquidAI/LFM2.5-VL-450M) for flood detection from Sentinel-2 satellite tile **pairs** (RGB + SWIR baseline + RGB + SWIR current). Output is a structured JSON flood-risk profile. The model is intended to run on a low-resource node (a satellite or a community ground station) and downlink only the JSON payload — not raw imagery — for offline humanitarian response in flood-affected regions. > **Status:** experimental. Trained on 88 paired samples (110 total, 22 held out for eval). The dataset is too small and noisy for an operational alert system; this is the *infrastructure*, not a deployable model. See the [Limitations](#limitations) section. ## Inputs Four images per inference, in this order: 1. **RGB-baseline** (B4-B3-B2 true color) — the location at a pre-event timestamp 2. **SWIR-baseline** (B12-B8-B4 false color) — same timestamp 3. **RGB-current** — the location at the timestamp to assess 4. **SWIR-current** — same timestamp Each is a Sentinel-2 PNG, ~5 km × 5 km, served as base64 data URL or HF Hub URL. ## Output ~~~json { "flood_present": true, "flood_severity": "moderate", "water_coverage_pct_estimate": "30-60%", "populated_area_affected": true, "infrastructure_at_risk": true, "river_overflow_visible": true, "image_quality_limited": false } ~~~ 7 fields. `flood_present` and the four booleans are change-relative — i.e. "is there flooding *vs the baseline tile*", not "is there water in this tile". `image_quality_limited` is the abstention signal — set true on cloudy or partial-coverage tiles. ## Files | file | size | purpose | |---|--:|---| | `model.safetensors` | ~860 MB | merged HF checkpoint, full fine-tune of LFM2.5-VL-450M | | `config.json`, `tokenizer*.json`, `chat_template.jinja`, etc. | small | standard HF transformers metadata | | `lfm2-flood-Q4_0.gguf` | ~245 MB | quantized backbone for llama.cpp | | `mmproj-lfm2-flood-F16.gguf` | ~189 MB | vision tower + projector for llama.cpp | ## Quick start (llama.cpp) ```bash # Download both files (or use `hf download`) hf download jpmarindiaz/lfm2-flood lfm2-flood-Q4_0.gguf --local-dir . hf download jpmarindiaz/lfm2-flood mmproj-lfm2-flood-F16.gguf --local-dir . # Serve OpenAI-compatible endpoint llama-server -m lfm2-flood-Q4_0.gguf --mmproj mmproj-lfm2-flood-F16.gguf -c 8192 --port 8765 ``` Then POST 4 images to `/v1/chat/completions` with a JSON-schema response_format. Full client code in the [humaid repo](https://github.com/jpmarindiaz/humaid) (`finetune-flood/src/evaluate.ts`, `finetune-flood/app/server.ts`). ## Training - **Base model:** `LiquidAI/LFM2.5-VL-450M` - **Backend:** [leap-finetune](https://github.com/Liquid4All/leap-finetune) on Modal H100 - **Training type:** vlm_sft, **full fine-tune** (no LoRA — the multimodal projector needs to relearn satellite multispectral imagery) - **Hyperparameters:** 3 epochs, per-device batch 2, gradient accumulation 8 (effective 16), LR 5e-5 cosine, vision-encoder LR multiplier 0.5, bf16 - **Wall time:** ~70 seconds on H100 - **Dataset:** 88 train + 22 eval pair samples from 9 documented La Mojana / Putumayo flood events. See [`jpmarindiaz/flood-detection-pair-colombia`](https://huggingface.co/datasets/jpmarindiaz/flood-detection-pair-colombia). ## Evaluation Compared against the labeler's self-consistency oracle and the un-fine-tuned base model on the same 110 pair samples: | field | opus oracle (n=30) | base LFM2.5-VL (n=110) | **this model (n=110)** | |---|--:|--:|--:| | valid_json | 1.00 | 1.00 | 1.00 | | fields_present | 1.00 | 1.00 | 1.00 | | flood_present | 0.67 | 0.66 | 0.66 | | flood_severity | 0.43 | 0.29 | 0.29 | | water_coverage_pct_estimate | 0.70 | 0.37 | 0.35 | | populated_area_affected | 0.73 | 0.51 | 0.51 | | infrastructure_at_risk | 0.73 | 0.54 | 0.54 | | river_overflow_visible | 0.67 | 0.60 | 0.60 | | image_quality_limited | 0.83 | 0.10 | **0.90** | | **overall** | **0.68** | 0.44 | **0.55** | | **avg latency (s)** | 3.87 | 0.53 | 0.53 | `overall` is the macro-average across the 7 fields. `valid_json` and `fields_present` are 1.0 because we use grammar-constrained JSON output via `response_format: {type: "json_schema"}`. ## Limitations 1. **Sentinel-2 wet-season cloud cover.** ~50% of acquisitions over La Mojana in Apr–Jun and Aug–Nov are >50% cloud. Operational pipelines (CopernicusLAC) use **Sentinel-1 SAR** (cloud-independent) for the same task. This model was trained only on Sentinel-2. 2. **Inter-labeler noise floor caps the ceiling.** Opus self-consistency on the hardest schema fields is 0.43–0.70. The student model can't exceed inter-labeler agreement. 3. **The fine-tune learned the dataset prior, not the task.** The +11-point overall gain (0.44 → 0.55) is concentrated almost entirely in `image_quality_limited` (0.10 → 0.90). Other fields essentially didn't move with 88 train samples × 3 epochs. 4. **Geographic scope: Colombia only.** Locations are 8 La Mojana municipalities + 6 Putumayo. May not generalize to other flood regimes (e.g. flash floods, glacial-melt rivers, hurricane storm surge). ## How it was built The full pipeline + scripts + playbook are in the [humaid repo on GitHub](https://github.com/jpmarindiaz/humaid). Key files: - `finetune-flood/PLAYBOOK.md` — end-to-end command sequence (fetch → label → build → upload → train → pull → package → serve → eval) - `finetune-flood/REPORT.md` — wrap-up findings (what worked, what didn't, the case for switching to Sentinel-1 SAR before resuming) - `finetune-flood/docs/` — overview, pipeline, data collection, labeling, evaluation, findings - `finetune-flood/scripts/convert_mmproj_lfm2vl.py` — patch around the upstream `convert_hf_to_gguf.py` issue with `lm_head.weight` in full-FT merged checkpoints - `finetune-flood/app/` — small Hono app for testing the model interactively ## Citation Trained 2026-05-08 as part of the [humaid](https://github.com/jpmarindiaz/humaid) project — offline-first humanitarian response toolkit for flood crises in Colombia. Built on: - [LFM2.5-VL-450M](https://huggingface.co/LiquidAI/LFM2.5-VL-450M) by Liquid AI - [leap-finetune](https://github.com/Liquid4All/leap-finetune) by Liquid AI - [SimSat](https://github.com/DPhi-Space/SimSat) by DPhi Space - The Liquid AI × DPhi Space [AI in Space hackathon](https://luma.com/n9cw58h0) wildfire-prevention example by [Pau Labarta Bajo](https://github.com/Paulescu)