File size: 3,687 Bytes
bad1832 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 | <h1 align="center">LingBot-Map: Geometric Context Transformer for Streaming 3D Reconstruction</h1>
<p align="center">
<a href="lingbot-map_paper.pdf"><img src="https://img.shields.io/static/v1?label=Paper&message=PDF&color=red&logo=arxiv"></a>
<a href="https://technology.robbyant.com/lingbot-map"><img src="https://img.shields.io/badge/Project-Website-blue"></a>
<a href=""><img src="https://img.shields.io/static/v1?label=%F0%9F%A4%97%20Model&message=HuggingFace&color=orange"></a>
<a href="LICENSE.txt"><img src="https://img.shields.io/badge/License-Apache--2.0-green"></a>
</p>
<p align="center">
<img src="assets/teaser.png" width="100%">
</p>
<p align="center">
<video src="https://gw.alipayobjects.com/v/huamei_vaouhm/afts/video/q0sdTr9Mm6IAAAAAmyAAAAgADglFAQJr" width="100%" autoplay loop muted playsinline></video>
</p>
---
# Quick Start
## Installation
**1. Create conda environment**
```bash
conda create -n lingbot-map python=3.10 -y
conda activate lingbot-map
```
**2. Install PyTorch (CUDA 12.8)**
```bash
pip install torch==2.9.1 torchvision==0.24.1 --index-url https://download.pytorch.org/whl/cu128
```
> For other CUDA versions, see [PyTorch Get Started](https://pytorch.org/get-started/locally/).
**3. Install lingbot-map**
```bash
pip install -e .
```
**4. Install FlashInfer (recommended)**
FlashInfer provides paged KV cache attention for efficient streaming inference:
```bash
# CUDA 12.8 + PyTorch 2.9
pip install flashinfer-python -i https://flashinfer.ai/whl/cu128/torch2.9/
```
> For other CUDA/PyTorch combinations, see [FlashInfer installation](https://docs.flashinfer.ai/installation.html).
> If FlashInfer is not installed, the model falls back to SDPA (PyTorch native attention) via `--use_sdpa`.
**5. Visualization dependencies (optional)**
```bash
pip install -e ".[vis]"
```
# Demo
## Streaming Inference from Images
```bash
python demo.py --model_path /path/to/checkpoint.pt \
--image_folder /path/to/images/
```
## Streaming Inference from Video
```bash
python demo.py --model_path /path/to/checkpoint.pt \
--video_path video.mp4 --fps 10
```
## Streaming with Keyframe Interval
Use `--keyframe_interval` to reduce KV cache memory by only keeping every N-th frame as a keyframe. Non-keyframe frames still produce predictions but are not stored in the cache. This is useful for long sequences
which excesses 320 frames.
```bash
python demo.py --model_path /path/to/checkpoint.pt \
--image_folder /path/to/images/ --keyframe_interval 6
```
## Windowed Inference (for long sequences, >3000 frames)
```bash
python demo.py --model_path /path/to/checkpoint.pt \
--video_path video.mp4 --fps 10 \
--mode windowed --window_size 64
```
## With Sky Masking
```bash
python demo.py --model_path /path/to/checkpoint.pt \
--image_folder /path/to/images/ --mask_sky
```
## Without FlashInfer (SDPA fallback)
```bash
python demo.py --model_path /path/to/checkpoint.pt \
--image_folder /path/to/images/ --use_sdpa
```
# Model Download
<!-- TODO: fill in model checkpoints -->
| Model Name | Huggingface Repository | Description |
| :--- | :--- | :--- |
| lingbot-map | | Base model checkpoint |
# License
This project is released under the Apache License 2.0. See [LICENSE](LICENSE.txt) file for details.
# Citation
```bibtex
@article{lingbot-map2026,
title={},
author={},
journal={arXiv preprint arXiv:},
year={2026}
}
```
# Acknowledgments
This work builds upon several excellent open-source projects:
- [VGGT](https://github.com/facebookresearch/vggt)
- [DINOv2](https://github.com/facebookresearch/dinov2)
- [Flashinfer](https://github.com/flashinfer-ai/flashinfer)
--- |