Hugging Face's logo

robbyant
/
lingbot-vision-vit-base

Image Feature Extraction
PyTorch
English
computer-vision
self-supervised-learning
vision-transformer
dense-prediction
depth-estimation
semantic-segmentation
Model card Files
xet
 Community
lingbot-vision-vit-base / README.md
BlackLing02's picture
BlackLing02
Update README.md
f606f8c verified
preview code
|
Raw
History Blame Contribute Delete
5.1 kB
metadata 
license: apache-2.0
language:
  - en
tags:
  - computer-vision
  - self-supervised-learning
  - vision-transformer
  - image-feature-extraction
  - dense-prediction
  - depth-estimation
  - semantic-segmentation
  - pytorch
datasets:
  - custom
library_name: pytorch
pipeline_tag: image-feature-extraction

LingBot-Vision

LingBot-Vision is a family of self-supervised Vision Transformer backbones for dense spatial perception. The models are pretrained with masked boundary modeling, a boundary-centric objective that encourages spatially structured patch features while retaining strong semantic representations.

This Hugging Face repository stores a backbone-only PyTorch checkpoint as model.pt. It is intended for inference, feature extraction, PCA visualization, and downstream dense prediction research.

Model Details

Model Description

LingBot-Vision learns dense patch representations that preserve boundaries, shapes, and semantic regions. The backbone is trained from random initialization with self-supervised teacher-student pretraining. During training, teacher-discovered boundary tokens are forced into the masked set, and boundary tokens receive both semantic self-distillation and categorical boundary-field supervision.

The released model family includes:

  • LingBot-Vision-Giant: ViT-g/16 backbone for highest-quality dense features.
  • LingBot-Vision-Large: ViT-L/16 backbone for strong dense features and practical inference.
  • LingBot-Vision-Base: ViT-B/16 backbone for balanced inference cost.
  • LingBot-Vision-Small: ViT-S/16 backbone for lightweight demos and downstream use.

Each checkpoint contains backbone weights only. Training-time heads, optimizer states, and boundary-target generation components are not included.

  • Developed by: Zelin Fu, Bin Tan, Changjiang Sun, Shaohui Liu, Kecheng Zheng, Yinghao Xu, Xing Zhu, Yujun Shen, Nan Xue
  • Model type: Vision Transformer backbone for dense visual representation learning
  • License: Apache 2.0

Model Sources

Related Models

Uses

Direct Use

  • Dense Feature Visualization: Extract frozen patch tokens and visualize their PCA components.
  • Image Feature Extraction: Use normalized patch tokens as spatial visual features.
  • Backbone Initialization: Initialize downstream dense prediction models with LingBot-Vision weights.

Downstream Use

  • Depth Estimation: Frozen patch tokens expose spatial structure to lightweight dense readouts.
  • Semantic Segmentation: Boundary-faithful features help align region transitions with object contours.
  • Video Object Segmentation: Frozen features support training-free label propagation and token matching.
  • Depth Completion: LingBot-Vision can serve as the visual encoder initialization for LingBot-Depth 2.0.

How to Load

Install the LingBot-Vision inference repository and dependencies:

git clone https://github.com/robbyant/lingbot-vision.git
cd lingbot-vision

conda create -n lingbot-vision python=3.10 -y
conda activate lingbot-vision

python -m pip install -r requirements.txt
python -m pip install -e .

Load a pretrained backbone:

import torch

from lbot_vision_infer import load_pretrained_backbone

device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if device == "cuda" else torch.float32

backbone, embed_dim = load_pretrained_backbone(
    variant="large",
    device=device,
    dtype=dtype,
)

print(backbone.patch_size, embed_dim)

The variant argument can be giant, large, base, or small. You can also pass an explicit Hugging Face model repo or a local directory to load_pretrained_backbone.

Technical Specifications

Model Architecture

  • Backbone: Vision Transformer with patch size 16
  • Released variants: ViT-g/16, ViT-L/16, ViT-B/16, ViT-S/16
  • Output: Normalized patch tokens from the frozen backbone
  • Checkpoint format: Backbone-only .pt file stored as model.pt
  • Training objective: Masked boundary modeling with self-distillation

Software Requirements

  • Python >= 3.10
  • PyTorch >= 2.0.0
  • huggingface_hub
  • omegaconf

Citation 

@article{lingbot-vision2026,
  title={Vision Pretraining for Dense Spatial Perception},
  author={Fu, Zelin and Tan, Bin and Sun, Changjiang and Liu, Shaohui and Zheng, Kecheng and Xu, Yinghao and Zhu, Xing and Shen, Yujun and Xue, Nan},
  year={2026}
}

Model Card Contact