Image Feature Extraction
Transformers
Safetensors
page
feature-extraction
gaze-estimation
gaze-target-estimation
dinov3
custom_code
Instructions to use Octopus1/page-vitb with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Octopus1/page-vitb with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-feature-extraction", model="Octopus1/page-vitb", trust_remote_code=True)# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Octopus1/page-vitb", trust_remote_code=True, dtype="auto") - Notebooks
- Google Colab
- Kaggle
Upload folder using huggingface_hub
Browse files- LICENSE +15 -0
- README.md +66 -0
- config.json +57 -0
- model.safetensors +3 -0
- preprocessor_config.json +25 -0
LICENSE
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Apache License
|
| 2 |
+
Version 2.0, January 2004
|
| 3 |
+
http://www.apache.org/licenses/
|
| 4 |
+
|
| 5 |
+
Licensed under the Apache License, Version 2.0 (the "License");
|
| 6 |
+
you may not use this file except in compliance with the License.
|
| 7 |
+
You may obtain a copy of the License at
|
| 8 |
+
|
| 9 |
+
http://www.apache.org/licenses/LICENSE-2.0
|
| 10 |
+
|
| 11 |
+
Unless required by applicable law or agreed to in writing, software
|
| 12 |
+
distributed under the License is distributed on an "AS IS" BASIS,
|
| 13 |
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
| 14 |
+
See the License for the specific language governing permissions and
|
| 15 |
+
limitations under the License.
|
README.md
ADDED
|
@@ -0,0 +1,66 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
library_name: transformers
|
| 4 |
+
base_model: []
|
| 5 |
+
tags:
|
| 6 |
+
- gaze-estimation
|
| 7 |
+
- gaze-target-estimation
|
| 8 |
+
- dinov3
|
| 9 |
+
pipeline_tag: image-feature-extraction
|
| 10 |
+
---
|
| 11 |
+
|
| 12 |
+
# PaGE-ViT-B
|
| 13 |
+
|
| 14 |
+
Base variant, general scenes. Distilled. Part of the [PaGE](https://huggingface.co/Octopus1/PaGE) gaze target estimation family.
|
| 15 |
+
|
| 16 |
+
- **Backbone:** DINOv3 ViT-B/16
|
| 17 |
+
- **Params:** ~90M
|
| 18 |
+
- **Scene input:** 512×512, **Head input:** 256×256, **Heatmap output:** 64×64
|
| 19 |
+
- **Source checkpoint:** `vitb_distill.pt`
|
| 20 |
+
|
| 21 |
+
## Self-contained weights
|
| 22 |
+
|
| 23 |
+
This checkpoint includes the full DINOv3 backbone weights in its `safetensors` files. **No external
|
| 24 |
+
DINOv3 weights are downloaded.** The model code (`modeling_page.py`) is loaded automatically from
|
| 25 |
+
[`Octopus1/PaGE`](https://huggingface.co/Octopus1/PaGE) via `auto_map` when you pass `trust_remote_code=True`.
|
| 26 |
+
|
| 27 |
+
## Installation
|
| 28 |
+
|
| 29 |
+
```bash
|
| 30 |
+
pip install torch torchvision timm "transformers>=4.56" safetensors pillow
|
| 31 |
+
```
|
| 32 |
+
|
| 33 |
+
## Usage
|
| 34 |
+
|
| 35 |
+
```python
|
| 36 |
+
from transformers import AutoModel, AutoImageProcessor
|
| 37 |
+
from PIL import Image
|
| 38 |
+
import torch
|
| 39 |
+
|
| 40 |
+
repo = "Octopus1/page-vitb"
|
| 41 |
+
model = AutoModel.from_pretrained(repo, trust_remote_code=True).eval()
|
| 42 |
+
processor = AutoImageProcessor.from_pretrained(repo, trust_remote_code=True)
|
| 43 |
+
|
| 44 |
+
scene = Image.open("scene.jpg").convert("RGB")
|
| 45 |
+
head = Image.open("head.jpg").convert("RGB")
|
| 46 |
+
|
| 47 |
+
inputs = processor(scene, head_crops=[head], bboxes=[[(0.10, 0.10, 0.30, 0.40)]])
|
| 48 |
+
|
| 49 |
+
with torch.no_grad():
|
| 50 |
+
out = model(inputs)
|
| 51 |
+
|
| 52 |
+
heatmap = out["heatmap"][0] # [Np, 64, 64]
|
| 53 |
+
inout = out["inout"][0] # [Np]
|
| 54 |
+
```
|
| 55 |
+
|
| 56 |
+
## Inputs / Outputs
|
| 57 |
+
|
| 58 |
+
See the family [README](https://huggingface.co/Octopus1/PaGE) for the full input/output spec.
|
| 59 |
+
|
| 60 |
+
- Input dict: `images` (list of `[B,3,512,512]`), `head_images` (list of `[sum(Np),3,256,256]`),
|
| 61 |
+
`bboxes` (per-image list of `(xmin,ymin,xmax,ymax)` in `[0,1]`).
|
| 62 |
+
- Output dict: `heatmap` (list of `[Np,64,64]`, sigmoid), `inout` (list of `[Np]`, sigmoid).
|
| 63 |
+
|
| 64 |
+
## License
|
| 65 |
+
|
| 66 |
+
Apache-2.0.
|
config.json
ADDED
|
@@ -0,0 +1,57 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"architectures": [
|
| 3 |
+
"PaGEModel"
|
| 4 |
+
],
|
| 5 |
+
"dim": 256,
|
| 6 |
+
"dino_feature_dropout": 0.1,
|
| 7 |
+
"dinov3_drop_path_rate": 0.0,
|
| 8 |
+
"dinov3_hidden_size": 768,
|
| 9 |
+
"dinov3_intermediate_size": 3072,
|
| 10 |
+
"dinov3_layer_norm_eps": 1e-05,
|
| 11 |
+
"dinov3_layerscale_value": 1.0,
|
| 12 |
+
"dinov3_num_attention_heads": 12,
|
| 13 |
+
"dinov3_num_hidden_layers": 12,
|
| 14 |
+
"dinov3_num_register_tokens": 4,
|
| 15 |
+
"dinov3_patch_size": 16,
|
| 16 |
+
"dinov3_use_gated_mlp": false,
|
| 17 |
+
"drop_path": 0.1,
|
| 18 |
+
"dtype": "float32",
|
| 19 |
+
"head_in_size": [
|
| 20 |
+
256,
|
| 21 |
+
256
|
| 22 |
+
],
|
| 23 |
+
"heatmap_out_size": [
|
| 24 |
+
64,
|
| 25 |
+
64
|
| 26 |
+
],
|
| 27 |
+
"image_mean": [
|
| 28 |
+
0.485,
|
| 29 |
+
0.456,
|
| 30 |
+
0.406
|
| 31 |
+
],
|
| 32 |
+
"image_std": [
|
| 33 |
+
0.229,
|
| 34 |
+
0.224,
|
| 35 |
+
0.225
|
| 36 |
+
],
|
| 37 |
+
"inout": true,
|
| 38 |
+
"mlp_layer": "geglu",
|
| 39 |
+
"mlp_ratio": 4.0,
|
| 40 |
+
"model_type": "page",
|
| 41 |
+
"n_head_self_attn_layers": 1,
|
| 42 |
+
"n_reg_tokens": 4,
|
| 43 |
+
"n_scene_head_interaction_layers": 5,
|
| 44 |
+
"n_scene_self_attn_layers": 1,
|
| 45 |
+
"num_heads": 8,
|
| 46 |
+
"pos_encoding": "rope",
|
| 47 |
+
"scene_in_size": [
|
| 48 |
+
512,
|
| 49 |
+
512
|
| 50 |
+
],
|
| 51 |
+
"transformers_version": "4.56.0",
|
| 52 |
+
"use_head_prompt": false,
|
| 53 |
+
"auto_map": {
|
| 54 |
+
"AutoConfig": "Octopus1/PaGE--modeling_page.PaGEConfig",
|
| 55 |
+
"AutoModel": "Octopus1/PaGE--modeling_page.PaGEModel"
|
| 56 |
+
}
|
| 57 |
+
}
|
model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cd03a4e5807c188903f00056113b668a0980d4fd992db336eac9c766b93e0617
|
| 3 |
+
size 749309236
|
preprocessor_config.json
ADDED
|
@@ -0,0 +1,25 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"image_processor_type": "PaGEImageProcessor",
|
| 3 |
+
"auto_map": {
|
| 4 |
+
"AutoImageProcessor": "Octopus1/PaGE--modeling_page.PaGEImageProcessor"
|
| 5 |
+
},
|
| 6 |
+
"scene_size": [
|
| 7 |
+
512,
|
| 8 |
+
512
|
| 9 |
+
],
|
| 10 |
+
"head_size": [
|
| 11 |
+
256,
|
| 12 |
+
256
|
| 13 |
+
],
|
| 14 |
+
"image_mean": [
|
| 15 |
+
0.485,
|
| 16 |
+
0.456,
|
| 17 |
+
0.406
|
| 18 |
+
],
|
| 19 |
+
"image_std": [
|
| 20 |
+
0.229,
|
| 21 |
+
0.224,
|
| 22 |
+
0.225
|
| 23 |
+
],
|
| 24 |
+
"resample": 2
|
| 25 |
+
}
|