Image Feature Extraction
Transformers
Safetensors
English
keural_vision
vision
vision-encoder
image-text
contrastive-learning
knowledge-distillation
adaptive-tokenization
Eval Results (legacy)
Instructions to use mkd-hika/keural-vision-encoder-mid with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use mkd-hika/keural-vision-encoder-mid with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-feature-extraction", model="mkd-hika/keural-vision-encoder-mid")# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("mkd-hika/keural-vision-encoder-mid", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Replace overflowing ASCII architecture with a native Mermaid block diagram
Browse files
README.md
CHANGED
|
@@ -111,36 +111,15 @@ print(out.saliency_scores.shape) # (1, 512) per-token importance
|
|
| 111 |
|
| 112 |
## Architecture
|
| 113 |
|
| 114 |
-
```
|
| 115 |
-
|
| 116 |
-
|
| 117 |
-
|
| 118 |
-
|
| 119 |
-
|
| 120 |
-
|
| 121 |
-
|
| 122 |
-
|
| 123 |
-
βΌ
|
| 124 |
-
βββββββββββββββββββββββββββββ
|
| 125 |
-
β ATB Tokenizer β Gumbel-Top-K + NMS + differentiable weighted pooling
|
| 126 |
-
β budget split: β global 5% Β· region 25% Β· detail 70%
|
| 127 |
-
β 512 default (up to 2048) β
|
| 128 |
-
βββββββββββββββββββββββββββββ
|
| 129 |
-
β
|
| 130 |
-
βΌ
|
| 131 |
-
βββββββββββββββββββββββββββββ
|
| 132 |
-
β HCT levels β global / region / detail + learned level embeddings
|
| 133 |
-
βββββββββββββββββββββββββββββ
|
| 134 |
-
β
|
| 135 |
-
βΌ
|
| 136 |
-
βββββββββββββββββββββββββββββ
|
| 137 |
-
β Spatial Transformer (24L) β RMSNorm Β· SwiGLU Β· continuous 2D-RoPE
|
| 138 |
-
β + [POOL] token β log-area scale bias (8 buckets)
|
| 139 |
-
βββββββββββββββββββββββββββββ
|
| 140 |
-
β
|
| 141 |
-
βΌ
|
| 142 |
-
pooled (B, 768) + tokens (B, N, 768)
|
| 143 |
-
+ level_ids + spatial_metadata + saliency_scores
|
| 144 |
```
|
| 145 |
|
| 146 |
The `pooled` image embedding is the `[POOL]` token after the final RMSNorm (no separate projection head). For contrastive training it is aligned against a trainable projection of a frozen CLIP text encoder.
|
|
|
|
| 111 |
|
| 112 |
## Architecture
|
| 113 |
|
| 114 |
+
```mermaid
|
| 115 |
+
flowchart TD
|
| 116 |
+
IMG["RGB Image Β· (B,3,H,W)"] --> STEM["CNN Stem (4-stage)<br/>3 β 128 β 256 β 512 β 768 Β· stride 16Γ"]
|
| 117 |
+
STEM --> SAL["Saliency Predictor<br/>per-pixel importance map [0,1]"]
|
| 118 |
+
STEM --> ATB["ATB Tokenizer Β· Gumbel-Top-K + NMS<br/>weighted pooling Β· budget 512 (up to 2048)<br/>global 5% Β· region 25% Β· detail 70%"]
|
| 119 |
+
SAL --> ATB
|
| 120 |
+
ATB --> HCT["HCT levels<br/>global / region / detail + learned level embeddings"]
|
| 121 |
+
HCT --> TF["Spatial Transformer Γ24<br/>RMSNorm Β· SwiGLU Β· continuous 2D-RoPE<br/>log-area scale bias (8 buckets) Β· [POOL] token"]
|
| 122 |
+
TF --> OUT["pooled (B,768) + tokens (B,N,768)<br/>+ level_ids + spatial_metadata + saliency_scores"]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 123 |
```
|
| 124 |
|
| 125 |
The `pooled` image embedding is the `[POOL]` token after the final RMSNorm (no separate projection head). For contrastive training it is aligned against a trainable projection of a frozen CLIP text encoder.
|