mkd-hika commited on
Commit
708bba7
Β·
verified Β·
1 Parent(s): 24aece0

Replace overflowing ASCII architecture with a native Mermaid block diagram

Browse files
Files changed (1) hide show
  1. README.md +9 -30
README.md CHANGED
@@ -111,36 +111,15 @@ print(out.saliency_scores.shape) # (1, 512) per-token importance
111
 
112
  ## Architecture
113
 
114
- ```
115
- RGB Image (B, 3, H, W)
116
- β”‚
117
- β–Ό
118
- β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
119
- β”‚ CNN Stem (4-stage) β”‚ 3β†’128β†’256β†’512β†’768, cumulative stride 16Γ—
120
- β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
121
- β”‚
122
- β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Ί Saliency Predictor β†’ per-pixel importance map [0,1]
123
- β–Ό
124
- β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
125
- β”‚ ATB Tokenizer β”‚ Gumbel-Top-K + NMS + differentiable weighted pooling
126
- β”‚ budget split: β”‚ global 5% Β· region 25% Β· detail 70%
127
- β”‚ 512 default (up to 2048) β”‚
128
- β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
129
- β”‚
130
- β–Ό
131
- β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
132
- β”‚ HCT levels β”‚ global / region / detail + learned level embeddings
133
- β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
134
- β”‚
135
- β–Ό
136
- β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
137
- β”‚ Spatial Transformer (24L) β”‚ RMSNorm Β· SwiGLU Β· continuous 2D-RoPE
138
- β”‚ + [POOL] token β”‚ log-area scale bias (8 buckets)
139
- β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
140
- β”‚
141
- β–Ό
142
- pooled (B, 768) + tokens (B, N, 768)
143
- + level_ids + spatial_metadata + saliency_scores
144
  ```
145
 
146
  The `pooled` image embedding is the `[POOL]` token after the final RMSNorm (no separate projection head). For contrastive training it is aligned against a trainable projection of a frozen CLIP text encoder.
 
111
 
112
  ## Architecture
113
 
114
+ ```mermaid
115
+ flowchart TD
116
+ IMG["RGB Image Β· (B,3,H,W)"] --> STEM["CNN Stem (4-stage)<br/>3 β†’ 128 β†’ 256 β†’ 512 β†’ 768 Β· stride 16Γ—"]
117
+ STEM --> SAL["Saliency Predictor<br/>per-pixel importance map [0,1]"]
118
+ STEM --> ATB["ATB Tokenizer Β· Gumbel-Top-K + NMS<br/>weighted pooling Β· budget 512 (up to 2048)<br/>global 5% Β· region 25% Β· detail 70%"]
119
+ SAL --> ATB
120
+ ATB --> HCT["HCT levels<br/>global / region / detail + learned level embeddings"]
121
+ HCT --> TF["Spatial Transformer Γ—24<br/>RMSNorm Β· SwiGLU Β· continuous 2D-RoPE<br/>log-area scale bias (8 buckets) Β· [POOL] token"]
122
+ TF --> OUT["pooled (B,768) + tokens (B,N,768)<br/>+ level_ids + spatial_metadata + saliency_scores"]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
123
  ```
124
 
125
  The `pooled` image embedding is the `[POOL]` token after the final RMSNorm (no separate projection head). For contrastive training it is aligned against a trainable projection of a frozen CLIP text encoder.