Update model card: training complete, full eval results, benchmark comparisons, saliency evolution, loss curves

.gitattributes

@@ -34,3 +34,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 pipeline_preview.gif filter=lfs diff=lfs merge=lfs -text
+benchmark_comparison_chart.png filter=lfs diff=lfs merge=lfs -text
+benchmark_recall_curve_chart.png filter=lfs diff=lfs merge=lfs -text
+saliency_evolution_9ckpt.png filter=lfs diff=lfs merge=lfs -text
+training_loss_curves.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -26,7 +26,7 @@ pipeline_tag: image-feature-extraction
   <img alt="params" src="https://img.shields.io/badge/params-183.5M-blue"/>
   <img alt="from scratch" src="https://img.shields.io/badge/trained-from%20scratch-success"/>
   <img alt="precision" src="https://img.shields.io/badge/precision-bfloat16-informational"/>
-  <img alt="status" src="https://img.shields.io/badge/status-actively%20training-orange"/>
+  <img alt="status" src="https://img.shields.io/badge/status-training%20complete-success"/>
 </p>
 
 > Developed by **MKD Co., Ltd.**
@@ -54,9 +54,9 @@ It introduces three mechanisms not found together in existing encoders (CLIP, Si
 - **Research** into adaptive tokenization, hierarchical token typing, and resolution-elastic position encoding.
 
 ### Limitations
-- **Actively training.** This checkpoint is a snapshot of an in-progress run; weights and downstream quality will change until training completes.
-- **Zero-shot alignment maturity.** Image↔text alignment is still converging; broad zero-shot benchmark numbers are not yet published (see [Evaluation](#evaluation)).
-- **Domain/bias.** Trained on web image–text pairs (DataComp-Medium); it inherits the coverage and biases of that distribution and is English-text aligned.
+- **Accuracy trails same-size peers.** Zero-shot ImageNet Top-1 is 37.2%, versus 68.6% for CLIP ViT-B/16 (~86M params) and 76.7% for SigLIP ViT-B/16 (~93M params) — both smaller than this model. This is attributed to a training-budget gap (18K steps on ~20M pairs, vs. peers trained on 12M-400M+ pairs over far more steps) rather than an architecture ceiling: training loss was still declining at the final step. See [Evaluation](#evaluation).
+- **Saliency mechanism is real but token placement is still settling.** The learned saliency map produces stable, object-tracking contours on images with a clear dominant subject. However, the *discrete* tokens it samples continued shifting meaningfully until very late in training (only ~50% overlap with the final checkpoint's token positions as of step 13-14K, reaching ~73% by step 16K) — exact token placement had not fully converged even at the final checkpoint.
+- **Domain/bias.** Trained on web image–text pairs; it inherits the coverage and biases of that distribution and is English-text aligned.
 - **Not a safety-filtered model.** No content moderation or de-biasing has been applied.
 
 ---
@@ -85,7 +85,7 @@ image = Image.open("example.jpg").convert("RGB")
 pixel_values = transform(image).unsqueeze(0)          # (1, 3, 384, 384)
 
 with torch.no_grad():
-    out = model(pixel_values=pixel_values)
+    out = model(pixel_values=pixel_values, token_budget=512)  # match training exactly
 
 image_embedding = torch.nn.functional.normalize(out.pooled, dim=-1)  # (1, 768)
 
@@ -96,6 +96,13 @@ print(out.spatial_metadata.shape)  # (1, 512, 4)    cx, cy, scale_w, scale_h
 print(out.saliency_scores.shape)   # (1, 512)       per-token importance
 ```
 
+> **Note:** pass `token_budget=512` explicitly. Training always used a fixed
+> budget of 512 tokens; leaving `token_budget` unset falls back to
+> resolution-based auto-scaling (e.g. 1,152 tokens at 384px), which the model
+> was not trained on. Evaluation showed this mismatch moves results by <1
+> point, so it's a minor effect, but 512 is the methodologically correct
+> setting to match training.
+
 ### Output fields
 
 | Field | Shape | Description |
@@ -135,7 +142,7 @@ The `pooled` image embedding is the `[POOL]` token after the final RMSNorm (no s
 | FFN | SwiGLU (hidden = 3072) |
 | Normalization | RMSNorm |
 | Position encoding | Continuous 2D-RoPE + scale bias |
-| Token budget | 512 default, up to 2048 |
+| Token budget | 512 default (training-matched), up to 2048 |
 | Precision | bfloat16 |
 | Input resolution | 384×384, 448×448 |
 
@@ -145,22 +152,45 @@ The `pooled` image embedding is the `[POOL]` token after the final RMSNorm (no s
 
 | | |
 |---|---|
-| Data | ~20M image–text pairs (DataComp-Medium) |
+| Data | ~20M image–text pairs |
 | Hardware | 2× NVIDIA H200 (141 GB) |
 | Objective | SigLIP contrastive + HCT contrastive + saliency regularizer + dual-teacher cosine distillation |
 | Teachers (frozen) | SigLIP-SO400M (1152-d) + InternViT-300M (1024-d) |
 | Optimizer | AdamW, lr 5e-4, cosine schedule, 2K warmup, weight decay 0.05 |
 | Effective batch | 3,072 (768 × 2 grad-accum × 2 GPUs, distributed `all_gather` negatives) |
-| Schedule | up to 18,000 optimizer steps (~2.7 epochs) |
+| Schedule | 18,000 optimizer steps, complete |
 | Precision | bfloat16 (mixed) |
+| Final loss | 2.87 total (down from 21.56 at step 0), still declining at completion |
 
 **Loss.** `L_total = L_primary + λ_hct·L_hct + λ_sal·β_sal·L_saliency + L_distill`, where `L_primary` and `L_hct` are sigmoid image↔text losses (on the pooled embedding and on the mean of global tokens respectively), `L_saliency` is an anti-collapse + total-variation regularizer on the saliency map, and `L_distill` is cosine distance to the two frozen teachers.
 
+![Training loss curves](training_loss_curves.png)
+
 ---
 
 ## Evaluation
 
-Benchmarking is **in progress** alongside the active training run. Comprehensive zero-shot classification and image–text retrieval results will be published here once training completes and evaluation stabilizes. The current focus is correctness of the image↔text alignment space; standardized numbers are intentionally withheld until they are trustworthy.
+Evaluated on the final checkpoint with `token_budget=512` (exactly matching training).
+
+| Benchmark | Result |
+|---|---|
+| ImageNet zero-shot Top-1 | **37.2%** |
+| ImageNet zero-shot Top-5 | 66.8% |
+| Flickr30K Image→Text R@1 / R@5 / R@10 | 42.9% / 72.7% / 80.6% |
+| Flickr30K Text→Image R@1 / R@5 / R@10 | 42.2% / 70.0% / 79.4% |
+| CIFAR-100 linear probe Top-1 | 75.0% |
+
+![Benchmark comparison](benchmark_comparison_chart.png)
+
+![Flickr30K Recall@k](benchmark_recall_curve_chart.png)
+
+**Context.** Against its own distillation teachers, Keural Mid retains ~45% of SigLIP-SO400M's zero-shot ImageNet accuracy (83.1%) and ~48-56% of its Flickr30K retrieval R@1, after 18K steps on ~20M pairs vs. teachers trained on billions of pairs. Against same-parameter-class peers (CLIP ViT-B/16, SigLIP ViT-B/16 — both smaller than this model), accuracy trails by roughly 2x. Both gaps are consistent with a **training-budget** shortfall rather than an architecture ceiling: loss was still declining at the final step. A compute/data-matched fixed-grid baseline is the recommended next experiment to isolate the ATB architecture's own contribution from this gap.
+
+### Adaptive tokenizer — saliency & token placement
+
+![Saliency evolution across training](saliency_evolution_9ckpt.png)
+
+The learned saliency map produces sharp, stable contours that track each image's dominant subject, visible from mid-training through the final checkpoint. Quantitatively, the continuous saliency map stabilizes fast (cosine similarity to the final checkpoint reaches ~0.83-0.90 within the first few thousand steps), but the *discrete* sampled token positions are considerably less stable — only ~50% overlap with the final checkpoint's tokens by step 13-14K, jumping to 73% at step 16K. Exact token placement was still moving late into training even where the saliency field itself looked converged.
 
 ---
 
@@ -186,6 +216,12 @@ Both teachers are frozen and used only during training; they are not required fo
 | Position encoding | Learned absolute | Learned absolute | Learned absolute | **Continuous 2D-RoPE** |
 | Saliency-aware | No | No | No | **Yes** |
 
+| Same-parameter-class peer | Params | ImageNet Zero-shot Top-1 | Flickr30K I2T R@1 |
+|---|---|---|---|
+| **Keural Mid** | 183.6M | 37.2% | 42.9% |
+| CLIP ViT-B/16 (OpenAI) | ~86M | 68.6% | 88.2% |
+| SigLIP ViT-B/16 | ~93M | 76.7% | — |
+
 ---
 
 ## Roadmap
@@ -193,7 +229,7 @@ Both teachers are frozen and used only during training; they are not required fo
 | Stage | Params | Status |
 |---|---|---|
 | PoC | 24.7M | Complete |
-| **Mid** | **183.5M** | **Training** |
+| **Mid** | **183.5M** | **Training complete — evaluation published** |
 | Large | ~1.1B | Planned |
 
 ---