Document verified official vision projector compatibility on RTX 5060 Ti

README.md

@@ -25,10 +25,12 @@ language:
 
 # Ornith 1.5 35B A3B AD-Q4 + Shisa 12K MTP GGUF
 
-Single-file, text-only GGUF combining AtomicChat's model-specific
+Single-file language-model GGUF combining AtomicChat's model-specific
 `AD-Q4_K-IQ4_XS` Ornith target with Shisa's 12K KL-distilled MTP head, quantized
 to `Q4_0`. It was assembled and tuned for `llama.cpp` on an NVIDIA RTX 5060 Ti
-16 GB while retaining the model's full 262,144-token context allocation.
+16 GB while retaining the model's full 262,144-token context allocation. The
+file itself does not contain a vision projector; compatibility with the
+official external Ornith BF16 `mmproj` is documented and verified below.
 
 中文摘要：这是 AtomicChat AD-Q4 主模型与 Shisa 12K 蒸馏 MTP 头的单文件
 GGUF 合并版。README 下面完整记录了权重来源、版本、量化/合并方法、运行补丁、
@@ -141,6 +143,30 @@ Why MTP1: on this machine MTP1 was faster than longer speculative depths.
 Setting `p-min=0` also beat the tested 0.30 and 0.60 thresholds. Q8 target and
 draft KV were retained for long-context quality.
 
+## Optional vision projector (not included)
+
+Vision was validated with the official external projector; it is deliberately
+not duplicated in this repository:
+
+- Repository: [`ornith-ai/Ornith-1.5-35B-A3B-GGUF`](https://huggingface.co/ornith-ai/Ornith-1.5-35B-A3B-GGUF)
+- Revision: `fbbaed45c2f0e200276ffa51701a24d45dc7f57e`
+- File: `mmproj-Ornith-1.5-35B-BF16.gguf`
+- Size: 902,822,016 bytes
+- SHA-256: `d9ce31026d1cb1f3f8d5152e2e2a014d9d2b302b6c93a7dc07bb0a0487f52837`
+- Declared license: MIT
+
+On a 16GB card, keep the projector in host RAM to preserve VRAM for Q8 KV and
+expert weights:
+
+```bash
+  --mmproj mmproj-Ornith-1.5-35B-BF16.gguf \
+  --no-mmproj-offload \
+  --image-min-tokens 1024
+```
+
+The remaining model, MTP, KV, fit, batch, and thread settings were identical to
+the 262K command above.
+
 ## RTX 5060 Ti 16GB results
 
 ### Test system
@@ -151,7 +177,7 @@ draft KV were retained for long-context quality.
 - Runtime: llama.cpp-based CUDA build at `4df29be4f`; an unrelated conditional
   `d2t` source change was present but inactive for this GGUF
 - Concurrency: one sequence
-- Workload: text only; no multimodal projector
+- Workload: text benchmarks plus the separate CPU-resident projector check below
 
 ### Memory placement at 262K
 
@@ -198,6 +224,23 @@ accepted 112 of 142 draft tokens (78.9%).
 This is a 27.9K-token performance check inside a fully allocated 262K context,
 not a claim that decode was benchmarked with 262K tokens already populated.
 
+### Vision compatibility check
+
+A 326KB JPEG of the Statue of Liberty was sent as an embedded base64 image, so
+the server did not receive a descriptive remote URL. With thinking disabled,
+the model returned:
+
+> The main landmark is the Statue of Liberty, located in New York City.
+
+- Image + text prompt: 1,687 tokens
+- CPU-projector/prompt processing: 33.43 seconds / 50.46 tok/s
+- Final answer: 16 tokens / 56.75 tok/s
+- MTP acceptance on the short final answer: 8 / 8
+- A separate 128-token thinking run decoded at 60.61 tok/s and accepted 57 / 69
+  MTP drafts (82.6%)
+- Peak observed GPU allocation: approximately 14,862 MiB
+- Result: correct landmark/city identification; no OOM or server restart
+
 ### 16GB stability boundary
 
 An aggressive all-Q8 experiment reduced `fit-target` to zero and moved roughly
@@ -218,7 +261,10 @@ stable profile.
 
 ## Limitations
 
-- Text-only testing; no `mmproj` is included.
+- No `mmproj` is included. Vision requires the separately downloaded official
+  projector cited above.
+- Vision validation currently covers one landmark image, not a broad OCR,
+  grounding, chart, document, or multimodal safety benchmark.
 - The MTP head was trained by Shisa with a 2,048-token training window. Shisa
   measured longer-context acceptance, but the training distribution does not
   directly cover 256K inputs.