FoolDev Claude Fable 5 commited on
Commit
87a254d
Β·
1 Parent(s): d9d9f49

docs: fix hardware-table eval contradiction + card consistency touches

Browse files

From a vetted suggestion pass on the card:
- Known-limitations note claimed the hardware table is "estimates, not
measured," but the Z13 row records a measured 28.71 tok/s β€” reworded to
carve out the measured row (the real defect).
- examples/README.md Setup now states `cd examples` up front; its first
runnable command silently assumed that cwd (a fresh clone lands at root).
- Loader-table `Ollama 0.24` -> `0.24+`, matching the open-ended form used
everywhere else (the behavior holds for every version since).
- Added `reasoning` + `distillation` frontmatter tags (CITATION.cff already
lists both as keywords; the card is positioned reasoning-first + distilled).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Files changed (3) hide show
  1. CHANGELOG.md +11 -0
  2. README.md +4 -2
  3. examples/README.md +2 -0
CHANGELOG.md CHANGED
@@ -15,6 +15,9 @@ track the **tooling and documentation**, not the underlying base model.
15
  verbatim; the vision client is skipped (it uses a deliberate image-specific
16
  prompt). Catches an example silently drifting from the shipped prompt β€” the
17
  gap that let the `llama_cpp_quickstart.py` prompt truncate (see Fixed).
 
 
 
18
 
19
  ### Changed (default sampling matched to the Fable teacher)
20
  - **Default sampling raised to a Fable-matched profile** in `params` and the
@@ -26,6 +29,14 @@ track the **tooling and documentation**, not the underlying base model.
26
  override.
27
 
28
  ### Fixed
 
 
 
 
 
 
 
 
29
  - **`scripts/load_bundle.sh` LFS-pointer size check is now portable.** The
30
  bundle resolver probed file size with GNU-only `stat -c '%s'`, which errors on
31
  macOS/BSD `stat` (that needs `-f%z`) β€” leaking a stderr line on an advertised
 
15
  verbatim; the vision client is skipped (it uses a deliberate image-specific
16
  prompt). Catches an example silently drifting from the shipped prompt β€” the
17
  gap that let the `llama_cpp_quickstart.py` prompt truncate (see Fixed).
18
+ - **Frontmatter tags `reasoning` + `distillation`.** The card positions Janus
19
+ as reasoning-first and distilled (and `CITATION.cff` already lists both as
20
+ keywords), but neither was a searchable Hub tag β€” added both.
21
 
22
  ### Changed (default sampling matched to the Fable teacher)
23
  - **Default sampling raised to a Fable-matched profile** in `params` and the
 
29
  override.
30
 
31
  ### Fixed
32
+ - **Hardware-table eval disclaimer no longer self-contradicts.** "Known
33
+ limitations" said the table numbers are "estimates, not measured," yet the
34
+ Z13 row records a measured 28.71 tok/s β€” reworded to carve out that row.
35
+ - **`examples/README.md` establishes its working directory.** The Setup
36
+ section's first runnable command assumed a `cd examples` that was never
37
+ stated (a fresh clone lands at repo root); added the note.
38
+ - **Loader-table Ollama version reads `0.24+`.** The lone bare `Ollama 0.24`
39
+ cell now matches the open-ended `0.24+` used everywhere else.
40
  - **`scripts/load_bundle.sh` LFS-pointer size check is now portable.** The
41
  bundle resolver probed file size with GNU-only `stat -c '%s'`, which errors on
42
  macOS/BSD `stat` (that needs `-f%z`) β€” leaking a stderr line on an advertised
README.md CHANGED
@@ -42,6 +42,8 @@ tags:
42
  - gguf
43
  - heretic
44
  - uncensored
 
 
45
  library_name: transformers
46
  pipeline_tag: image-text-to-text
47
  ---
@@ -248,7 +250,7 @@ The projector and other-quant text decoders live at
248
  |---|---|---|---|
249
  | **llama.cpp** (`llama-mtmd-cli`, `llama-server --mmproj`) | βœ… | βœ… | Reference path. Upstream has the `qwen35moe` arch entry. |
250
  | **llama-cpp-python** | βœ… | βœ… | See `examples/llama_cpp_vision.py`. |
251
- | **Ollama 0.24** | βœ… | ❌ | Text inference works: Ollama's Go engine has the `qwen35` / `qwen35moe` arch entries. Vision (mmproj) is still broken: the C++ llama.cpp fallback that Ollama switches to when an mmproj is attached lacks those entries. `ollama create` accepts a dual-`FROM` (text + mmproj) and `ollama show` reports `vision` capability β€” but the **first inference request** fails with `error loading model architecture: unknown model architecture: 'qwen35moe'`, and once mmproj is attached this blocks text inference too. See [ollama/ollama#14575](https://github.com/ollama/ollama/issues/14575) (open β€” the earlier #15898 was closed as its duplicate, and the sync PR #15899 was closed unmerged). |
252
  | **LM Studio** | βœ… | βœ… | Uses upstream llama.cpp directly. |
253
 
254
  ### Vision via llama.cpp
@@ -431,7 +433,7 @@ print(resp.choices[0].message.tool_calls)
431
  - **Thinking traces can loop.** Like most reasoning-distilled models, Janus-35B occasionally gets stuck repeating itself inside `<think>` tags. Mitigations: lower temperature to 0.4-0.6, raise `repeat_penalty` to 1.08, or set a `<think>`-token budget cap if your loader supports it.
432
  - **Large tool-call arguments can be dropped.** Ollama's JSON-in-XML tool format makes the model JSON-escape the entire arguments object inline; for a big/complex payload (e.g. a file's `content` in a `write_file` call) the model can fail to escape it, so the field arrives `undefined` and the call fails. Qwen's native `<function=…><parameter=…>` format (raw values, no escaping) was tested as a fix but parses unreliably through Ollama, so the template deliberately keeps JSON-in-XML. Mitigation: write large files in smaller pieces per call.
433
  - **Uncensored base β€” not aligned with any specific safety policy.** This is a personal repackage of an open-weight base whose refusal behavior has been abliterated away (the llmfan46 Heretic base). There is no RLHF refusal layer; the model will attempt most requests, so downstream safety is entirely the operator's responsibility.
434
- - **No formal evaluation in this card.** Numbers in the hardware table are estimates, not measured. If you produce real benchmarks (MMLU, HumanEval, etc.) and want them included, file a PR.
435
 
436
  ## Related models
437
 
 
42
  - gguf
43
  - heretic
44
  - uncensored
45
+ - reasoning
46
+ - distillation
47
  library_name: transformers
48
  pipeline_tag: image-text-to-text
49
  ---
 
250
  |---|---|---|---|
251
  | **llama.cpp** (`llama-mtmd-cli`, `llama-server --mmproj`) | βœ… | βœ… | Reference path. Upstream has the `qwen35moe` arch entry. |
252
  | **llama-cpp-python** | βœ… | βœ… | See `examples/llama_cpp_vision.py`. |
253
+ | **Ollama 0.24+** | βœ… | ❌ | Text inference works: Ollama's Go engine has the `qwen35` / `qwen35moe` arch entries. Vision (mmproj) is still broken: the C++ llama.cpp fallback that Ollama switches to when an mmproj is attached lacks those entries. `ollama create` accepts a dual-`FROM` (text + mmproj) and `ollama show` reports `vision` capability β€” but the **first inference request** fails with `error loading model architecture: unknown model architecture: 'qwen35moe'`, and once mmproj is attached this blocks text inference too. See [ollama/ollama#14575](https://github.com/ollama/ollama/issues/14575) (open β€” the earlier #15898 was closed as its duplicate, and the sync PR #15899 was closed unmerged). |
254
  | **LM Studio** | βœ… | βœ… | Uses upstream llama.cpp directly. |
255
 
256
  ### Vision via llama.cpp
 
433
  - **Thinking traces can loop.** Like most reasoning-distilled models, Janus-35B occasionally gets stuck repeating itself inside `<think>` tags. Mitigations: lower temperature to 0.4-0.6, raise `repeat_penalty` to 1.08, or set a `<think>`-token budget cap if your loader supports it.
434
  - **Large tool-call arguments can be dropped.** Ollama's JSON-in-XML tool format makes the model JSON-escape the entire arguments object inline; for a big/complex payload (e.g. a file's `content` in a `write_file` call) the model can fail to escape it, so the field arrives `undefined` and the call fails. Qwen's native `<function=…><parameter=…>` format (raw values, no escaping) was tested as a fix but parses unreliably through Ollama, so the template deliberately keeps JSON-in-XML. Mitigation: write large files in smaller pieces per call.
435
  - **Uncensored base β€” not aligned with any specific safety policy.** This is a personal repackage of an open-weight base whose refusal behavior has been abliterated away (the llmfan46 Heretic base). There is no RLHF refusal layer; the model will attempt most requests, so downstream safety is entirely the operator's responsibility.
436
+ - **No formal evaluation in this card.** Most numbers in the hardware table are estimates; the Z13 row (28.71 tok/s at Q4_K_M) is measured. If you produce real benchmarks (MMLU, HumanEval, etc.) and want them included, file a PR.
437
 
438
  ## Related models
439
 
examples/README.md CHANGED
@@ -18,6 +18,8 @@ so expect warmer sampling from the Ollama path.
18
 
19
  ## Setup
20
 
 
 
21
  ### Ollama
22
 
23
  Pull straight from HF (gets the bundled Q4_K_M GGUF + this repo's
 
18
 
19
  ## Setup
20
 
21
+ > These commands run from this `examples/` directory β€” `cd examples` first (the main README clones into the repo root).
22
+
23
  ### Ollama
24
 
25
  Pull straight from HF (gets the bundled Q4_K_M GGUF + this repo's