geoffmunn commited on
Commit
ca2026c
·
verified ·
1 Parent(s): c497359

Em-dashes removed

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -23,11 +23,11 @@ pipeline_tag: text-classification
23
 
24
  The Qwen3Guard-Stream models are specialised versions of the Qwen3 series, designed specifically for real-time safety and moderation tasks rather than general-purpose language generation.
25
 
26
- These models incorporate custom architectural componentssuch as non-standard attention mechanisms, proprietary gating layers, or unique tokenizersthat deviate significantly from the Transformer-based architectures (like LLaMA, Mistral, or Gemma) that the GGUF format and the llama.cpp runtime were explicitly engineered to support.
27
 
28
  Because GGUF is not a universal model serialisation format but rather one tightly coupled to the structural assumptions of models compatible with llama.cpp, models like Qwen3Guard-Stream cannot be directly converted to GGUF, regardless of quantisation level.
29
 
30
- The absence of official support in the llama.cpp codebase for Qwen's architecture - especially its attention variants, rotary embedding scheme, and vocabulary layout means that **even a successful file conversion would result in runtime errors or incorrect inference behavior.**
31
 
32
  **This last point is where I ran into trouble** - I modified llama.cpp to convert the Qwen3Guard-Gen and SafeRL models, but the Stream models will not work.
33
 
 
23
 
24
  The Qwen3Guard-Stream models are specialised versions of the Qwen3 series, designed specifically for real-time safety and moderation tasks rather than general-purpose language generation.
25
 
26
+ These models incorporate custom architectural components - such as non-standard attention mechanisms, proprietary gating layers, or unique tokenizers - that deviate significantly from the Transformer-based architectures (like LLaMA, Mistral, or Gemma) that the GGUF format and the llama.cpp runtime were explicitly engineered to support.
27
 
28
  Because GGUF is not a universal model serialisation format but rather one tightly coupled to the structural assumptions of models compatible with llama.cpp, models like Qwen3Guard-Stream cannot be directly converted to GGUF, regardless of quantisation level.
29
 
30
+ The absence of official support in the llama.cpp codebase for Qwen's architecture - especially its attention variants, rotary embedding scheme, and vocabulary layout - means that **even a successful file conversion would result in runtime errors or incorrect inference behavior.**
31
 
32
  **This last point is where I ran into trouble** - I modified llama.cpp to convert the Qwen3Guard-Gen and SafeRL models, but the Stream models will not work.
33