--- license: cc-by-nc-4.0 base_model: feyninc/pulpie-orange-small pipeline_tag: token-classification tags: - gguf - eurobert - token-classification - html - content-extraction - pulpie --- # Pulpie Orange Small — GGUF GGUF conversion of [feyninc/pulpie-orange-small](https://huggingface.co/feyninc/pulpie-orange-small), a 210M parameter EuroBERT token-classification model for Pulpie HTML main-content extraction. ## ⚠️ Not a language model This is an **encoder classifier**, not a causal LM. The GGUF files expose per-token `other`/`main` classifier logits via `llama-embedding`, not `llama-cli` generation. ## Files | Quant | Size | Notes | |-------|------|-------| | F16 | 431 MB | Full precision baseline | | Q8_0 | 233 MB | 8-bit, verified accurate | | Q6_K | 182 MB | | | Q5_K_M | 169 MB | | | Q4_K_M | 157 MB | | | Q3_K_M | 143 MB | | | Q2_K | 130 MB | Most aggressive quantization | ## Verification | Variant | Max diff vs PyTorch | E2E extraction | Prediction agreement | |---------|--------------------|-----------------|---------------------| | F16 | 0.070 | ✅ 3 main blocks | 100% | | Q8_0 | 0.072 | ✅ 3 main blocks | 100% | | Q6_K – Q2_K | not tested | not tested | — | > **Note:** Only F16 and Q8_0 were numerically verified against the original PyTorch model. Lower quants (Q6_K → Q2_K) passed load + inference checks but output consistency was not validated. Use F16 or Q8_0 for production. Full results: [`verification_report.json`](./verification_report.json) ## Usage ```bash llama-embedding \ --model pulpie-orange-small-Q8_0.gguf \ --prompt "

Hello world

<|sep|>" \ --pooling none \ --embd-normalize -1 \ --embd-output-format json ``` Each output row is `[other_logit, main_logit]`. Pulpie classifies at `<|sep|>` token positions and keeps blocks where `main_logit > other_logit`. `llama-cli` can load the files but cannot generate — these GGUFs have no causal LM head. ## Python example ```python import json, re, subprocess out = subprocess.check_output([ "llama-embedding", "-m", "pulpie-orange-small-Q8_0.gguf", "-p", html_chunk, "--pooling", "none", "--embd-normalize", "-1", "--embd-output-format", "json", ], text=True) data = json.loads(re.search(r'\{.*\}', out, re.S).group(0)) logits = [row["embedding"] for row in data["data"]] ``` ## Conversion notes Stock llama.cpp supports EuroBERT embeddings but not `EuroBertForTokenClassification`. A 103-line patch was applied to: - Register `EuroBertForTokenClassification` in the HF converter - Map `classifier.weight`/`classifier.bias` → GGUF `cls.output.*` - Write classifier labels `["other", "main"]` with `embedding_length_out=2` - Apply the classifier head in the EuroBERT runtime graph Quantization was done with `llama-quantize` from the patched build.