File size: 2,839 Bytes
78b487c 403290b 78b487c 403290b 78b487c 403290b 78b487c 403290b 78b487c 403290b 78b487c 403290b 78b487c 403290b 78b487c 403290b 78b487c 403290b 78b487c 403290b 78b487c 403290b 78b487c 403290b 78b487c 403290b 78b487c 403290b 78b487c 403290b 78b487c 403290b 78b487c 403290b 78b487c 403290b 78b487c 403290b | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 | ---
license: cc-by-nc-4.0
base_model: feyninc/pulpie-orange-small
pipeline_tag: token-classification
tags:
- gguf
- eurobert
- token-classification
- html
- content-extraction
- pulpie
---
# Pulpie Orange Small — GGUF
GGUF conversion of [feyninc/pulpie-orange-small](https://huggingface.co/feyninc/pulpie-orange-small), a 210M parameter EuroBERT token-classification model for Pulpie HTML main-content extraction.
## ⚠️ Not a language model
This is an **encoder classifier**, not a causal LM. The GGUF files expose per-token `other`/`main` classifier logits via `llama-embedding`, not `llama-cli` generation.
## Files
| Quant | Size | Notes |
|-------|------|-------|
| F16 | 431 MB | Full precision baseline |
| Q8_0 | 233 MB | 8-bit, verified accurate |
| Q6_K | 182 MB | |
| Q5_K_M | 169 MB | |
| Q4_K_M | 157 MB | |
| Q3_K_M | 143 MB | |
| Q2_K | 130 MB | Most aggressive quantization |
## Verification
| Variant | Max diff vs PyTorch | E2E extraction | Prediction agreement |
|---------|--------------------|-----------------|---------------------|
| F16 | 0.070 | ✅ 3 main blocks | 100% |
| Q8_0 | 0.072 | ✅ 3 main blocks | 100% |
| Q6_K – Q2_K | not tested | not tested | — |
> **Note:** Only F16 and Q8_0 were numerically verified against the original PyTorch model. Lower quants (Q6_K → Q2_K) passed load + inference checks but output consistency was not validated. Use F16 or Q8_0 for production.
Full results: [`verification_report.json`](./verification_report.json)
## Usage
```bash
llama-embedding \
--model pulpie-orange-small-Q8_0.gguf \
--prompt "<html><body><p>Hello world</p></body></html><|sep|>" \
--pooling none \
--embd-normalize -1 \
--embd-output-format json
```
Each output row is `[other_logit, main_logit]`. Pulpie classifies at `<|sep|>` token positions and keeps blocks where `main_logit > other_logit`.
`llama-cli` can load the files but cannot generate — these GGUFs have no causal LM head.
## Python example
```python
import json, re, subprocess
out = subprocess.check_output([
"llama-embedding", "-m", "pulpie-orange-small-Q8_0.gguf",
"-p", html_chunk,
"--pooling", "none",
"--embd-normalize", "-1",
"--embd-output-format", "json",
], text=True)
data = json.loads(re.search(r'\{.*\}', out, re.S).group(0))
logits = [row["embedding"] for row in data["data"]]
```
## Conversion notes
Stock llama.cpp supports EuroBERT embeddings but not `EuroBertForTokenClassification`. A 103-line patch was applied to:
- Register `EuroBertForTokenClassification` in the HF converter
- Map `classifier.weight`/`classifier.bias` → GGUF `cls.output.*`
- Write classifier labels `["other", "main"]` with `embedding_length_out=2`
- Apply the classifier head in the EuroBERT runtime graph
Quantization was done with `llama-quantize` from the patched build.
|