| --- |
| license: cc-by-nc-4.0 |
| library_name: mlx |
| pipeline_tag: token-classification |
| base_model: feyninc/pulpie-orange-small |
| tags: |
| - mlx |
| - eurobert |
| - token-classification |
| - html |
| - content-extraction |
| - boilerplate-removal |
| - web-scraping |
| - encoder |
| - custom-code |
| --- |
| |
| # Pulpie Orange Small MLX |
|
|
| This repository contains MLX weights for |
| [`feyninc/pulpie-orange-small`](https://huggingface.co/feyninc/pulpie-orange-small), |
| a 210M-parameter EuroBERT token-classification model for main-content extraction |
| from HTML. |
|
|
| The source checkpoint is an encoder-only EuroBERT model with RoPE, RMSNorm, |
| SwiGLU MLP layers, and a token-classification head. It is not a decoder-only |
| LLM, so this conversion does not use `mlx-lm`'s standard LLM model classes. |
| Instead, this repository includes `modeling_eurobert_mlx.py`, a small MLX |
| implementation of the source architecture with matching parameter names. |
|
|
| ## Files |
|
|
| | File | Purpose | |
| | --- | --- | |
| | `model-bf16.safetensors` | Native 16-bit BF16 MLX weights converted from the source checkpoint. | |
| | `model-8bit.safetensors` | MLX affine 8-bit weight-quantized variant. | |
| | `model-4bit.safetensors` | MLX affine 4-bit weight-quantized variant. | |
| | `modeling_eurobert_mlx.py` | MLX EuroBERT token-classification loader. | |
| | `mlx_config.json` | Variant metadata and quantization settings. | |
| | `verification_report.json` | Load, numerical, extraction, latency, and compute-cost results. | |
| | `scripts/convert_to_mlx.py` | Reproducible conversion script. | |
| | `scripts/verify_mlx.py` | Reproducible verification script. | |
|
|
| ## Usage |
|
|
| Install dependencies: |
|
|
| ```bash |
| pip install -r requirements.txt |
| ``` |
|
|
| Run a forward pass: |
|
|
| ```python |
| import sys |
| import mlx.core as mx |
| from huggingface_hub import snapshot_download |
| from transformers import AutoTokenizer |
| |
| repo_dir = snapshot_download("Mike0021/pulpie-orange-small-mlx") |
| sys.path.insert(0, repo_dir) |
| |
| from modeling_eurobert_mlx import load_model |
| |
| tokenizer = AutoTokenizer.from_pretrained(repo_dir, trust_remote_code=True) |
| model = load_model(repo_dir, variant="bf16") # "bf16", "8bit", or "4bit" |
| |
| inputs = tokenizer( |
| ["<h1>Apple MLX conversion</h1><p>Main article text.</p>"], |
| return_tensors="np", |
| padding=True, |
| ) |
| input_ids = mx.array(inputs["input_ids"]) |
| attention_mask = mx.array(inputs["attention_mask"]) |
| |
| logits = model(input_ids, attention_mask) |
| mx.eval(logits) |
| print(logits.shape) # batch, tokens, 2 |
| ``` |
|
|
| Minimal Pulpie-style extraction: |
|
|
| ```python |
| import numpy as np |
| import mlx.core as mx |
| from pulpie.chunker import extract_blocks, pack_chunks, tokenize_blocks |
| from pulpie.model_utils import extract_item_ids, predictions_to_labels |
| from pulpie.reconstruct import extract_main_html |
| from pulpie.simplify import simplify |
| |
| html = ( |
| "<html><body><article><h1>Apple MLX conversion</h1>" |
| "<p>This article explains how to convert a EuroBERT content extraction " |
| "model to MLX format.</p></article></body></html>" |
| ) |
| |
| simplified, map_html = simplify(html) |
| blocks = extract_blocks(simplified) |
| item_ids = extract_item_ids(blocks) |
| sep_id = tokenizer.convert_tokens_to_ids("<|sep|>") |
| chunks = pack_chunks( |
| tokenize_blocks(blocks, tokenizer), |
| max_tokens=8192, |
| sep_token_id=sep_id, |
| bos_token_id=tokenizer.bos_token_id, |
| eos_token_id=tokenizer.eos_token_id, |
| ) |
| |
| predictions = [0] * len(blocks) |
| for chunk_ids, block_indices in chunks: |
| ids = mx.array([chunk_ids]) |
| mask = mx.ones_like(ids) |
| logits = model(ids, mask) |
| mx.eval(logits) |
| logits_np = np.array(logits.astype(mx.float32))[0] |
| sep_positions = np.where(np.array(chunk_ids) == sep_id)[0] |
| preds = logits_np[sep_positions].argmax(axis=-1).tolist() |
| for i, block_idx in enumerate(block_indices): |
| predictions[block_idx] = int(preds[i]) |
| |
| labels = predictions_to_labels(item_ids, predictions) |
| print(extract_main_html(map_html, labels)) |
| ``` |
|
|
| ## Conversion Methodology |
|
|
| 1. Downloaded `feyninc/pulpie-orange-small` from the Hugging Face Hub. |
| 2. Inspected `config.json`, `configuration_eurobert.py`, and |
| `modeling_eurobert.py`. |
| 3. Confirmed the model is `EuroBertForTokenClassification` with 12 layers, |
| hidden size 768, 12 attention heads, head dim 64, max length 8192, BF16 |
| source weights, and 2 output labels. |
| 4. Confirmed current MLX can be installed on Linux with `mlx[cpu]`, so no cloud |
| Mac was required for conversion or load verification. |
| 5. Implemented a custom MLX EuroBERT token-classification module with matching |
| state-dict keys and the source architecture behavior. |
| 6. Saved the BF16 MLX weights with `mlx.core.save_safetensors`. |
| 7. Created 8-bit and 4-bit variants with `mlx.nn.quantize`, using affine |
| weight quantization, group size 64, over MLX `Linear` and `Embedding` |
| modules. |
| 8. Verified each variant with `scripts/verify_mlx.py`. |
|
|
| ## Verification Results |
|
|
| Verification was run on Linux x86_64 using `mlx[cpu]`. The PyTorch reference was |
| the original source checkpoint loaded in float32 with eager attention. The BF16 |
| variant is expected to have small dtype-level differences versus that float32 |
| reference; quantized variants have larger differences. |
| |
| ### Load Checks |
| |
| | Variant | Load result | Test logits shape | Test logits dtype | |
| | --- | --- | --- | --- | |
| | BF16 | Pass | `[1, 3, 2]` | `mlx.core.bfloat16` | |
| | 8-bit | Pass | `[1, 3, 2]` | `mlx.core.bfloat16` | |
| | 4-bit | Pass | `[1, 3, 2]` | `mlx.core.bfloat16` | |
| |
| ### Numerical Accuracy |
| |
| Test inputs: `["A", "B", "C"]`, token shape `[3, 2]`. |
| |
| | Variant | Max abs diff vs PyTorch fp32 | Mean abs diff | MLX CPU latency | |
| | --- | ---: | ---: | ---: | |
| | BF16 | 0.0452327728 | 0.0191817340 | 716.88 ms | |
| | 8-bit | 1.2797489166 | 0.5423613191 | 9380.58 ms | |
| | 4-bit | 2.2551989555 | 1.1897996664 | 9323.18 ms | |
| |
| PyTorch fp32 eager latency on the same input was 50.09 ms on this Linux CPU. |
| The quantized MLX CPU path is slow on this host and should not be read as an |
| Apple Silicon benchmark. |
| |
| ### End-to-End Extraction |
| |
| Sample HTML: |
| |
| ```html |
| <html><body><article><h1>Apple MLX conversion</h1><p>This article explains how to convert a EuroBERT content extraction model to MLX format.</p></article></body></html> |
| ``` |
| |
| Pulpie preprocessing produced 2 blocks and one 50-token chunk. All variants |
| loaded, classified both blocks as `main`, and reconstructed non-empty HTML. |
| |
| | Variant | Predictions | Non-empty output | MLX CPU extraction latency | |
| | --- | --- | --- | ---: | |
| | BF16 | `[1, 1]` | Pass | 5469.05 ms | |
| | 8-bit | `[1, 1]` | Pass | 78333.61 ms | |
| | 4-bit | `[1, 1]` | Pass | 77852.02 ms | |
| |
| Full machine-readable results are in `verification_report.json`. |
|
|
| ## Limitations |
|
|
| - This is a custom MLX encoder/token-classification implementation, not an |
| `mlx-lm` decoder model. |
| - The 8-bit and 4-bit variants are weight-only affine MLX quantizations. They |
| load and pass a small extraction test, but full WebMainBench quality was not |
| re-evaluated. |
| - Linux CPU quantized latency is poor in this environment. MLX is primarily |
| intended for Apple Silicon GPU execution. |
| - The source tokenizer currently emits a Transformers regex warning. The |
| verifier keeps the tokenizer behavior used by the published `pulpie` package |
| rather than changing token IDs during conversion. |
|
|
| ## Compute Cost |
|
|
| No paid cloud Mac or hosted GPU was used. Conversion and verification were done |
| locally on Linux x86_64 with the MLX CPU package. Incremental compute cost: |
| `$0.00`. |
| |
| ## License |
| |
| The source model weights are licensed under CC BY-NC 4.0. This converted |
| checkpoint follows the same non-commercial license. The included conversion and |
| loader code is provided for interoperability with the converted weights. |
| |
