Instructions to use UWV/leesplank-noot-eurollm-1.7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use UWV/leesplank-noot-eurollm-1.7b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="UWV/leesplank-noot-eurollm-1.7b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("UWV/leesplank-noot-eurollm-1.7b")
model = AutoModelForCausalLM.from_pretrained("UWV/leesplank-noot-eurollm-1.7b", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use UWV/leesplank-noot-eurollm-1.7b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "UWV/leesplank-noot-eurollm-1.7b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "UWV/leesplank-noot-eurollm-1.7b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/UWV/leesplank-noot-eurollm-1.7b

SGLang

How to use UWV/leesplank-noot-eurollm-1.7b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "UWV/leesplank-noot-eurollm-1.7b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "UWV/leesplank-noot-eurollm-1.7b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "UWV/leesplank-noot-eurollm-1.7b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "UWV/leesplank-noot-eurollm-1.7b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use UWV/leesplank-noot-eurollm-1.7b with Docker Model Runner:
```
docker model run hf.co/UWV/leesplank-noot-eurollm-1.7b
```

leesplank-noot-eurollm-1.7b / README.md

yhavinga

Update README.md

44a3b73 verified 9 months ago

preview code

Raw

History Blame Contribute Delete

7.5 kB

	---
	license: apache-2.0
	language: nl
	tags:
	- text-simplification
	- dutch
	- accessibility
	- government
	metrics:
	- sari
	base_model: utter-project/EuroLLM-1.7B
	datasets: UWV/Leesplank_NL_wikipedia_simplifications_preprocessed
	library_name: transformers
	---

	# Leesplank Noot EuroLLM-1.7B

	Dutch text simplification model achieving SARI 66.44 ± 0.32 at 27.5 tokens/second

	## Model Details

	* Model: `UWV/leesplank-noot-eurollm-1.7b`
	* Base: EuroLLM-1.7B (utter-project/EuroLLM-1.7B)
	* Task: Dutch text simplification for government communication
	* Performance: SARI 66.44 at 27.5 tokens/second
	* License: Apache 2.0

	This model fine-tunes EuroLLM-1.7B on 1.89M Dutch Wikipedia simplifications to produce accessible text for citizens with reading difficulties, compliant with EU accessibility guidelines and the AI Act.

	## Quick Start

	```python
	from transformers import pipeline

	# Load EuroLLM model
	model = pipeline(
	"text-generation",
	model="UWV/leesplank-noot-eurollm-1.7b",
	torch_dtype="auto",
	device_map="auto"
	)

	# Text to simplify
	text = "Een pekdruppelexperiment is een langetermijnexperiment dat het vloeien van een stuk pek meet over vele jaren. Pek is een verzamelnaam voor een aantal vloeistoffen met een zeer hoge viscositeit, zoals teer en bitumen, die er bij kamertemperatuur uitzien als een vaste stof, maar in feite zeer dik vloeibaar zijn en uiteindelijk druppels vormen."

	# EuroLLM: no system prompt
	messages = [{
	"role": "user",
	"content": f"Vereenvoudig: {text}"
	}]

	# Generate simplified text
	output = model(
	messages,
	max_new_tokens=150,
	return_full_text=False,
	do_sample=False, # Greedy decoding for consistency
	eos_token_id=model.tokenizer.eos_token_id
	)

	# Print results
	print(f"Original: {text}")
	print(f"\nSimplified: {output[0]['generated_text']}")
	```

	## Architecture & Training

	### Model Architecture
	- Parameters: 1.7B total
	- Layers: 24 transformer blocks
	- Hidden Size: 2048
	- Attention Heads: 16 (with 8 KV heads using GQA)
	- MLP Dimension: 5632
	- Context Length: 1024 tokens (training max_seq_length), 4096 theoretical limit
	- Precision: bfloat16 (no quantization)
	- Activation: SwiGLU
	- Position Encoding: RoPE (θ=10000)
	- Vocabulary Size: 128,000

	### Training Configuration
	```yaml
	### Training Configuration
	learning_rate: 7e-5
	epochs: 2
	warmup_steps: 200
	scheduler: cosine
	max_gradient_norm: 1.0
	optimizer: AdamW (β₁=0.9, β₂=0.999, ε=1e-8)
	neftune_noise_alpha: 5
	per_device_train_batch_size: 32
	gradient_accumulation_steps: 2
	packing_enabled: true (~5x efficiency gain)
	eval_steps: 100
	save_steps: 200
	logging_steps: 10
	```

	### Hardware & Distributed Training
	- GPUs: 4× AMD MI300X (192GB total VRAM)
	- FSDP: Full sharding with activation checkpointing
	- Effective Batch Size: 256 (32 × 4 GPUs × 2 gradient accumulation)
	- Packing: Enabled for efficient sequence batching
	- Total Training Steps: 2,960
	- Training Time: 6 hours 47 minutes (2.0 epochs, 2,960 steps)
	- Peak Memory: 71GB per GPU


	### Training Progress

	The following charts show the training dynamics over 2,960 steps (2 epochs with sequence packing):

	![Training Loss](./train_loss_noot_eurollm.png)
	Training loss showing rapid initial convergence from 2.0 to ~1.3 within 500 steps, stabilizing at 1.127

	![Evaluation Loss](./eval_loss_noot_eurollm.png)
	Evaluation loss measured every 100 steps, final value: 1.041

	Key observations:
	- Rapid convergence in first 500 steps due to effective initialization from base model
	- Smooth cosine learning rate schedule with 200 warmup steps
	- Consistent improvement throughout both epochs
	- Stable training with no signs of overfitting (eval loss < train loss)

	### Dataset Engineering

	The source dataset (`UWV/Leesplank_NL_wikipedia_simplifications_preprocessed`) is pre-sorted by Levenshtein (edit) distance between original and simplified text, grouping similar simplification patterns together.

	Our training strategy combines two approaches to leverage this structure:

	1. 70% Ordered Samples: Preserves the Levenshtein distance ordering, allowing the model to learn progressively from similar simplification patterns
	2. 30% Shuffled Samples: Breaks domain clusters, improves generalization across diverse simplification types

	Each sample receives one of 20 instruction variants:
	```python
	SIMPLIFICATION_PROMPTS = [
	"Vereenvoudig deze tekst:",
	"Kun je deze tekst makkelijker maken?",
	"Herschrijf dit in eenvoudigere taal:",
	# ... 17 more variations
	]
	# 20% get short prompt "Vereenvoudig:" for length diversity
	```

	Dataset statistics:
	- Training: 1,886,765 paragraph pairs
	- Validation: 1,024 samples (held-out)
	- Median Length: 312 tokens (99.4% < 1024)

	## Performance Metrics

	### Primary Benchmark (3 runs × 1000 samples, beam_search)

	\| Metric \| EuroLLM-Leesplank \|
	\|--------\|-------------------\|
	\| SARI Mean \| 66.44 ± 0.32 \|
	\| 95% CI \| [65.66, 67.23] \|
	\| Speed (tok/s) \| 19.04 \|
	\| Time/sample \| 2.31s \|
	\| Median SARI \| 66.08 \|
	\| Q1-Q3 Range \| 57.97-73.71 \|
	\| Std Dev \| 12.56-12.92 \|

	### Decoding Strategy Comparison (100 samples)

	\| Strategy \| SARI Score \| Speed (tok/s) \| Recommendation \|
	\|----------\|------------\|---------------\|----------------\|
	\| Greedy \| 66.69 ± 11.48 \| 27.50 \| ✓ Production \|
	\| Beam (n=5) \| 66.81 ± 12.56 \| 19.04 \| Research only \|

	Key Finding: Beam search provides negligible quality improvement (+0.12 SARI) for 31% speed penalty. Use greedy decoding in production.

	## Usage Instructions

	### 1. Never Use System Prompts

	This model was trained without system prompts. Including one catastrophically degrades performance:

	```python
	# WRONG - SARI drops to ~28
	messages = [
	{"role": "system", "content": "Je bent een assistent..."},
	{"role": "user", "content": "Vereenvoudig: ..."}
	]

	# CORRECT - SARI 66.44
	messages = [
	{"role": "user", "content": "Vereenvoudig: ..."}
	]
	```

	### 2. Optimal Generation Settings

	```python
	generation_config = {
	"max_new_tokens": 256, # Sufficient for paragraph simplification
	"do_sample": False, # Greedy decoding for speed
	"temperature": 1.0, # Ignored with do_sample=False
	"top_p": 1.0, # Ignored with do_sample=False
	"repetition_penalty": 1.0, # No penalty needed
	"pad_token_id": tokenizer.eos_token_id,
	}
	```

	## Limitations & Biases

	### Scope Limitations
	1. Language: Dutch only, no multilingual capability
	2. Domain: Optimized for Wikipedia/government text, may underperform on medical/legal documents
	3. Granularity: Paragraph-level simplification only (not word or sentence level)
	4. Context: 1024 token practical limit (theoretical 4096)

	### Known Biases
	- Simplification Style: Tends toward Wikipedia Simple Dutch conventions
	- Explanation Preference: Often adds explanatory clauses for complex terms
	- Length Bias: May expand text when explaining concepts (simplification ≠ shortening)

	## Deployment Recommendations

	### Hardware Requirements
	- Minimum: 8GB VRAM (RTX 3060, A10)
	- Recommended: 16GB VRAM (RTX 4080, A40)
	- CPU Inference: Possible but ~20x slower

	## Citation

	```bibtex
	@software{leesplank_noot_2025,
	author = {UWV InnovatieHub},
	title = {Leesplank Noot: Dutch Text Simplification},
	year = {2025},
	publisher = {HuggingFace},
	url = {https://huggingface.co/UWV/leesplank-noot-eurollm-1.7b}
	}
	```