Instructions to use birgermoell/oellm-9b-256k-theta64m-prelude with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use birgermoell/oellm-9b-256k-theta64m-prelude with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="birgermoell/oellm-9b-256k-theta64m-prelude")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("birgermoell/oellm-9b-256k-theta64m-prelude")
model = AutoModelForCausalLM.from_pretrained("birgermoell/oellm-9b-256k-theta64m-prelude")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use birgermoell/oellm-9b-256k-theta64m-prelude with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "birgermoell/oellm-9b-256k-theta64m-prelude"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "birgermoell/oellm-9b-256k-theta64m-prelude",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/birgermoell/oellm-9b-256k-theta64m-prelude

SGLang

How to use birgermoell/oellm-9b-256k-theta64m-prelude with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "birgermoell/oellm-9b-256k-theta64m-prelude" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "birgermoell/oellm-9b-256k-theta64m-prelude",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "birgermoell/oellm-9b-256k-theta64m-prelude" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "birgermoell/oellm-9b-256k-theta64m-prelude",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use birgermoell/oellm-9b-256k-theta64m-prelude with Docker Model Runner:
```
docker model run hf.co/birgermoell/oellm-9b-256k-theta64m-prelude
```

OELLM 9B — 256K context (ABF, θ=64M) — prelude (1T-token base)

256K long-context extension of the OpenEuroLLM prelude base (Qwen3 dense 9B, qwen3_9b_hf_baby at iter_0124800 ≈ 1T tokens). Continues the θ-law curriculum from our 128K model (oellm-9b-128k-theta32m-prelude) to the next octave. Base model — not instruction-tuned. Multilingual (37 European languages).

Headline: the θ-law extends to 256K

Far-position (depth-0) retrieval is a RoPE high-dimension OOD problem; the critical rotary base θ doubles per context-length octave. This model confirms the law at 256K:

context	θ	depth-0
128K	32M	✓
256K	64M	100% ✓

Method (staged native ABF)

prelude(4K) → 16K(θ=500k) → 32K(θ=1M) → 64K(θ=2M) → 128K(θ=32M) → 256K(θ=64M). Continued-pretraining in Megatron-LM on LUMI (16× MI250X), CP=16, θ=64M, selective recompute, 1B tokens for the 256K stage. Data: 60% genuine-256K long documents (arXiv/books/code/RFC/docsite concatenated to 256K) + 40% multilingual long-context mix (Jouni Luoma's 152-source blend) — the multilingual portion prevents non-English regression during the 256K stage.

Evaluation (base-LM forced-choice NIAH, chance 25%)

scripts/eval_base_lm_niah.py. The 256K forward needs multi-GPU (device_map="auto" over a full 8-GCD node; one GCD OOMs at 256K).

@256K (multilingual sweep in progress; ~92% so far, depth-0 = 100%):

depth	0.0	0.25	0.5	0.75	1.0
accuracy	100%	(settling)	100%	(settling)	100%

@128K retention: 98% (158/160), depth-0 100% — extending to 256K did not degrade 128K (depth 0.5 = 93%, the mild mid-depth softness carried from the 128K model; see that model's card).

Architecture

Qwen3 dense: 36 layers, hidden 4096, FFN 12288, 32 heads / 8 KV (GQA), kv-channels 128, qk-layernorm, RMSNorm, SwiGLU, untied embeddings, vocab 262144. Config: rope_theta=64000000, max_position_embeddings=262144.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
# 256K context needs several GPUs; use device_map="auto".
m = AutoModelForCausalLM.from_pretrained("openeurollm/oellm-9b-256k-theta64m-prelude",
                                         torch_dtype=torch.bfloat16, device_map="auto")
tok = AutoTokenizer.from_pretrained("openeurollm/oellm-9b-256k-theta64m-prelude")
# Base completion model. Keep rope_theta=64M / max_position=262144 for 256K.

Caveats

Base model (no instruction tuning). Single-needle NIAH; broader RULER not yet run.
Keep rope_theta=64000000, max_position_embeddings=262144 for 256K.
Mild "lost-in-the-middle" inherited from the 1T base (a property of the more-pretrained base; under investigation — larger long-context budget is the leading fix).
Repo/code: https://github.com/BirgerMoell/openeuro-longctx-datamix (θ analysis in docs/depth0_diagnosis_theta_sweep.md). Sibling: oellm-9b-128k-theta32m-prelude.

Downloads last month: 347