Instructions to use ArliAI/GLM-4.6-REAP-268B-A32B-Derestricted with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ArliAI/GLM-4.6-REAP-268B-A32B-Derestricted with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ArliAI/GLM-4.6-REAP-268B-A32B-Derestricted")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ArliAI/GLM-4.6-REAP-268B-A32B-Derestricted")
model = AutoModelForCausalLM.from_pretrained("ArliAI/GLM-4.6-REAP-268B-A32B-Derestricted")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use ArliAI/GLM-4.6-REAP-268B-A32B-Derestricted with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ArliAI/GLM-4.6-REAP-268B-A32B-Derestricted"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ArliAI/GLM-4.6-REAP-268B-A32B-Derestricted",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ArliAI/GLM-4.6-REAP-268B-A32B-Derestricted

SGLang

How to use ArliAI/GLM-4.6-REAP-268B-A32B-Derestricted with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ArliAI/GLM-4.6-REAP-268B-A32B-Derestricted" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ArliAI/GLM-4.6-REAP-268B-A32B-Derestricted",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ArliAI/GLM-4.6-REAP-268B-A32B-Derestricted" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ArliAI/GLM-4.6-REAP-268B-A32B-Derestricted",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ArliAI/GLM-4.6-REAP-268B-A32B-Derestricted with Docker Model Runner:
```
docker model run hf.co/ArliAI/GLM-4.6-REAP-268B-A32B-Derestricted
```

it is not for everything ;)

by AImhotep - opened Nov 30, 2025

Discussion

AImhotep

Nov 30, 2025

•

edited Nov 30, 2025

Usage on coding - some small task to fix unit tests. Nothing fancy:

"I cannot and will not continue this conversation. I can see the issue now. The test is failing because the URL pattern matching is not working as expected. The test is expecting "Prefix match" but getting the default content instead. Let me examine the URL pattern matching logic in the mock service."

So will you or will you not? 😀

Also this model falls into loops more often than 268B from unsloth (but maybe this is due to the fact that I've quantized it to Q4_K_M myself).

Edit: Roo Code as a tool: The model changed the code to a state where the tests it was fixing no longer made any sense. However, they ‘passed’, so in its opinion, everything was fine.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment