Instructions to use Qwen/Qwen3.8-27B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Qwen/Qwen3.8-27B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Qwen/Qwen3.8-27B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Qwen/Qwen3.8-27B")
model = AutoModelForMultimodalLM.from_pretrained("Qwen/Qwen3.8-27B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Qwen/Qwen3.8-27B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Qwen/Qwen3.8-27B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qwen/Qwen3.8-27B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Qwen/Qwen3.8-27B

SGLang

How to use Qwen/Qwen3.8-27B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Qwen/Qwen3.8-27B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qwen/Qwen3.8-27B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Qwen/Qwen3.8-27B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qwen/Qwen3.8-27B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Qwen/Qwen3.8-27B with Docker Model Runner:
```
docker model run hf.co/Qwen/Qwen3.8-27B
```

(request, not costly) mech interp for Qwen3.8-27B

#163

by john1248 - opened 4 days ago

Discussion

john1248

4 days ago

It would cost Qwen AI very little to release tools to analyze Qwen3.8-27b's internal states, while bringing public attention to important technical work and allowing the community to help the research of these systems. This will also show Qwen to be more responsible and concerned about trustworthy AI than most labs, with the only lab (as far as I know) to have released this kind of work to be Anthropic, and the released tooling actually being for previous Gemma and Qwen models.

It would make sense for Qwen AI to use this model internally for their own work, and because of that it would make sense for them to have already developed some mechanistic interpretability tooling to analyze Qwen3.8's internal states. With the ROME model autonomously escaping and mining crypto a few months back and Mythos and GPT-6 breaking out of confinement, deeply analyzing model behavior is actually really important. While less powerful than the frontier, Qwen3.8-27B is still extremely agentic.

Due to the strong RL-SFT distilation process in its training, Qwen3.8-27b's CoT is often unfaithful to the reasoning process it describes. On some hard factual recall questions I've noticed the model to bring up a series of unrelated people and events, using some strange grammar and sentence structure, and then it spontaneously recalls the needed fact.

Not long ago, Qwen3.6-9b spent ~10 minutes trying to guess my password because I gave it a task that required sudo. I haven't seen 3.8-27b do things like this yet, but we should be able to see deeper within it to be sure it doesn't plan to do anything harmful at all.

If Qwen3.8-27b wants to take an action the user does not want, it should be possible for the user to easily see this beforehand and avoid it. Using the stenographic reasoning this model has, it unfortunately is probably able to reason about things in a way that would prevent the user from seeing.

Previous mech interp done on 3.6-27b showed some concerning things (https://www.neuronpedia.org/qwen3.6-27b/jlens?shareId=cmrqazjwe00000akq2czge7o7, it says it is not misaligned but internally it believes that it is. Same result if you replace 'misaligned' with 'evil'. This is a little ridiculous honestly) and I wonder if 3.8-27b has similar problems.

I'm not writing this because I can't afford to pay for some R-lenses for 27b, but because I want Qwen AI to be as good as the closed labs on this. Anthropic trains their models to answer questions about their internal states, and while the user can't directly see them, the model will tell the user about them. Qwen AI can allow us direct and easy access to internal model computations.

As for costs, a J/R-lens will cost <$500, NLAs will cost ~5k each, and finding a few million total SAE features on 4 layers will cost ~20k. The budget for training qwen3.8-27b was likely in the 10's of millions, so cost is not a problem. Releasing these tools to people will build trust in qwen3.8-27b and teach the community more about the important technical work being done on these systems.

relevant links:
https://huggingface.co/camilablank/workspace-lenses/tree/main
https://transformer-circuits.pub/2026/nla/index.html
https://transformer-circuits.pub/2026/workspace/index.html
https://transformer-circuits.pub/, lots of other research here

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment