Instructions to use atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2

SGLang

How to use atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2 with Docker Model Runner:
```
docker model run hf.co/atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2
```

Qwen3 — 25% Compressed from Qwen3-14B (English · Chat)

Part of the Efficient and Robust AI System (E-AI) Project by Vincent-Daniel Yun. A compressed edition of Qwen/Qwen3-14B with 10 of 40 transformer layers removed (30 layers remain, ≈11.47B parameters), then instruction-tuned so it runs at lower memory and latency.

📅 Version: V2

What's new in V2

Full MMLU-Pro reporting — overall score and a per-subject breakdown vs the dense 14B (below).
Instruction-tuning refresh that improves reasoning-heavy benchmarks over V1/V1.5.
English only. For open-domain factual questions, pair with retrieval (RAG); best with short answers.

⚠️ Language support — English only. Tuned on English data. Other languages (e.g., Korean, Chinese, Japanese) are not officially supported and may degrade.

Method

The pruning method and the recovery method used to build this model are proprietary, undisclosed methods created by Vincent-Daniel Yun and are not released. The compressed model is then instruction-tuned (distilled from the base model). Only the resulting model is shared.

Results (measured)

PPL on 2048-token context (lower is better); downstream tasks and MMLU are 0-shot accuracy via lm-eval-harness (higher is better).

Metric	Qwen3-14B (dense)	This model (25%)
PPL · WikiText2 ↓	8.64	19.32
PPL · C4 ↓	13.0	22.47
PPL · PTB ↓	14.79	30.31
MMLU ↑	0.7729	0.6860

Performance by subject (MMLU-Pro)

MMLU-Pro is a harder, reasoning-focused MMLU — 12,032 questions across 14 subjects. We ran the full test set (no subsampling) vs the dense Qwen3-14B, via lm-eval-harness: 5-shot, greedy decoding (temperature 0), up to 256 generated tokens (max_gen_toks=256, the harness default), scored by exact match.

Subject	Dense 14B	This model (25%)	Retained
Psychology	0.732	0.597	81%
Engineering	0.361	0.286	79%
Economics	0.722	0.549	76%
Biology	0.806	0.610	76%
Philosophy	0.549	0.403	73%
Health	0.654	0.445	68%
Law	0.349	0.237	68%
Other	0.609	0.407	67%
Computer Science	0.627	0.417	67%
History	0.583	0.383	66%
Physics	0.495	0.310	63%
Math	0.603	0.346	57%
Business	0.598	0.295	49%
Chemistry	0.450	0.204	45%
Overall (official)	0.565	0.375	66%

Accuracy is best retained on knowledge- and reading-heavy subjects (psychology, economics, biology, health) and lowest on multi-step quantitative subjects (chemistry, math, physics).

Model family — pick your size

Model	Layers	Params	MMLU ↑
Qwen3-14B (base, uncompressed)	40	14.77B	0.773
20%	32	12.13B	0.716
➡ 25% (this model)	30	11.47B	0.686
30%	28	10.80B	0.651
35%	26	~10.1B	0.572

Quantization

4-bit / 8-bit quantization works — this is a standard Qwen3 architecture, so bitsandbytes loading and other PTQ methods apply on top of the compression for the largest memory savings.

from transformers import AutoModelForCausalLM, BitsAndBytesConfig
m = AutoModelForCausalLM.from_pretrained(
    "atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2", trust_remote_code=True, device_map="cuda",
    quantization_config=BitsAndBytesConfig(load_in_4bit=True))

Usage — Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

m = AutoModelForCausalLM.from_pretrained(
    "atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2", trust_remote_code=True, dtype=torch.float16, device_map="cuda")
tok = AutoTokenizer.from_pretrained("atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2", trust_remote_code=True)

ids = tok("The capital of France is", return_tensors="pt").to("cuda")
print(tok.decode(m.generate(**ids, max_new_tokens=20)[0]))

trust_remote_code=True is required: the model ships a small custom decoder layer in modeling_qwen3_recovered.py.

Usage — vLLM

vLLM uses its own model implementations, so the custom decoder layer is loaded via a tiny plugin (provided in this repo under vllm_plugin/). Install it once, then serve normally:

pip install ./vllm_plugin   # from a checkout of this repo's vllm_plugin/ folder

from vllm import LLM, SamplingParams
llm = LLM(model="atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2", trust_remote_code=True, dtype="float16")
print(llm.generate(["The capital of France is"], SamplingParams(max_tokens=20))[0].outputs[0].text)

Other backends: TGI / SGLang / llama.cpp each use their own model graphs and would need an analogous custom decoder layer; they are not supported out of the box.

License

Apache-2.0, inherited from the base model Qwen/Qwen3-14B.

Acknowledgements

Thanks to Prof. Sai Praneeth Karimireddy (USC) and Prof. Sunwoo Lee (Inha University) for their guidance, and to Alibaba (the Qwen team) for the Qwen3-14B base model.