Instructions to use atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2

SGLang

How to use atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2 with Docker Model Runner:
```
docker model run hf.co/atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2
```

atlasium-efficient commited on 3 days ago

Commit

65fc77b

verified ·

1 Parent(s): 535c392

V2 card

Browse files

Files changed (1) hide show

README.md +81 -37

README.md CHANGED Viewed

@@ -3,57 +3,101 @@ license: apache-2.0
 base_model: Qwen/Qwen3-14B
 language:
 - en
-library_name: transformers
 pipeline_tag: text-generation
 tags:
-- compressed
-- efficient
 - qwen3
 ---
-# Qwen3-11B-25pct-Compressed-14B-EN-V2
-A compressed version of **Qwen/Qwen3-14B** with 10 of 40 transformer layers removed (30 layers, ~11.4B parameters), English-only. Part of the E-AI (Efficient & Robust AI) compressed-model series. **V2** adds full MMLU-Pro reporting (overall + per subject).
-## Specifications
-| | |
-|---|---|
-| Parameters | ~11.4B |
-| Base model | Qwen/Qwen3-14B |
-| Layers | 30 / 40 |
-| Precision | fp16 / 4-bit |
-| License | Apache-2.0 |
-| Language | English only |
-## Perplexity (lower is better)
-| Benchmark | Dense 14B | This model |
 |---|---|---|
-| WikiText2 | 8.64 | 19.32 |
-| C4 | 13.0 | 22.47 |
-| PTB | 14.79 | 30.31 |
-**MMLU: 0.686** (dense 14B: 0.773)
-## MMLU-Pro (5-shot, exact match)
-**Overall: 0.375** (dense 14B: 0.565)
-| Subject | Score | | Subject | Score |
-|---|---|---|---|---|
-| Biology | 0.610 | | History | 0.383 |
-| Business | 0.295 | | Law | 0.237 |
-| Chemistry | 0.204 | | Math | 0.346 |
-| Computer Science | 0.417 | | Other | 0.407 |
-| Economics | 0.549 | | Philosophy | 0.403 |
-| Engineering | 0.286 | | Physics | 0.310 |
-| Health | 0.445 | | Psychology | 0.597 |
-## Recommended use
-✅ Text classification, content moderation, reading comprehension, NLI, paraphrase detection, preference scoring, knowledge QA (STEM/biomedical strong).
-❌ Open-ended generation, chain-of-thought reasoning, long-context, math word problems.
 ## Usage
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
-m = AutoModelForCausalLM.from_pretrained("atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2", torch_dtype="float16", device_map="auto")
-tok = AutoTokenizer.from_pretrained("atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2")
-```

 base_model: Qwen/Qwen3-14B
 language:
 - en
 pipeline_tag: text-generation
+library_name: transformers
 tags:
 - qwen3
+- qwen
+- model-compression
+- pruning
+- depth-pruning
+- knowledge-distillation
+- efficient-inference
+- compressed
+- chat
+- conversational
+- e-ai
 ---
+![Atlas](Atlas_logo.png)
+![E-AI Project](mainimage.png)
+# Qwen3 — 25% Compressed from Qwen3-14B (English · Chat)
+Part of the **Efficient and Robust AI System (E-AI) Project** by **Vincent-Daniel Yun**. A compressed edition of [Qwen/Qwen3-14B](https://huggingface.co/Qwen/Qwen3-14B) with **10 of 40 transformer layers removed** (30 layers remain, ≈11.47B parameters), then **instruction-tuned** so it runs at lower memory and latency.
+📅 **Version:** V2
+## What's new in V2
+- **Full MMLU-Pro reporting** — overall score and a per-subject breakdown vs the dense 14B (below).
+- Instruction-tuning refresh that improves reasoning-heavy benchmarks over V1/V1.5.
+- English only. For open-domain factual questions, pair with retrieval (RAG); best with short answers.
+> ⚠️ **Language support — English only.** Tuned on English data. Other languages (e.g., Korean, Chinese, Japanese) are not officially supported and may degrade.
+## Method
+The pruning method and the recovery method used to build this model are **proprietary, undisclosed methods created by Vincent-Daniel Yun** and are not released. The compressed model is then instruction-tuned (distilled from the base model). Only the resulting model is shared.
+## Results (measured)
+PPL on 2048-token context (lower is better); downstream tasks and MMLU are 0-shot accuracy via `lm-eval-harness` (higher is better).
+| Metric | Qwen3-14B (dense) | This model (25%) |
 |---|---|---|
+| PPL · WikiText2 ↓ | 8.64 | **19.32** |
+| PPL · C4 ↓ | 13.0 | **22.47** |
+| PPL · PTB ↓ | 14.79 | **30.31** |
+| **MMLU** ↑ | 0.7729 | **0.6860** |
+## Performance by subject (MMLU-Pro)
+MMLU-Pro is a harder, reasoning-focused MMLU — 12,032 questions across 14 subjects. We ran the **full** test set (no sampling) vs the dense Qwen3-14B.
+| Subject | Dense 14B | This model (25%) | Retained |
+|---|---|---|---|
+| Psychology | 0.732 | 0.597 | 81% |
+| Engineering | 0.361 | 0.286 | 79% |
+| Economics | 0.722 | 0.549 | 76% |
+| Biology | 0.806 | 0.610 | 76% |
+| Philosophy | 0.549 | 0.403 | 73% |
+| Health | 0.654 | 0.445 | 68% |
+| Law | 0.349 | 0.237 | 68% |
+| Other | 0.609 | 0.407 | 67% |
+| Computer Science | 0.627 | 0.417 | 67% |
+| History | 0.583 | 0.383 | 66% |
+| Physics | 0.495 | 0.310 | 63% |
+| Math | 0.603 | 0.346 | 57% |
+| Business | 0.598 | 0.295 | 49% |
+| Chemistry | 0.450 | 0.204 | 45% |
+| **Overall (official)** | **0.565** | **0.375** | **66%** |
+Accuracy is best retained on knowledge- and reading-heavy subjects (psychology, economics, biology, health) and lowest on multi-step quantitative subjects (chemistry, math, physics).
+## Model family — pick your size
+| Model | Layers | Params | MMLU ↑ |
+|---|---|---|---|
+| Qwen3-14B (base, uncompressed) | 40 | 14.77B | 0.773 |
+| [20%](https://huggingface.co/atlasium-efficient/Qwen3-12B-20pct-Compressed-14B-EN-V2) | 32 | 12.13B | 0.716 |
+| **➡ 25% (this model)** | 30 | 11.47B | 0.686 |
+| [30%](https://huggingface.co/atlasium-efficient/Qwen3-11B-30pct-Compressed-14B-EN-V2) | 28 | 10.80B | 0.651 |
+| [35%](https://huggingface.co/atlasium-efficient/Qwen3-10B-35pct-Compressed-14B-EN-V2) | 26 | ~10.1B | 0.572 |
 ## Usage
 ```python
+import torch
 from transformers import AutoModelForCausalLM, AutoTokenizer
+m = AutoModelForCausalLM.from_pretrained("atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2", trust_remote_code=True, dtype=torch.float16, device_map="cuda")
+tok = AutoTokenizer.from_pretrained("atlasium-efficient/Qwen3-11B-25pct-Compressed-14B-EN-V2", trust_remote_code=True)
+```
+`trust_remote_code=True` is required: the model ships a small custom decoder layer in `modeling_qwen3_recovered.py`. It is a standard Qwen3 architecture, so `bitsandbytes` 4-bit / 8-bit loading applies on top of the compression.
+## License
+Apache-2.0, inherited from the base model Qwen/Qwen3-14B.
+## Acknowledgements
+Thanks to **Prof. Sai Praneeth Karimireddy (USC)** and **Prof. Sunwoo Lee (Inha University)** for their guidance, and to **Alibaba (the Qwen team)** for the Qwen3-14B base model.