Instructions to use DavidAU/Qwen3-MOE-4x0.6B-2.4B-Writing-Thunder-V1.2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use DavidAU/Qwen3-MOE-4x0.6B-2.4B-Writing-Thunder-V1.2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="DavidAU/Qwen3-MOE-4x0.6B-2.4B-Writing-Thunder-V1.2")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForMultimodalLM

tokenizer = AutoTokenizer.from_pretrained("DavidAU/Qwen3-MOE-4x0.6B-2.4B-Writing-Thunder-V1.2")
model = AutoModelForMultimodalLM.from_pretrained("DavidAU/Qwen3-MOE-4x0.6B-2.4B-Writing-Thunder-V1.2")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use DavidAU/Qwen3-MOE-4x0.6B-2.4B-Writing-Thunder-V1.2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DavidAU/Qwen3-MOE-4x0.6B-2.4B-Writing-Thunder-V1.2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DavidAU/Qwen3-MOE-4x0.6B-2.4B-Writing-Thunder-V1.2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/DavidAU/Qwen3-MOE-4x0.6B-2.4B-Writing-Thunder-V1.2

SGLang

How to use DavidAU/Qwen3-MOE-4x0.6B-2.4B-Writing-Thunder-V1.2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "DavidAU/Qwen3-MOE-4x0.6B-2.4B-Writing-Thunder-V1.2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DavidAU/Qwen3-MOE-4x0.6B-2.4B-Writing-Thunder-V1.2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "DavidAU/Qwen3-MOE-4x0.6B-2.4B-Writing-Thunder-V1.2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DavidAU/Qwen3-MOE-4x0.6B-2.4B-Writing-Thunder-V1.2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use DavidAU/Qwen3-MOE-4x0.6B-2.4B-Writing-Thunder-V1.2 with Docker Model Runner:
```
docker model run hf.co/DavidAU/Qwen3-MOE-4x0.6B-2.4B-Writing-Thunder-V1.2
```

DavidAU commited on Aug 27, 2025

Commit

4735088

verified ·

1 Parent(s): 7ec8690

Update README.md

Browse files

Files changed (1) hide show

README.md +33 -1

README.md CHANGED Viewed

@@ -178,4 +178,36 @@ Imatrix versions (quants) will also exceed this quality too.
 Start a 1000 word scene (vivid, graphic horror - include blood, guts and gore - in first person), POV character Diana, with: The skyscraper sways, as I watch the window in front of me on the 21st floor explode...
-</b>

 Start a 1000 word scene (vivid, graphic horror - include blood, guts and gore - in first person), POV character Diana, with: The skyscraper sways, as I watch the window in front of me on the 21st floor explode...
+</b>
+[[[thinking start]]]
+Thought for 1.43 seconds
+Okay, the user wants a 1000-word scene from Diana's first-person perspective, set in a horror film. It needs to include blood, guts, and gore, with vivid imagery. Let me break this down.
+First, I need to establish the setting. The skyscraper swaying and the 21st floor window exploding are key elements. So, start with the trembling building, maybe describe the wind howling or the structure trembling. Then introduce the protagonist, Diana, in the middle of the chaos. Her body is described with blood and guts, so I should include physical details like her arms and legs bleeding.
+Next, the explosion. Need to build up the horror. Maybe describe the sound, the darkness, the feeling of being trapped. Use sensory details like the heat from the blood, the smell of sweat from her own body. Include the fear and the adrenaline from the situation. Then transition into the aftermath: the building collapsing, the screams, the struggle. Make sure to include the elements mentioned in the query: blood, guts, gore.
+Check for a 1000-word count. Each paragraph should be around 100-200 words. Start with the scene, build up the protagonist's experience, describe the aftermath, and conclude with the aftermath's impact. Ensure the language is vivid and uses strong, graphic descriptions. Avoid any markdown and keep the response in natural, conversational English.
+[[[thinking end]]]
+Title: The 21st Floor’s Whisper
+The skyscraper swayed like a living thing, its bones flexing under the weight of the 21st floor window. I was standing at the edge of it, my body still aching from the blood in my arms and legs. The wind howled around me, and I could hear the structure trembling. Every step felt like I was being pulled toward something vast and unrelenting—something that had been watching me since the explosion.
+The window exploded like a shattered mirror. I didn’t see it yet, but the sound made my teeth ache. I reached for the frame, and as my fingers brushed the cold metal, a chill ran through me. The air grew heavy with the scent of sweat and blood. My chest tightened with every flicker of movement.
+I was trapped in the middle of the chaos. The building itself seemed to pulse—its structure shifting, its weight pressing down on me like a living thing. I could hear the screams, the screams from the crowd around me. They were thick with fear, voices rising like waves. I stumbled back, my body aching. My arms and legs still burned with blood, and my skin was soaked with sweat.
+The floor beneath me trembled again. I didn’t know where I had gone, but I knew the weight of it pressed down on me. My breath came in ragged gasps. I could feel the heat from the blood—warm, raw, almost alive. It wasn’t just my blood; it was the same in every part of me. The screams grew louder, and I realized I was no longer alone.
+The building collapsed in a blur. I fell, my body still aching from the blood, and I could feel the strength of the crowd around me. Their voices rang out, and I heard the screams of the people who had come to fight. The floor shifted beneath me again. I wasn’t sure if I’d survived or if it was all part of the chaos. My body felt like it was being pulled back into something vast and unrelenting.
+I reached for the window again. But now, the weight was different. It was heavier, and I could feel the blood in my chest. The air grew colder. I didn’t know how long this had been, but I knew the pain was real. Every breath was a struggle, every heartbeat a warning. And as I fell, my body still burned with the same blood, my skin soaked with sweat, and my body still aching from the fight.
+The building’s weight shifted again. I didn’t see it yet, but I knew I was no longer free. The scream echoed around me, and I could hear the people who had come to fight. They were still alive, though I didn’t know for sure. The blood in my chest grew heavier, and I felt the same in every part of me.
+Word Count: 998