Text Generation
MLX
Safetensors
English
Chinese
deepseek_v4
deepseek-v4
deepseek
quantized
apple-silicon
mixture-of-experts
Mixture of Experts
mxtq
turboquant
jang
jangtq
conversational
3.4-bit
Instructions to use osmapi/DeepSeek-V4-Flash-TQ-Q3.4-MLX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use osmapi/DeepSeek-V4-Flash-TQ-Q3.4-MLX with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("osmapi/DeepSeek-V4-Flash-TQ-Q3.4-MLX") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- MLX LM
How to use osmapi/DeepSeek-V4-Flash-TQ-Q3.4-MLX with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "osmapi/DeepSeek-V4-Flash-TQ-Q3.4-MLX"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "osmapi/DeepSeek-V4-Flash-TQ-Q3.4-MLX" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "osmapi/DeepSeek-V4-Flash-TQ-Q3.4-MLX", "messages": [ {"role": "user", "content": "Hello"} ] }'
Add files using upload-large-folder tool
Browse files- model-00013-of-00118.safetensors +3 -0
- model-00019-of-00118.safetensors +3 -0
- model-00021-of-00118.safetensors +3 -0
- model-00024-of-00118.safetensors +3 -0
- model-00052-of-00118.safetensors +3 -0
- model-00057-of-00118.safetensors +3 -0
- model-00058-of-00118.safetensors +3 -0
- model-00060-of-00118.safetensors +3 -0
- model-00065-of-00118.safetensors +3 -0
- model-00093-of-00118.safetensors +3 -0
- model-00096-of-00118.safetensors +3 -0
- model-00099-of-00118.safetensors +3 -0
- model-00100-of-00118.safetensors +3 -0
- model-00105-of-00118.safetensors +3 -0
- model-00118-of-00118.safetensors +3 -0
model-00013-of-00118.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:32f5622d96c54bd53a34a51f2bfab156efe41bc44f8bcb1c4a159aece164e267
|
| 3 |
+
size 1074790753
|
model-00019-of-00118.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a5e3b575b3b85378c20b0f7303b6d05902543590a1d47876950cea1bdf5dad7a
|
| 3 |
+
size 1074790753
|
model-00021-of-00118.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4f2132b7885d1098275ebe9efc165757945b89f9b4f987002aa92b1fa78d69eb
|
| 3 |
+
size 1074790753
|
model-00024-of-00118.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cf2bf5cb0ec2bed192288ec68b249ff37d0f289e4263ff48e2557c16988740fb
|
| 3 |
+
size 1076888266
|
model-00052-of-00118.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:acf44b2913147a9961cf87970bcc25177ecd8cfdefe23887b6bca40ffc8da89b
|
| 3 |
+
size 1075839337
|
model-00057-of-00118.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6a76a6aa37c08b81ce3e73d29ae20159dad0e959bf3c264afb78f9a1e0786ebb
|
| 3 |
+
size 1074790753
|
model-00058-of-00118.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:bdac38f1565c2c671170ac12fbfe881969339b9fbe44fa1db673fee6858860d7
|
| 3 |
+
size 1076888258
|
model-00060-of-00118.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:134480a0b4e634692dd85aea7b0fa7a90d463748cfe14185b7bd311bb99e5e31
|
| 3 |
+
size 1075839337
|
model-00065-of-00118.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9f5e196f24a18ec516f6736261aa264a5546cb4090edbf3436e3f2f0df1a2758
|
| 3 |
+
size 1074790753
|
model-00093-of-00118.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cfc1d5ac5a249dd2147dbf7316c4f0587e5f8336861129ec1c4eba6f31ef50fd
|
| 3 |
+
size 1074790761
|
model-00096-of-00118.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6a906f50fc3b9295f9c319b6e459267fc336b4c4fe5a3bc64da8579801894ad8
|
| 3 |
+
size 1074790761
|
model-00099-of-00118.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f1e2d12f3527673a65d66d0a0a745101366a2a0c15f9d83ee93e53320ad2e571
|
| 3 |
+
size 1074790761
|
model-00100-of-00118.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:083912fcea791175efaf3ad3f79c3079d6881488e117feaf549605d40750d05b
|
| 3 |
+
size 1074790753
|
model-00105-of-00118.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0ddc73d5bab611a166a3f2b6db212c034f6b892c6045f946a44779ea876ae31c
|
| 3 |
+
size 1076888258
|
model-00118-of-00118.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ad17a57e9a115b4d1be55ea8cf39329dd9311a60d803e30f0a74ca0c3bebb445
|
| 3 |
+
size 33573936
|