# Qwen3 4B Q4

Qwen3 is the latest generation of Qwen series. It is a large language model with 1.7B parameters, optimized for mobile devices. It is capable of running functions on mobile devices and has been trained to follow instructions and generate long texts (32K tokens). It is more resilient to the diversity of system prompts, enhancing role-play implementation and condition-setting for chatbots. It has up to 128K tokens and can generate up to 32K tokens. It supports for over 100 languages, including Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, Arabic, and more.  

**Model Intention:** It is 4B of Qwen3 series that is excellent for summary, translation and MCP tool calling  

**Model URL:** [https://huggingface.co/flyingfishinwater/good_and_small_models/resolve/main/Qwen3-4B-IQ4_NL.gguf?download=true](https://huggingface.co/flyingfishinwater/good_and_small_models/resolve/main/Qwen3-4B-IQ4_NL.gguf?download=true)  

**Model Info URL:** [https://huggingface.co/Qwen/Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B)  

**Model License:** [License Info](https://www.apache.org/licenses/LICENSE-2.0.txt)  

**Model Description:** Qwen3 is the latest generation of Qwen series. It is a large language model with 1.7B parameters, optimized for mobile devices. It is capable of running functions on mobile devices and has been trained to follow instructions and generate long texts (32K tokens). It is more resilient to the diversity of system prompts, enhancing role-play implementation and condition-setting for chatbots. It has up to 128K tokens and can generate up to 32K tokens. It supports for over 100 languages, including Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, Arabic, and more.  

**Developer:** [https://huggingface.co/Qwen](https://huggingface.co/Qwen)  

**File Size:** 2230 MB  

**Context Length:** 1024 tokens  

**Prompt Format:** 

```

```

**Template Name:** qwen  

**Add BOS Token:** Yes  

**Add EOS Token:** No  

**Parse Special Tokens:** Yes  


---

# GLM Edge 4B Chat

GLM-4 is the latest generation of pre-trained models in the GLM-4 series launched by Zhipu AI. In the evaluation of data sets in semantics, mathematics, reasoning, code, and knowledge, GLM-4 has shown superior performance beyond Llama-3. In addition to multi-round conversations, GLM-4-Chat also has advanced features such as web browsing, code execution, custom tool calls (Function Call), and long text reasoning (supporting up to 128K context). This generation of models has added multi-language support, supporting 26 languages including Japanese, Korean, and German.  

**Model Intention:** It is the latest generation of pre-trained models in the GLM-4 series launched by Zhipu AI  

**Model URL:** [https://huggingface.co/flyingfishinwater/good_and_small_models/resolve/main/glm-edge-4b-chat.Q4_K_M.gguf?download=true](https://huggingface.co/flyingfishinwater/good_and_small_models/resolve/main/glm-edge-4b-chat.Q4_K_M.gguf?download=true)  

**Model Info URL:** [https://huggingface.co/THUDM](https://huggingface.co/THUDM)  

**Model License:** [License Info](https://huggingface.co/THUDM/glm-edge-4b-chat/raw/main/LICENSE)  

**Model Description:** GLM-4 is the latest generation of pre-trained models in the GLM-4 series launched by Zhipu AI. In the evaluation of data sets in semantics, mathematics, reasoning, code, and knowledge, GLM-4 has shown superior performance beyond Llama-3. In addition to multi-round conversations, GLM-4-Chat also has advanced features such as web browsing, code execution, custom tool calls (Function Call), and long text reasoning (supporting up to 128K context). This generation of models has added multi-language support, supporting 26 languages including Japanese, Korean, and German.  

**Developer:** [https://huggingface.co/THUDM](https://huggingface.co/THUDM)  

**File Size:** 2627 MB  

**Context Length:** 1024 tokens  

**Prompt Format:** 

```
{% for item in messages %}{% if item['role'] == 'system' %}<|system|>
{{ item['content'] }}{% elif item['role'] == 'user' %}<|user|>
{{ item['content'] }}{% elif item['role'] == 'assistant' %}<|assistant|>
{{ item['content'] }}{% endif %}{% endfor %}{% if add_generation_prompt %}<|assistant|>
{% endif %}
```

**Template Name:** glm  

**Add BOS Token:** Yes  

**Add EOS Token:** No  

**Parse Special Tokens:** Yes  


---

# Gemma 3n E2B it

Gemma 3n models are designed for efficient execution on low-resource devices. They are capable of multimodal input, handling text, image, video, and audio input, and generating text outputs, with open weights for pre-trained and instruction-tuned variants. These models were trained with data in over 140 spoken languages.  

**Model Intention:** Gemma 3n models are designed for efficient execution on low-resource devices. They are capable of multimodal input  

**Model URL:** [https://huggingface.co/flyingfishinwater/good_and_small_models/resolve/main/gemma-3n-E2B-it-Q4_0.gguf?download=true](https://huggingface.co/flyingfishinwater/good_and_small_models/resolve/main/gemma-3n-E2B-it-Q4_0.gguf?download=true)  

**Model Info URL:** [https://huggingface.co/google/gemma-3n-E2B-it](https://huggingface.co/google/gemma-3n-E2B-it)  

**Model License:** [License Info](https://choosealicense.com/licenses/apache-2.0/)  

**Model Description:** Gemma 3n models are designed for efficient execution on low-resource devices. They are capable of multimodal input, handling text, image, video, and audio input, and generating text outputs, with open weights for pre-trained and instruction-tuned variants. These models were trained with data in over 140 spoken languages.  

**Developer:** [https://huggingface.co/google](https://huggingface.co/google)  

**Update Date:** 2025-06-27  

**File Size:** 2720 MB  

**Context Length:** 8000 tokens  

**Prompt Format:** 

```

```

**Template Name:** chatml  

**Add BOS Token:** Yes  

**Add EOS Token:** No  

**Parse Special Tokens:** Yes  


---

# SmolLM2 1.7B

SmolLM2 was trained on 11 trillion tokens and demonstrates significant advances over other small models, particularly in instruction following, knowledge, reasoning, and mathematics.  

**Model Intention:** SmolLM2 is capable of solving a wide range of tasks while being lightweight enough to run on-device  

**Model URL:** [https://huggingface.co/flyingfishinwater/good_and_small_models/resolve/main/smollm2-1.7b-instruct-q4_k_m.gguf?download=true](https://huggingface.co/flyingfishinwater/good_and_small_models/resolve/main/smollm2-1.7b-instruct-q4_k_m.gguf?download=true)  

**Model Info URL:** [https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct)  

**Model License:** [License Info](https://choosealicense.com/licenses/apache-2.0/)  

**Model Description:** SmolLM2 was trained on 11 trillion tokens and demonstrates significant advances over other small models, particularly in instruction following, knowledge, reasoning, and mathematics.  

**Developer:** [https://huggingface.co/HuggingFaceTB](https://huggingface.co/HuggingFaceTB)  

**Update Date:** 2024-11-02  

**File Size:** 1060 MB  

**Context Length:** 8192 tokens  

**Prompt Format:** 

```
<|im_start|>system
{{system}}<|im_end|>
<|im_start|>user
{{prompt}}<|im_end|>
<|im_start|>assistant

```

**Template Name:** chatml  

**Add BOS Token:** Yes  

**Add EOS Token:** No  

**Parse Special Tokens:** Yes  


---

# Phi4 mini 4B

Phi-4-mini-instruct is a lightweight open model built upon synthetic data and filtered publicly available websites - with a focus on high-quality, reasoning dense data. The model is intended for broad multilingual commercial and research use. The model provides uses for general purpose AI systems and applications which require: 1). Memory/compute constrained environments; 2). Latency bound scenarios; 3) Strong reasoning (especially math and logic).  The model is designed to accelerate research on language and multimodal models, for use as a building block for generative AI powered features.  

**Model Intention:** Phi-4-mini-instruct is a lightweight model focused on high-quality, reasoning dense data. It supports 128K token context length  

**Model URL:** [https://huggingface.co/flyingfishinwater/good_and_small_models/resolve/main/Phi-4-mini-instruct-Q4_K_M.gguf?download=true](https://huggingface.co/flyingfishinwater/good_and_small_models/resolve/main/Phi-4-mini-instruct-Q4_K_M.gguf?download=true)  

**Model Info URL:** [https://huggingface.co/microsoft/Phi-4-mini-instruct](https://huggingface.co/microsoft/Phi-4-mini-instruct)  

**Model License:** [License Info](https://choosealicense.com/licenses/mit/)  

**Model Description:** Phi-4-mini-instruct is a lightweight open model built upon synthetic data and filtered publicly available websites - with a focus on high-quality, reasoning dense data. The model is intended for broad multilingual commercial and research use. The model provides uses for general purpose AI systems and applications which require: 1). Memory/compute constrained environments; 2). Latency bound scenarios; 3) Strong reasoning (especially math and logic).  The model is designed to accelerate research on language and multimodal models, for use as a building block for generative AI powered features.  

**Developer:** [https://huggingface.co/microsoft](https://huggingface.co/microsoft)  

**File Size:** 2020 MB  

**Context Length:** 2048 tokens  

**Prompt Format:** 

```
{% for message in messages %}{% if message['role'] == 'system' and 'tools' in message and message['tools'] is not none %}{{ '<|' + message['role'] + '|>' + message['content'] + '<|tool|>' + message['tools'] + '<|/tool|>' + '<|end|>' }}{% else %}{{ '<|' + message['role'] + '|>' + message['content'] + '<|end|>' }}{% endif %}{% endfor %}{% if add_generation_prompt %}{{ '<|assistant|>' }}{% else %}{{ eos_token }}{% endif %}
```

**Template Name:** llama3.2  

**Add BOS Token:** Yes  

**Add EOS Token:** No  

**Parse Special Tokens:** Yes  


---

# Qwen3 1.7B

Qwen3 1.7B is one of the small models in the Qwen series, designed for efficiency and speed. It can run seamlessly on edge devices, enabling rapid inference and real-time applications. This compact model is ideal for testing scenarios, prototyping, or deployment in resource-constrained environments.  

**Model Intention:** The 1.7B model in the Qwen3 series is a small model designed for fast predictions and function calls.  

**Model URL:** [https://huggingface.co/flyingfishinwater/good_and_small_models/resolve/main/Qwen3-1.7B-Q4_K_M.gguf?download=true](https://huggingface.co/flyingfishinwater/good_and_small_models/resolve/main/Qwen3-1.7B-Q4_K_M.gguf?download=true)  

**Model Info URL:** [https://huggingface.co/Qwen/Qwen3-1.7B](https://huggingface.co/Qwen/Qwen3-1.7B)  

**Model License:** [License Info](https://www.apache.org/licenses/LICENSE-2.0.txt)  

**Model Description:** Qwen3 1.7B is one of the small models in the Qwen series, designed for efficiency and speed. It can run seamlessly on edge devices, enabling rapid inference and real-time applications. This compact model is ideal for testing scenarios, prototyping, or deployment in resource-constrained environments.  

**Developer:** [https://huggingface.co/Qwen](https://huggingface.co/Qwen)  

**File Size:** 1110 MB  

**Context Length:** 2048 tokens  

**Prompt Format:** 

```

```

**Template Name:** qwen  

**Add BOS Token:** Yes  

**Add EOS Token:** No  

**Parse Special Tokens:** Yes  


---

# ERNIE-4.5 0.3B

ERNIE 4.5 is a series of open source models created by Baidu. The advanced capabilities of the ERNIE 4.5 models, particularly the MoE-based A47B and A3B series, are underpinned by several key technical innovations: 1. Multimodal Heterogeneous MoE Pre-Training; 2. Scaling-Efficient Infrastructure; 3. Modality-Specific Post-Training  

**Model Intention:** ERNIE-4.5-0.3B-Base is a text dense Base model for testing the model's architecture.  

**Model URL:** [https://huggingface.co/flyingfishinwater/good_and_small_models/resolve/main/ERNIE-4.5-0.3B-PT-Q4_0.gguf?download=true](https://huggingface.co/flyingfishinwater/good_and_small_models/resolve/main/ERNIE-4.5-0.3B-PT-Q4_0.gguf?download=true)  

**Model Info URL:** [https://huggingface.co/baidu/ERNIE-4.5-0.3B-Base-PT](https://huggingface.co/baidu/ERNIE-4.5-0.3B-Base-PT)  

**Model License:** [License Info](https://www.apache.org/licenses/LICENSE-2.0.txt)  

**Model Description:** ERNIE 4.5 is a series of open source models created by Baidu. The advanced capabilities of the ERNIE 4.5 models, particularly the MoE-based A47B and A3B series, are underpinned by several key technical innovations: 1. Multimodal Heterogeneous MoE Pre-Training; 2. Scaling-Efficient Infrastructure; 3. Modality-Specific Post-Training  

**Developer:** [https://huggingface.co/baidu](https://huggingface.co/baidu)  

**File Size:** 233 MB  

**Context Length:** 2048 tokens  

**Prompt Format:** 

```

```

**Template Name:** qwen  

**Add BOS Token:** Yes  

**Add EOS Token:** No  

**Parse Special Tokens:** Yes  


---

# SmolLM3 3B

SmolLM3 is a fully open model that offers strong performance at the 3B–4B scale. The model is a decoder-only transformer using GQA and NoPE (with 3:1 ratio), it was pretrained on 11.2T tokens with a staged curriculum of web, code, math and reasoning data. Post-training included midtraining on 140B reasoning tokens.  

**Model Intention:** SmolLM3 is a 3B parameter language model designed to push the boundaries of small models. It supports 6 languages (English, French, Spanish, German, Italian, and Portuguese), advanced reasoning and long context.  

**Model URL:** [https://huggingface.co/flyingfishinwater/good_and_small_models/resolve/main/SmolLM3-Q4_K_M.gguf?download=true](https://huggingface.co/flyingfishinwater/good_and_small_models/resolve/main/SmolLM3-Q4_K_M.gguf?download=true)  

**Model Info URL:** [https://huggingface.co/HuggingFaceTB/SmolLM3-3B](https://huggingface.co/HuggingFaceTB/SmolLM3-3B)  

**Model License:** [License Info](https://www.apache.org/licenses/LICENSE-2.0.txt)  

**Model Description:** SmolLM3 is a fully open model that offers strong performance at the 3B–4B scale. The model is a decoder-only transformer using GQA and NoPE (with 3:1 ratio), it was pretrained on 11.2T tokens with a staged curriculum of web, code, math and reasoning data. Post-training included midtraining on 140B reasoning tokens.  

**Developer:** [https://huggingface.co/HuggingFaceTB](https://huggingface.co/HuggingFaceTB)  

**File Size:** 1920 MB  

**Context Length:** 2048 tokens  

**Prompt Format:** 

```

```

**Template Name:** qwen  

**Add BOS Token:** Yes  

**Add EOS Token:** No  

**Parse Special Tokens:** Yes  


---