Gemma-SEA-LION-v4.5-E2B-IT-GGUF

Last update: 2026-06-18

This repository contains the GGUF quantized weights for Gemma-SEA-LION-v4.5-E2B-IT, an ultra-efficient 2.3B effective parameter model (5.1B including embeddings) built upon the state-of-the-art gemma-4-E2B-it architecture.

The model is heavily optimized for Southeast Asian (SEA) languages, cultures, and context. It has undergone rigorous logit distillation from the larger google/gemma-4-31B-it model on an updated, human-curated dataset (aisingapore/SEA-Instruct-2602). This model brings premium-tier agentic capability, reasoning, and tool-use to edge servers, laptops, and mobile devices with minimal computational overhead.

Model Details

Model Description

SEA-LION stands for Southeast Asian Languages In One Network.

We performed post-training in English and SEA languages on gemma-4-E2B-it, a multimodal learning model using the Gemma 4 architecture, to create Gemma-SEA-LION-v4.5-E2B.

For tokenization, the model employs the default tokenizer used in gemma-4-E2B-it.

• Developed by: AI Products Pillar, AI Singapore
• Funded by: Singapore NRF
• Shared by: AI Products Pillar, AI Singapore
• Model type: Transformer Decoder with Vision and Audio Encoder
• Training Stage: Post-Training (Logit Distillation & Model Merging)
• Context length: 128k
• Language(s): fine-tuned on Burmese, Indonesian, Filipino, Malay, Tamil, Thai, and Vietnamese
• License: MIT
• Quantized from model: aisingapore/Gemma-SEA-LION-v4.5-E2B-IT

Available Quantized Versions

We provide multiple quantization formats to optimize deployment trade-offs between memory footprint and output quality.

File / Quantization	File Size
Q4_K_M	3.43 GB
Q6_K	3.85 GB
Q8_0	4.97 GB
F16	9.31 GB

Model Strengths

• Efficient Agentic Base: Operating effectively at a 2.3B parameter footprint, the model achieves state-of-the-art parameter efficiency, maximizing throughput in resource-constrained environments.
• Native System Prompt & Thinking: Built on the Gemma 4 foundation, this model natively supports the system role and complex configurable reasoning/thinking states.
• 128K Context Window: Supports large-context parsing, local document search, and extensive multi-turn conversations.
• Multilingual Fluency: Fine-tuned on native language data covering Burmese, Indonesian, Filipino (Tagalog), Malay, Tamil, Thai, and Vietnamese, alongside excellent English capabilities.

How to Get Started

Using llama.cpp

You can serve the model using llama.cpp (add other flags to customize your setup):

llama-server -ngl -1 \
    --mmproj /directory/path/to/mmproj-Gemma-SEA-LION-v4.5-E2B-IT-F16.gguf \
    -m /directory/path/to/Gemma-SEA-LION-v4.5-E2B-IT-Q4_K_M.gguf

You can proceed to make API calls to the model:

curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Gemma-SEA-LION-v4.5-E2B-IT-Q4_K_M.gguf",
    "messages": [
        {
        "role": "user",
        "content": "Teach me a greeting in Bahasa Indonesia."
        }
      ],
    "chat_template_kwargs": {
      "enable_thinking": false
      }
  }'

You can also access the UI at http://localhost:8080 to chat with the model directly.

Uses

Out-of-Scope Use

The model has not been aligned for safety. Developers and users should perform their own safety fine-tuning and related security measures. In no event shall the authors be held liable for any claims, damages, or other liabilities arising from the use of the released weights and codes.

Bias, Risks, and Limitations

The model was not tested for robustness against adversarial prompting. It is important for users to be aware that our model exhibits certain limitations that warrant consideration. Like many LLMs, the model can hallucinate and occasionally generates irrelevant content, introducing fictional elements that are not grounded in the provided context. Users should also exercise caution in interpreting and validating the model's responses due to the potential inconsistencies.

More Information

This is the repository for the commercial instruction-tuned model. The model has not been aligned for safety. Developers and users should perform their own safety fine-tuning and related security measures. In no event shall the authors be held liable for any claims, damages, or other liabilities arising from the use of the released weights and codes.

For more info, please contact us at sealion@aisingapore.org

Acknowledgement

This project is supported by the National Research Foundation Singapore and Infocomm Media Development Authority (IMDA), Singapore under its National Large Language Model Funding Initiative.

Contact

sealion@aisingapore.org