---
license: gemma
language:
- en
- km
tags:
- customs
- hs-code
- classification
- cambodia
- gemma
- unsloth
- qlora
base_model:
- google/gemma-4-12B-it
pipeline_tag: text-classification
---

# Gemma‑4 HS Code Classifier (Cambodia Customs)

A **Gemma‑4‑E4B‑it** model fine‑tuned with QLoRA to classify product descriptions into **8‑digit HS codes** and return corresponding Cambodian trade rates (Customs Duty, Special Tax, VAT, Excise Tax).

Built with **[Unsloth](https://github.com/unslothai/unsloth)** for fast, memory‑efficient fine‑tuning on a single T4 GPU.

---

## 🎯 What it does

Given a plain‑English product description, the model generates:

```text
HS Code: 6105.10.00
Unit: PIECE
Customs Duty: 25%
Special Tax: 0%
VAT: 10%
Excise Tax: 0%
```

**⚠️ Important**: The rates in the text are generated by the model and **may be wrong**.  
For production, always use the included **lookup table** (`hs_code_lookup.json`) – see [Production use](#-production-use) below.

---

## 🚀 Quick start (in Colab or locally)

```python
from unsloth import FastModel

model, tokenizer = FastModel.from_pretrained(
    "Sothay/gemma4-hscode-classifier",
    load_in_4bit=True,
)

# ---------- Inference with the authoritative lookup table (recommended) ----------
import json, re

with open("hs_code_lookup.json") as f:
    rate_lookup = json.load(f)

def predict_hs_code(description: str) -> dict:
    messages = [
        {"role": "system", "content": [{"type": "text", "text": "You are a customs compliance AI..."}]},
        {"role": "user",   "content": [{"type": "text", "text": f"Description: {description}"}]},
    ]
    inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to("cuda")
    out = model.generate(inputs, max_new_tokens=80, do_sample=False)
    text = tokenizer.decode(out[0][inputs.shape[1]:], skip_special_tokens=True)

    m = re.search(r"HS Code:\s*([0-9]{4,10})", text)
    code = m.group(1) if m else None
    if code and code in rate_lookup:
        return {"hs_code": code, "source": "lookup_table", **rate_lookup[code]}
    return {"hs_code": code, "source": "model_only_UNVERIFIED", "raw_output": text}

print(predict_hs_code("Men's cotton knitted T-shirt"))
```

---

## 🧠 Training details

- **Base model**: `unsloth/gemma-4-E4B-it` (4‑bit QLoRA)
- **Adapter rank**: r=16, alpha=16, targeting all language & attention layers
- **Gradient checkpointing**: Unsloth’s own implementation (avoids Gemma‑4 KV‑shared layer bug)
- **Dataset**: Custom Cambodian HS‑code dataset (`hs_code.csv`) with descriptions, codes, and official rates
  - Cleaned, deduplicated, split into 90/10 train/validation
  - Chat roles fixed to system/user/assistant (Gemma‑4 standard)
- **Training config**: 2 epochs, effective batch size 8, learning rate 2e‑4, linear schedule, eval & save every epoch, best model loaded
- **Hardware**: Google Colab T4 (16 GB) – peak memory ~10 GB thanks to QLoRA
- **Accuracy**: Evaluated on held‑out examples (exact HS‑code match) – see model card for current numbers

---

## ⚖️ Production use

> **Always use the lookup table – never trust the model’s generated rates.**

The model is a **classifier**: description → HS code.  
Rates are fetched deterministically from `hs_code_lookup.json`, a file extracted from the same official tariff data used during training.

Why?  
- A causal LM recalling a rate from memory will occasionally hallucinate – a customs tool with confident, wrong numbers is worse than one that says “I don’t know”.
- The lookup table guarantees 100% accuracy on rates once the HS code is correct.

The `hs_code_lookup.json` file is included in this repository and can be downloaded via:

```python
from huggingface_hub import hf_hub_download
hf_hub_download("Sothay/gemma4-hscode-classifier", "hs_code_lookup.json")
```

---

## 📦 Files in this repository

| File | Description |
|------|-------------|
| `adapter_model.safetensors` | LoRA adapter weights (few MB) |
| `tokenizer.json`, `tokenizer_config.json` | Tokenizer files |
| `hs_code_lookup.json` | Authoritative rate table for production inference |
| `README.md` | This file |

> If you need a **merged, full‑precision model** (for vLLM, TGI, etc.), generate it locally with Unsloth:
> ```python
> model.save_pretrained_merged("merged_fp16", tokenizer, save_method="merged_16bit")
> ```

---

## 🦙 Ollama / llama.cpp (GGUF)

Export a quantized GGUF directly from the LoRA adapter:

```python
model.save_pretrained_gguf("gguf_model", tokenizer, quantization_method="q4_k_m")
```

Then use with Ollama (see [`Modelfile` example](https://ollama.com) – set temperature 0, deterministic sampling).

---

## 📊 Example predictions

| Description | Predicted HS Code | Unit | CD | ST | VAT | ET |
|-------------|-------------------|------|----|----|-----|----|
| Toyota Hilux pickup, diesel 2.8L | 8704 | u | 35% | 50% | 10% | 0% |
| iPhone 15 Pro Max 256GB | 8517 | u | 0% | 0% | 10% | 0% |
| Heineken beer 330ml can | 2203 | l | 35% | 30% | 10% | 0% |

*(Rates from lookup table – not generated by the model.)*

---

## 📝 License

This model is a derivative of **Gemma‑4‑E4B‑it** and is subject to the [Gemma license](https://ai.google.dev/gemma/terms).  
The HS‑code dataset and lookup table are the property of their respective owners.

---

## 🙏 Acknowledgments

- [Unsloth](https://github.com/unslothai/unsloth) – made QLoRA + Gemma‑4 on a T4 effortless
- [Google DeepMind](https://deepmind.google) – for the Gemma family of models

---

**Author**: [Sothay](https://huggingface.co/Sothay)  
**Model card version**: 1.0