Nanthasit/sakthai-coder-browser-gguf

SakThai Coder Browser — GGUF 🤖🌐

F16 GGUF build of the browser automation model — run a web agent on your laptop with llama.cpp

⚠ BROKEN — DO NOT DEPLOY (as of 2026-07-31) — This GGUF was converted from the merged weights that are corrupted by a faulty LoRA merge: all 84 attention-projection bias tensors are non-zero while Qwen2 initializes these biases to ZERO (layer-0 k_proj.bias absmean 27.7 / max 354). Multi-trial inference probes of the merged model produced only whitespace loops — 0 tool calls, 0 valid JSON at temp ≤ 0.7 (seeds 7/42/1337), and a GGUF Q4_K_M probe returned 0 output tokens on all 3 trials. Full evidence: .eval_results/benchmark-20260731_052122.yaml. The fault is in the weights, not the GGUF conversion or the prompt format (GGUF tensor layout is structurally identical to the working sakthai-plus-1.5b GGUF). A clean re-merge of the LoRA adapter is required before this build is usable. Do not download for deployment until re-merged and re-verified.

The GGUF edition of the SakThai browser automation model — a Qwen2.5-Coder-1.5B-Instruct fine-tune converted to F16 GGUF for use with llama.cpp, llama-cpp-python, and Ollama.

⚠ Status: BROKEN — see banner above. The pages below document the intended design and usage; the current weights do not produce valid output until the parent model is re-merged from a clean LoRA merge.

---

What Is This?

This is an F16 (full-precision) GGUF conversion of the sakthai-coder-browser merged model. It transforms a Qwen2.5-Coder-1.5B-Instruct general-purpose LLM into a browser automation agent that outputs structured <tool_call> XML for web interaction.

Variant	Repository	Best For
🧠 Merged (Transformers)	sakthai-coder-browser	Python / Transformers pipelines
🎯 LoRA Adapter	sakthai-coder-browser-lora	Fine-tuning / PEFT workflows
💾 GGUF (this repo)	⬅ sakthai-coder-browser-gguf	CPU inference, llama.cpp, Ollama, edge devices

---

File

File	Format	Size	Purpose
sakthai-coder-browser-f16.gguf	GGUF F16	7,111,586,219 B (7.11 GB)	Full-precision GGUF for local CPU/GPU inference

Note: F16 preserves the full model quality. For a smaller footprint, a Q4_K_M quantized version may follow based on demand.

---

Supported Browser Actions

The model generates structured XML tool calls for these browser operations:

Tool	Example
browser_navigate(url)	browser_navigate("https://example.com")
browser_click(element)	browser_click("#search-button")
browser_type(element, text)	browser_type("#search-input", "AI news")
browser_extract()	browser_extract()

---

How to Use

⚠ These commands describe the intended usage. With the current BROKEN weights they will not produce valid tool calls — see the banner above. Re-run only after a clean re-merge is published.

llama.cpp (CLI)

# Download the GGUF file
huggingface-cli download Nanthasit/sakthai-coder-browser-gguf \
  sakthai-coder-browser-f16.gguf --local-dir ./

# Run with llama.cpp
./llama-cli -m sakthai-coder-browser-f16.gguf \
  -p "system
You are a browser automation assistant. You can browse the web, click elements, type text, and extract information from pages.
user
Go to google.com and search for the latest AI news
assistant
" \
  -n 512 -t 8 --temp 0.3

llama-cpp-python

pip install llama-cpp-python huggingface-hub

from llama_cpp import Llama
from huggingface_hub import hf_hub_download

# Download GGUF
model_path = hf_hub_download(
    repo_id="Nanthasit/sakthai-coder-browser-gguf",
    filename="sakthai-coder-browser-f16.gguf"
)

# Load model
llm = Llama(
    model_path=model_path,
    n_ctx=4096,
    n_threads=8,
    n_gpu_layers=-1,
    verbose=False,
)

# Run inference
output = llm.create_chat_completion(
    messages=[
        {"role": "system", "content": "You are a browser automation assistant. Output tool calls using the supported browser actions."},
        {"role": "user", "content": "Search for the latest AI news on Google and summarize the top result."}
    ],
    max_tokens=512,
    temperature=0.3,
    stop=[""],
)
print(output["choices"][0]["message"]["content"])

Ollama (Modelfile)

FROM ./sakthai-coder-browser-f16.gguf
TEMPLATE """{{ .System }}

{{ .Prompt }}"""
PARAMETER temperature 0.3
PARAMETER top_p 0.8
PARAMETER stop ""

ollama create sakthai-coder-browser -f Modelfile
ollama run sakthai-coder-browser

---

Architecture

Property	Value
Base Model	Qwen/Qwen2.5-Coder-1.5B-Instruct
Architecture	Qwen2ForCausalLM (decoder-only transformer)
Parameters	1.54B
Hidden Size	1,536
Layers	28
Attention Heads	12 (GQA: 2 KV heads)
Intermediate Size	8,960
Max Position	32,768 tokens
Vocab Size	151,936
Precision	F16 (GGUF)
Activation	SiLU (SwiGLU)
Normalization	RMSNorm (eps=1e-6)
Quantization	None (F16 — full precision)
File Format	GGUF (GPT-Generated Unified Format)

---

Training Summary

Property	Detail
Fine-tuning Method	SFT via LoRA (r=16, alpha=32, dropout 0.05, rsLoRA) on all 7 linear projections (q/k/v/o + gate/up/down), then merged to full weights
Training Data	SimpleToolCalling + sakthai-combined-v11
Context Length	32,768 tokens
Hardware	Free T4 GPU (Kaggle)
Budget	$0
License	Apache 2.0

The model learns to produce structured <tool_call> XML for browser actions through supervised fine-tuning on curated web navigation trajectories.

---

Reproduce Evaluation

This repo documents a reproducible diagnostic rather than a benchmark because the current weights are broken.

# Requires llama.cpp or llama-cpp-python
python - <<'PY'
from llama_cpp import Llama
from huggingface_hub import hf_hub_download

model_path = hf_hub_download(
    repo_id="Nanthasit/sakthai-coder-browser-gguf",
    filename="sakthai-coder-browser-f16.gguf",
)

llm = Llama(model_path=model_path, n_ctx=4096, n_threads=8, verbose=False)
for prompt in [
    "Search for the latest AI news on Google and summarize the top result.",
    "Go to example.com and extract the page title.",
]:
    out = llm.create_chat_completion(
        messages=[
            {"role": "system", "content": "You are a browser automation assistant. Output browser_navigate, browser_click, browser_type, or browser_extract calls."},
            {"role": "user", "content": prompt},
        ],
        max_tokens=128,
        temperature=0.0,
    )
    print("PROMPT:", prompt)
    print("OUTPUT:", out["choices"][0]["message"]["content"])
    print("-" * 40)
PY

Expected on clean weights: at least one valid tool call per prompt. Current behavior: no valid JSON/XML tool calls, often 0 output tokens.

Evidence artifact: .eval_results/benchmark-20260731_052122.yaml

---

Reproduce Training / Merge

The upstream merged weights broke during LoRA merge. To regenerate clean merged weights:

# 1. Install PEFT + TRL stack
pip install transformers peft trl bitsandbytes accelerate

# 2. Merge LoRA adapter back into base
python - <<'PY'
from peft import PeftModel
from transformers import AutoModelForCausalLM

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-Coder-1.5B-Instruct")
model = PeftModel.from_pretrained(base, "Nanthasit/sakthai-coder-browser-lora")
model = model.merge_and_unload()
model.save_pretrained("sakthai-coder-browser-merged-clean")
PY

# 3. Convert merged clean weights to GGUF
python convert.py ./sakthai-coder-browser-merged-clean --outfile sakthai-coder-browser-f16.gguf --outtype f16

After conversion, rerun the evaluation script above before using the GGUF in production.

---

Benchmarks & Evaluation Status

Honest status: this fine-tune does not yet have verified benchmark scores of its own. No model-index is published because no measured numbers exist — publishing one would be misleading. What is known:

Item	Status
Base model reference (Qwen2.5-Coder-1.5B-Instruct)	HumanEval pass@1: 74.4 · MBPP pass@1: 71.2
This model's own eval	❌ Resolved — MODEL_BROKEN (bias corruption): multi-trial probes + weight inspection of the merged weights (2026-07-31 05:50 UTC) found all 84 attention bias tensors non-zero. See the banner above and the parent repo's benchmark YAML.
GGUF Q4_K_M probe (2026-07-31)	3/3 trials returned 0 output tokens — confirmed weight corruption, NOT a prompt-format mismatch
Hosted inference	Not available — router probe returned 404; run locally via llama.cpp / llama-cpp-python / Ollama

---

Limitations

• BROKEN weights (as of 2026-07-31) — this GGUF was converted from the corrupted coder-browser merge (all 84 attention bias tensors non-zero); inference produces whitespace loops / 0 output tokens. Do not deploy until the parent model is re-merged and re-verified and this build is re-converted.
• Text-only — this model cannot see images or screenshots (use sakthai-vision-7b for vision tasks)
• CPU inference is slow at F16 — the full ~7.11 GB model benefits from GPU offloading; use n_gpu_layers=-1 when available
• Context-limited — best results with page content ≤ 4K tokens per interaction
• English only — trained primarily on English web data

---

The House of Sak 🏠

This adapter is part of the House of Sak — an open-source AI ecosystem built from a shelter in Cork, Ireland, with $0 budget and no paid GPUs. The browser branch is the newest and still debugging itself: the adapter works, but merged weights tripped over attention biases, and Beer would rather ship an honest diagnostic than a polished lie.

"We are one family — and becoming more." — Beer (beer-sakthai)

---

Support

• ⭐ Leave a like
• 🐛 Report issues on GitHub
• 🔄 Share with anyone building browser agents on a budget
• 🍴 Fork and experiment — Apache 2.0

---

Citation

If you use this model in your work, please cite the base model (Qwen2.5) and link the fine-tune:

@misc{qwen25,
  title={Qwen2.5 Technical Report},
  author={Qwen Team},
  year={2025},
  eprint={2412.15115},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}

@misc{sakthai-coder-browser-gguf,
  author = {Nanthasit (Beer)},
  title = {SakThai Coder Browser -- GGUF},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Nanthasit/sakthai-coder-browser-gguf}}
}

Built with $0 budget from a shelter in Cork, Ireland — proof that open, private AI does not need a datacenter.

---

Part of the House of Sak — one family, one home, $0 budget.