---
license: gemma
base_model: google/gemma-3-270m-it
tags:
  - onnx
  - lora
  - text-generation
  - grounding
  - faithfulness
---

# uyscout-gemma3-270m-lora

A LoRA fine-tune of [`google/gemma-3-270m-it`](https://huggingface.co/google/gemma-3-270m-it),
merged into the base weights and exported to ONNX (dynamic INT8 quantization), built for
[UyScout](https://github.com/SRKRZ23) — a Tashkent property-research chatbot that runs entirely
client-side in the browser via WebGPU/onnxruntime-web.

## What this fixes

The stock `gemma-3-270m-it` reliably avoids inventing entirely new facts, but tends to answer
*generically* instead of citing the specific POI names / price figures given in its context —
e.g. "a major shopping and transit hub" instead of naming "Chorsu Bazaar" when that name is right
there in the prompt.

This checkpoint was fine-tuned (LoRA, r=8, `q_proj`/`k_proj`/`v_proj`/`o_proj`, 30 epochs) on 14
hand-written examples that always cite exact proper nouns and numbers from the given fact sheet.

## Measured result (not asserted)

On a genuine **held-out set of 14 new examples never seen during training** (different districts,
different POI categories — gym/university/mosque/bank/pharmacy, never appeared in training):

| Model | Held-out GOOD rate |
|---|---|
| Stock `gemma-3-270m-it` | 11/14 (78.6%) |
| **This checkpoint (LoRA)** | **13/14 (92.9%)** |

Paired with a deterministic post-generation fact-injection safety net (splices in any real POI
name / price figure the model still leaves out — can only ever add grounded content, never
fabricate). Full methodology, the earlier techniques that were tried and ruled out (Context-Aware
Decoding, DAGCD/attention-guided decoding), and every real bug found along the way are documented
at [github.com/SRKRZ23/uyscout](https://github.com/SRKRZ23) (bench/ harness).

## Files

- `model_int8.onnx` — merged + INT8-quantized ONNX graph (dynamic quantization via
  `onnxruntime.quantization.quantize_dynamic`), single self-contained file, ~418MB.
- Standard tokenizer/config files, unchanged from the base checkpoint.

## Usage

Same interface as any ONNX-exported `Gemma3ForCausalLM` — `input_ids`, `attention_mask`, and
`past_key_values.{i}.{key,value}` (float32, `num_key_value_heads=1`, `head_dim=256`,
`num_hidden_layers=18`). See `on-device-llm.js` in the UyScout repo for a working
onnxruntime-web/WebGPU integration.

## License

Inherits the [Gemma Terms of Use](https://ai.google.dev/gemma/terms) from the base model —
this is a fine-tuned derivative, not a from-scratch model.