justinthelaw/teapot-profile-qa-browser-1024

Teapot Profile-QA Browser 1024

Description

This model is a browser-oriented ONNX export of a local LoRA continuation from teapotai/teapotllm. It is tuned for public resume/profile Q&A prompts that fit within a 1024-token browser context budget.

Hugging Face model metadata uses the official text-generation task category; the browser runtime still loads this T5-style export with the Transformers.js text2text-generation pipeline.

The target use case is a static portfolio or resume site that runs inference in the browser with Transformers.js, without API routes, hosted inference, server actions, or cloud training. The profile schema is intentionally generic for repo reuse: identity, current_role, experience, projects, education, recommendations, skills, and interests.

Browser Artifacts

The repository payload contains tokenizer/config files at the root and Transformers.js ONNX files under onnx/:

• encoder_model_int8.onnx
• decoder_model_merged_int8.onnx
• encoder_model_uint8.onnx
• decoder_model_merged_uint8.onnx

The export gate rejects external .onnx.data files so the model can be loaded as self-contained browser assets.

How to Use

import { pipeline } from "@huggingface/transformers";

const generator = await pipeline(
  "text2text-generation",
  "justinthelaw/teapot-profile-qa-browser-1024",
  { dtype: "int8" },
);

const result = await generator(prompt, { max_new_tokens: 160 });

Use dtype: "uint8" as a browser fallback if the target environment has issues with signed int8 ONNX weights.

Training

• Base model: teapotai/teapotllm
• Method: local LoRA/QLoRA continuation, no full fine-tune and no cloud training
• Promoted checkpoint: teapot-profile-qa-lora-v5/checkpoint-40
• LoRA: rank 16, alpha 32, dropout 0.03, target modules q and v
• 8GB-safe settings: 4-bit base loading, batch size 1, gradient accumulation 8, gradient checkpointing, short eval batches
• Final continuation window: train loss 0.0330 at step 40
• Best validation eval loss: 0.0287

Software

• Training: PyTorch, Transformers, PEFT, bitsandbytes, Datasets
• Export: Optimum ONNX export, ONNX Runtime dynamic quantization
• Browser runtime: Transformers.js with ONNX Runtime Web/WASM
• Browser packaging: text2text-generation-with-past export with decoder_model_merged and subgraph-enabled ONNX quantization

Hardware

Training was designed for a local 8GB NVIDIA laptop GPU profile, with GPU health checks for nvidia-smi, /dev/nvidia*, CUDA-enabled PyTorch, and torch.cuda.is_available(). Export and card preparation can run on CPU after training completes.

Evaluation

Run	Macro	Refusal Accuracy	Multi-Turn Accuracy
Teapot baseline, test	0.7114	0.4000	0.7917
Promoted checkpoint, validation	0.9792	1.0000	1.0000
Promoted checkpoint, test	0.9753	1.0000	1.0000

Promoted checkpoint test macro by task:

Task	Macro
chronology	0.7500
education	1.0000
multi_hop	0.8214
multi_turn	1.0000
recommendations	1.0000
refusal	1.0000
single_turn	1.0000

Intended Uses

• Browser-only profile or resume Q&A.
• Static portfolio demos where answers must stay grounded in public profile context.
• Forks that replace the included public facts with another person's public resume/profile sections.

Limitations

This is not a general assistant. The dataset is synthetic and profile-specific, so production use should regenerate data from the target person's public facts and rerun local evaluation. The model should refuse private or unsupported facts when the public profile context does not answer.

Responsible AI Considerations

Keep factual context public, review generated examples for private-data leakage, and preserve refusal examples for sensitive or absent facts such as home addresses, phone numbers, salary, and classified information. Do not use this model for background checks, hiring decisions, legal advice, medical advice, or identity verification.

Release Notes

• 2026-06-19: Initial local browser profile-QA export with int8 and uint8 ONNX variants.

License

MIT. The base model card for teapotai/teapotllm also lists MIT.