Instructions to use pauldtyler/naic-guide-llama3.2-3b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use pauldtyler/naic-guide-llama3.2-3b with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="pauldtyler/naic-guide-llama3.2-3b",
	filename="naic-guide-f16.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use pauldtyler/naic-guide-llama3.2-3b with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf pauldtyler/naic-guide-llama3.2-3b:F16
# Run inference directly in the terminal:
llama cli -hf pauldtyler/naic-guide-llama3.2-3b:F16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf pauldtyler/naic-guide-llama3.2-3b:F16
# Run inference directly in the terminal:
llama cli -hf pauldtyler/naic-guide-llama3.2-3b:F16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf pauldtyler/naic-guide-llama3.2-3b:F16
# Run inference directly in the terminal:
./llama-cli -hf pauldtyler/naic-guide-llama3.2-3b:F16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf pauldtyler/naic-guide-llama3.2-3b:F16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf pauldtyler/naic-guide-llama3.2-3b:F16

Use Docker

docker model run hf.co/pauldtyler/naic-guide-llama3.2-3b:F16

LM Studio
Jan
Ollama
How to use pauldtyler/naic-guide-llama3.2-3b with Ollama:
```
ollama run hf.co/pauldtyler/naic-guide-llama3.2-3b:F16
```

Unsloth Studio

How to use pauldtyler/naic-guide-llama3.2-3b with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for pauldtyler/naic-guide-llama3.2-3b to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for pauldtyler/naic-guide-llama3.2-3b to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for pauldtyler/naic-guide-llama3.2-3b to start chatting

How to use pauldtyler/naic-guide-llama3.2-3b with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf pauldtyler/naic-guide-llama3.2-3b:F16

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "pauldtyler/naic-guide-llama3.2-3b:F16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use pauldtyler/naic-guide-llama3.2-3b with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf pauldtyler/naic-guide-llama3.2-3b:F16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default pauldtyler/naic-guide-llama3.2-3b:F16

Run Hermes

hermes

Atomic Chat new
Docker Model Runner
How to use pauldtyler/naic-guide-llama3.2-3b with Docker Model Runner:
```
docker model run hf.co/pauldtyler/naic-guide-llama3.2-3b:F16
```

Lemonade

How to use pauldtyler/naic-guide-llama3.2-3b with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull pauldtyler/naic-guide-llama3.2-3b:F16

Run and chat with the model

lemonade run user.naic-guide-llama3.2-3b-F16

List all available models

lemonade list

NAIC Guide - Llama 3.2 3B Fine-tuned (Experimental)

Note: This is an experimental model created for learning and research purposes. It should not be used for actual insurance advice.

A fine-tuned model that answers questions about life insurance and annuities using official NAIC (National Association of Insurance Commissioners) Buyer's Guides.

Overview

This model was created as an experiment in fine-tuning LLMs on domain-specific regulatory content. It demonstrates how a small language model can be trained to:

Answer questions from specific source documents
Cite sources (guide name, section)
Refuse off-topic questions

Training

Base Model: Llama 3.2 3B Instruct
Method: LoRA fine-tuning with MLX on Apple Silicon
Training Data: ~1,700 synthetic Q&A pairs generated from 5 official NAIC Buyer's Guides covering:
- Annuities (general and fixed)
- Life insurance
- Cancer insurance
- Long-term care insurance

Usage with Ollama

Download naic-guide-f16.gguf

Create a Modelfile:

FROM ./naic-guide-f16.gguf
PARAMETER temperature 0.3
SYSTEM "You are a precise advisor that answers questions about life insurance and annuities using ONLY the official NAIC Buyer's Guides. Always cite the guide name and section. If something isn't covered in the guides, say so. Never give personalized financial, legal, or tax advice."

Register with Ollama:
```
ollama create naic-guide -f Modelfile
```
Run:
```
ollama run naic-guide
```

Example

Q: What is a surrender charge?

A: A surrender charge is a fee you'll have to pay if you cancel your policy before being eligible for the full return of premiums (GUIDE: Anb Annuities Fixed, SECTION: Fees, Charges, and Adjustments).

Limitations

This is an experimental model for educational purposes only
Not a substitute for professional insurance advice
Knowledge is limited to the 5 NAIC guides used in training
May occasionally hallucinate or provide incomplete information

License

This model inherits the Llama 3.2 license. The training content is derived from publicly available NAIC consumer guides.

Downloads last month: 3

GGUF

Model size

3B params

Architecture

llama

Hardware compatibility

16-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support