Instructions to use patlegu/opnsense-agent-mistral with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use patlegu/opnsense-agent-mistral with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("unsloth/mistral-7b-instruct-v0.3-bnb-4bit")
model = PeftModel.from_pretrained(base_model, "patlegu/opnsense-agent-mistral")

llama-cpp-python

How to use patlegu/opnsense-agent-mistral with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="patlegu/opnsense-agent-mistral",
	filename="mistral-7b-instruct-v0.3.Q4_K_M.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use patlegu/opnsense-agent-mistral with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf patlegu/opnsense-agent-mistral:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf patlegu/opnsense-agent-mistral:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf patlegu/opnsense-agent-mistral:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf patlegu/opnsense-agent-mistral:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf patlegu/opnsense-agent-mistral:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf patlegu/opnsense-agent-mistral:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf patlegu/opnsense-agent-mistral:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf patlegu/opnsense-agent-mistral:Q4_K_M

Use Docker

docker model run hf.co/patlegu/opnsense-agent-mistral:Q4_K_M

LM Studio
Jan

vLLM

How to use patlegu/opnsense-agent-mistral with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "patlegu/opnsense-agent-mistral"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "patlegu/opnsense-agent-mistral",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/patlegu/opnsense-agent-mistral:Q4_K_M

Ollama
How to use patlegu/opnsense-agent-mistral with Ollama:
```
ollama run hf.co/patlegu/opnsense-agent-mistral:Q4_K_M
```

Unsloth Studio new

How to use patlegu/opnsense-agent-mistral with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for patlegu/opnsense-agent-mistral to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for patlegu/opnsense-agent-mistral to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for patlegu/opnsense-agent-mistral to start chatting

Pi new

How to use patlegu/opnsense-agent-mistral with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf patlegu/opnsense-agent-mistral:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "patlegu/opnsense-agent-mistral:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use patlegu/opnsense-agent-mistral with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf patlegu/opnsense-agent-mistral:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default patlegu/opnsense-agent-mistral:Q4_K_M

Run Hermes

hermes

Docker Model Runner
How to use patlegu/opnsense-agent-mistral with Docker Model Runner:
```
docker model run hf.co/patlegu/opnsense-agent-mistral:Q4_K_M
```

Lemonade

How to use patlegu/opnsense-agent-mistral with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull patlegu/opnsense-agent-mistral:Q4_K_M

Run and chat with the model

lemonade run user.opnsense-agent-mistral-Q4_K_M

List all available models

lemonade list

🛡️ OPNsense Agent Model (LorA V7)

A specialized language model for managing OPNsense Firewalls via function calling.

This model is a fine-tune of Mistral-7B-Instruct-v0.3 designed to translate natural language user requests into valid OPNsense API calls (JSON). It is intended to be the "brain" of an autonomous security agent (e.g., using CrewAI or LangChain).

DISCLAIMER
Do not use this in production. This is a first version that still needs a lot of work to be truly finalized.

🚀 Key Features

100% Valid JSON Output: Rigorously trained to output only valid JSON tool calls.
Full API Coverage: Supports 36 core OPNsense functions (Firewall rules, Aliases, NAT, System internals).
Deterministic: Version V7 was trained on a deterministic dataset to ensure strict adherence to user-provided arguments (IPs, ports, etc.).
GGUF Ready: Optimized for local inference with Ollama (Quantized Q4_K_M).

📦 Usage with Ollama

Run this model locally with Ollama to interact with your firewall using natural language.

1. Pull the Model

ollama run hf.co/patlegu/opnsense-agent

2. Interactive Mode

Note: The model generates the JSON command but does not execute it directly. You need an agent script to execute the output.

>>> Block traffic from IP 192.168.1.50 on WAN interface

# Output:
{
  "tool_calls": [
    {
      "name": "add_filter_rule",
      "arguments": {
        "action": "block",
        "interface": "wan",
        "protocol": "any",
        "source": "192.168.1.50",
        "description": "Block traffic from 192.168.1.50"
      }
    }
  ]
}

3. Create Custom Modelfile (Recommended)

To prevent hallucinations and strictly enforce the OPNsense schema, use the provided Modelfile (available in this repository).

# 1. Download the Modelfile
wget https://huggingface.co/patlegu/opnsense-agent/raw/main/Modelfile

# 2. Create the custom agent model
ollama create opnsense-agent -f Modelfile

# 3. Run the agent
ollama run opnsense-agent

4. Test with Example Prompts

We provide a comprehensive list of test scenarios in ollama_test_prompts.txt covering:

🛡️ Firewall Rules (Block/Allow IPs, Ports)
🏷️ Aliases (Create, Add to, List)
🌐 NAT (Port Forwarding, Outbound)
🔧 System Admin (Apply changes, Logs, States)

Example Test:

>>> Block traffic from 192.168.1.50 on WAN interface

See ollama_test_prompts.txt for 12+ ready-to-use scenarios.

🐍 Usage with Python (CrewAI / LangChain)

This model shines when integrated into an agent workflow.

from crewai import Agent, Task
from langchain_openai import ChatOpenAI

# 1. Connect to local Ollama
opnsense_llm = ChatOpenAI(
    model="ollama/opnsense-agent",
    base_url="http://localhost:11434/v1",
    api_key="NA"
)

# 2. Define the Agent
firewall_admin = Agent(
    role='Firewall Admin',
    goal='Secure the network',
    backstory='Expert admin managing OPNsense via API.',
    llm=opnsense_llm,
    tools=[add_filter_rule_tool, add_alias_tool],  # Your actual Python functions
    verbose=True
)

# 3. Execute Task
task = Task(
    description="Block the attacker IP 203.0.113.66 immediately",
    agent=firewall_admin
)

📊 Performance (V7)

The model was evaluated on a held-out test set of 50 complex scenarios.

Metric	Score	Note
Function Selection	100%	Always chooses the correct API endpoint
Argument Accuracy	100%	Perfect extraction of IPs, ports, and enums
JSON Validity	100%	No syntax errors

(Evaluation date: Jan 2026)

🛠️ Supported Functions

The model understands the following 36 functions:

Firewall: add_filter_rule, delete_filter_rule, get_filter_rule, toggle_filter_rule, set_filter_rule
Aliases: add_alias, add_to_alias, delete_alias, delete_from_alias, get_alias, list_alias_content, set_alias, find_alias_references, flush_alias
NAT: add_nat_port_forward, add_nat_outbound, add_nat_one_to_one, delete_nat_outbound, delete_nat_one_to_one
States: get_firewall_states, kill_firewall_states, flush_firewall_states, get_state_summary (inferred)
System/Diagnostics: apply_firewall_changes, revert_firewall_changes, create_firewall_savepoint, cancel_firewall_rollback, get_firewall_log, get_firewall_statistics, get_rule_statistics, get_interface_list
Categories/GeoIP: add_category, delete_category, list_categories, get_geoip_database, list_geoip_countries, update_bogons

⚖️ License

Apache 2.0. Based on Mistral-7B-Instruct-v0.3.

Downloads last month: 16

GGUF

Model size

7B params

Architecture

llama

Hardware compatibility

4-bit

Model tree for patlegu/opnsense-agent-mistral

Base model

mistralai/Mistral-7B-v0.3

Finetuned

mistralai/Mistral-7B-Instruct-v0.3

Adapter

(923)

this model