Instructions to use FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF:NVFP4
# Run inference directly in the terminal:
llama cli -hf FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF:NVFP4

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF:NVFP4
# Run inference directly in the terminal:
llama cli -hf FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF:NVFP4

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF:NVFP4
# Run inference directly in the terminal:
./llama-cli -hf FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF:NVFP4

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF:NVFP4
# Run inference directly in the terminal:
./build/bin/llama-cli -hf FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF:NVFP4

Use Docker

docker model run hf.co/FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF:NVFP4

LM Studio
Jan
Ollama
How to use FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF with Ollama:
```
ollama run hf.co/FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF:NVFP4
```

Unsloth Studio

How to use FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF to start chatting

How to use FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF:NVFP4

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF:NVFP4"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF with Docker Model Runner:
```
docker model run hf.co/FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF:NVFP4
```

Lemonade

How to use FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF:NVFP4

Run and chat with the model

lemonade run user.Ministral-3-14B-Instruct-2512-NVFP4-GGUF-NVFP4

List all available models

lemonade list

Hermes Agent

How to use FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF:NVFP4

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF:NVFP4

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF:NVFP4

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF:NVFP4" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Ministral 3 14B Instruct-2512 — NVFP4 GGUF

NVFP4 quantization of mistralai/Ministral-3-14B-Instruct-2512, a 14B parameter coding and vision model from Mistral AI.

About the Model

Ministral 3 14B is a dense transformer with 40 layers, 5120 hidden dimension, and 24-layer Pixtral ViT vision encoder. It supports:

Code generation and debugging across multiple languages
Vision understanding via multimodal image input
Tool calling with native function calling support
131K context window
393K maximum context length

Quantization

This GGUF was quantized from the FP8_E4M3 source weights using llama.cpp (build 537). The source safetensors were dequantized to F16 during conversion, then quantized to NVFP4 format.

NVFP4 (Microscaling FP4) uses block-wise quantization with shared exponents per block, providing better precision than standard FP4 for the same memory footprint.

Files

File	Size	Description
`ministral-3-14b-instruct-2512-nvfp4.gguf`	~6.9 GB	NVFP4 quantized model weights
`mmproj-ministral-3-14b-instruct-2512-f16.gguf`	~878 MB	Vision projector (F16, unquantized)

Usage

llama.cpp

# Server mode with OpenAI-compatible API
llama-server \
  -m ministral-3-14b-instruct-2512-nvfp4.gguf \
  --mmproj mmproj-ministral-3-14b-instruct-2512-f16.gguf \
  -ngl 99 \
  --host 0.0.0.0 \
  --port 8080

# Direct inference
llama-cli \
  -m ministral-3-14b-instruct-2512-nvfp4.gguf \
  --mmproj mmproj-ministral-3-14b-instruct-2512-f16.gguf \
  -ngl 99 \
  -p "Write a Python function to compute fibonacci numbers"

LM Studio

Download both files from this repository
Load the main GGUF file in LM Studio
Load the mmproj file for vision support
Set GPU offload layers to maximum

Architecture

Parameters: 14B (dense transformer)
Layers: 40
Hidden dimension: 5120
Attention heads: 32 (8 KV heads for GQA)
Vision encoder: 24-layer Pixtral ViT
Context: 131K (native), 393K (extended)
Vocabulary: Mistral Tekken tokenizer

Hardware Requirements

Minimum: 8 GB VRAM for text-only, 10 GB for vision
Recommended: 16 GB VRAM for full GPU offload
Disk: ~7.8 GB for model + mmproj

Quantization Details

Metric	Value
Source format	FP8_E4M3 (safetensors)
Intermediate	F16 GGUF
Output format	NVFP4
Approximate BPW	~4.6
Quantized with	llama.cpp build 537

License

Apache 2.0 — same as the base model.

Downloads last month: 120

GGUF

Model size

14B params

Architecture

mistral3

Hardware compatibility

4-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF

Base model

mistralai/Ministral-3-14B-Base-2512

Quantized

mistralai/Ministral-3-14B-Instruct-2512

Quantized

(30)

this model