Instructions to use AutomatosX/AX-Ministral-3-8B-Instruct-2512-MLX-AXQ-6bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AutomatosX/AX-Ministral-3-8B-Instruct-2512-MLX-AXQ-6bit with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("AutomatosX/AX-Ministral-3-8B-Instruct-2512-MLX-AXQ-6bit")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use AutomatosX/AX-Ministral-3-8B-Instruct-2512-MLX-AXQ-6bit with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "AutomatosX/AX-Ministral-3-8B-Instruct-2512-MLX-AXQ-6bit"

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "AutomatosX/AX-Ministral-3-8B-Instruct-2512-MLX-AXQ-6bit"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

MLX LM

How to use AutomatosX/AX-Ministral-3-8B-Instruct-2512-MLX-AXQ-6bit with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "AutomatosX/AX-Ministral-3-8B-Instruct-2512-MLX-AXQ-6bit"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "AutomatosX/AX-Ministral-3-8B-Instruct-2512-MLX-AXQ-6bit"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "AutomatosX/AX-Ministral-3-8B-Instruct-2512-MLX-AXQ-6bit",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Hermes Agent

How to use AutomatosX/AX-Ministral-3-8B-Instruct-2512-MLX-AXQ-6bit with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "AutomatosX/AX-Ministral-3-8B-Instruct-2512-MLX-AXQ-6bit"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default AutomatosX/AX-Ministral-3-8B-Instruct-2512-MLX-AXQ-6bit

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use AutomatosX/AX-Ministral-3-8B-Instruct-2512-MLX-AXQ-6bit with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "AutomatosX/AX-Ministral-3-8B-Instruct-2512-MLX-AXQ-6bit"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "AutomatosX/AX-Ministral-3-8B-Instruct-2512-MLX-AXQ-6bit" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Why only 6bit? Protection floors lift the low-memory plan to ~5.99 BPW, essentially the same as this ~6.00 BPW pack (~6.7 GB). A separate AXQ-4bit sibling would not save memory — use this pack only.

AX-Ministral-3-8B-Instruct-2512-MLX-AXQ-6bit

An AXQuant (AXQ) mixed-precision MLX checkpoint for Apple Silicon, converted directly from the BF16 source model. The language path is quantized while the vision tower are preserved as BF16 sidecars when present.

Development evidence — not a certified AXQuant release. This package has conversion and artifact-integrity records, but it does not publish measured quality, long-context, kernel-speed, or MTP-speed evidence. Do not interpret the AXQ product label as a benchmark claim.

Stable-name v2. main serves the audited v2 artifact for backward compatibility. The same revision is tagged v2; the replaced artifact remains recoverable at legacy-pre-v2.

Model details

Property	Value
Base model	mistralai/Ministral-3-8B-Instruct-2512-BF16
Source revision	`f6fae9795746f63c9be8344932f01275f3c63734`
Product family	`mistral3`
Source architecture	`Mistral3ForConditionalGeneration` (dense); text path optimized
Main-model parameters	8.92B logical parameters
Quantizer	AXQuant `1.2.0`
Hub budget class	`6bit`
Artifact edition	`v2`
AXQuant base precision class	`6bit`
Planned storage-adjusted BPW	5.9999
Measured main-model BPW	6.0000
Measured total BPW	6.0000
Safetensors weight size	6.69 GB
Approximate complete download	6.71 GB
Configured maximum context	262,144 tokens; practical limits depend on unified memory
MLX-LM compatibility	Standard text inference, compatibility level B
AX Engine native execution	Not established; no validated native manifest is included
MTP present	`False`
Vision sidecar present	`True`

This repository contains MLX Safetensors. It does not contain PyTorch or GGUF weights.

Why there is no AXQ-4bit pack

On Ministral-3-8B, AXQuant protection floors raise the low-memory (~~4.8 BPW) plan to about 5.99 BPW, while the 6bit plan sits at about 6.00 BPW. Download size and layout are nearly the same (~~6.7 GB); there is no meaningful size or precision ladder between the two product names.

The former …-AXQ-4bit sibling was retired so users are not offered a false “smaller 4bit” choice. AutomatosX keeps only this repository as the public AXQ pack for this base.


Measured main-model BPW	~6.00
Package size	~6.7 GB
Why not 4bit	Near floor-collapse; no useful 4 vs 6 trade-off

Choosing an AXQ pack

AXQ 4bit / 6bit names are storage-budget product classes, not a promise that every tensor uses that width. On this base there is no distinct 4bit Hub pack — see Why there is no AXQ-4bit pack above.

Sibling	Intended trade-off
(none published)	This 6bit pack is the only public AXQ checkpoint for this base.

See the AutomatosX MLX model catalog for related MLX and OptiQ alternatives.

Download

python -m pip install -U huggingface_hub
hf download AutomatosX/AX-Ministral-3-8B-Instruct-2512-MLX-AXQ-6bit --local-dir ./AX-Ministral-3-8B-Instruct-2512-MLX-AXQ-6bit

Allow at least 6.71 GB of free disk space. Pin the resulting Hub commit in reproducible deployments rather than relying indefinitely on main.

Run with MLX-LM

python -m pip install -U mlx-lm
mlx_lm.generate \
  --model AutomatosX/AX-Ministral-3-8B-Instruct-2512-MLX-AXQ-6bit \
  --prompt "Explain mixed-precision quantization in three sentences." \
  --max-tokens 128 \
  --temp 0.0

MLX-LM compatibility covers standard text/backbone inference. It may ignore AXQuant runtime metadata and optional sidecars (vision.safetensors, mtp.safetensors); this command therefore does not establish MTP acceleration or vision-language quality. The artifact records MLX 0.32.0 and MLX-LM 0.31.3 from conversion.

AX Engine status

This package does not include a validated native model-manifest.json, so AX Engine execution is not established by this release. The AX Engine fields in axquant_runtime.json describe the intended compatibility contract, not observed runtime evidence. Use the MLX-LM path above for standard text/backbone inference. The artifact records AX Engine version not recorded, but version discovery alone is not a runtime check.

Quantization layout

Main-weight precision	Parameters	Share
`4bit`	7.42B	83.15%
`8bit`	536.87M	6.02%
`bf16`	965.63M	10.83%

Quantization methods: affine, bf16.
Group sizes used by quantized assignments: 32, 64.
MTP sidecar: not included.
Vision sidecar: 222 tensors, 428.47M parameters, 0.86 GB, BF16.
Optimization scope: text-path.
Support tier: convertible.

BF16 sidecars, when present, are included in total download size. Their presence does not by itself establish MTP acceleration or vision-language quality.

Evidence and validation status

Check	Status
Planning evidence	`architecture_prior`
Calibration	none; the allocation is based on architecture priors
Quantizer execution	239/239 recorded module conversions succeeded; 0 fallbacks
AX Engine native manifest	not included
Quality versus BF16 or uniform baselines	Not published; no quality-retention claim
MTP acceptance and speed	not measured; no MTP speedup claim
AX Engine kernel evidence	`unmeasured`
Vision-language quality	Not evaluated or claimed; vision tensors are preserved at BF16
Long-context quality	262,144-token capacity is config metadata, not a validated claim
Release certification	Not certified; formal AXQuant M0-M8 gates are not closed

Intended use and limitations

Intended for local development and evaluation on Apple Silicon with MLX-compatible runtimes.
No minimum unified-memory figure is claimed; loadability depends on model size, context length, KV-cache policy, runtime buffers, and other processes using unified memory.
Architecture-prior allocation is not measured sensitivity. It must not be presented as measured model quality.
Vision weights are byte-preserved at BF16, but this release does not claim validated VLM quality.
The configured context window can require substantially more memory as the KV cache grows.
AX Engine execution is not established because this package has no validated native manifest.
Upstream capabilities, limitations, biases, and responsible-use guidance still apply.

Provenance and audit files

axquant_manifest.json: package identity, byte accounting, runtime contract, software versions, and file checksums.
axquant_plan.json: per-tensor precision decisions and planning evidence.
axquant_quantizer_execution.json: conversion coverage and fallback records.
axquant_runtime.json: declared AX Engine and MLX-LM compatibility metadata; runtime checks remain separate evidence.
axquant_vision_sidecar_manifest.json: protected vision tensor provenance.

All published provenance uses repository-relative paths. Local source paths are stripped before publication. The checkpoint was converted from BF16 rather than re-quantized from an OptiQ artifact. Parallel OptiQ repositories use a different quantizer and should not be assumed to have identical BPW or quality.

License

The checkpoint follows the upstream model license where applicable (often Apache License 2.0). See the mistralai/Ministral-3-8B-Instruct-2512-BF16 model card for license terms, model limitations, and responsible-use guidance.