Instructions to use unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF",
	filename="Devstral-Small-2-24B-Instruct-2512-BF16.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:UD-Q4_K_XL
# Run inference directly in the terminal:
llama cli -hf unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:UD-Q4_K_XL

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:UD-Q4_K_XL
# Run inference directly in the terminal:
llama cli -hf unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:UD-Q4_K_XL

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:UD-Q4_K_XL
# Run inference directly in the terminal:
./llama-cli -hf unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:UD-Q4_K_XL

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:UD-Q4_K_XL
# Run inference directly in the terminal:
./build/bin/llama-cli -hf unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:UD-Q4_K_XL

Use Docker

docker model run hf.co/unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:UD-Q4_K_XL

LM Studio
Jan
Ollama
How to use unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF with Ollama:
```
ollama run hf.co/unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:UD-Q4_K_XL
```

Unsloth Studio

How to use unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF to start chatting

How to use unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:UD-Q4_K_XL

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:UD-Q4_K_XL"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:UD-Q4_K_XL

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:UD-Q4_K_XL

Run Hermes

hermes

Atomic Chat new
Docker Model Runner
How to use unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF with Docker Model Runner:
```
docker model run hf.co/unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:UD-Q4_K_XL
```

Lemonade

How to use unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:UD-Q4_K_XL

Run and chat with the model

lemonade run user.Devstral-Small-2-24B-Instruct-2512-GGUF-UD-Q4_K_XL

List all available models

lemonade list

danielhanchen commited on Dec 15, 2025

Commit

c4433c8

verified ·

1 Parent(s): b803431

Add files using upload-large-folder tool

Browse files

Files changed (32) hide show

.gitattributes +1 -0
Devstral-Small-2-24B-Instruct-2512-BF16.gguf +2 -2
Devstral-Small-2-24B-Instruct-2512-IQ4_NL.gguf +2 -2
Devstral-Small-2-24B-Instruct-2512-IQ4_XS.gguf +2 -2
Devstral-Small-2-24B-Instruct-2512-Q2_K.gguf +2 -2
Devstral-Small-2-24B-Instruct-2512-Q2_K_L.gguf +2 -2
Devstral-Small-2-24B-Instruct-2512-Q3_K_M.gguf +2 -2
Devstral-Small-2-24B-Instruct-2512-Q3_K_S.gguf +2 -2
Devstral-Small-2-24B-Instruct-2512-Q4_0.gguf +2 -2
Devstral-Small-2-24B-Instruct-2512-Q4_1.gguf +2 -2
Devstral-Small-2-24B-Instruct-2512-Q4_K_M.gguf +2 -2
Devstral-Small-2-24B-Instruct-2512-Q4_K_S.gguf +2 -2
Devstral-Small-2-24B-Instruct-2512-Q5_K_M.gguf +2 -2
Devstral-Small-2-24B-Instruct-2512-Q5_K_S.gguf +2 -2
Devstral-Small-2-24B-Instruct-2512-Q6_K.gguf +2 -2
Devstral-Small-2-24B-Instruct-2512-Q8_0.gguf +2 -2
Devstral-Small-2-24B-Instruct-2512-UD-IQ1_M.gguf +2 -2
Devstral-Small-2-24B-Instruct-2512-UD-IQ1_S.gguf +2 -2
Devstral-Small-2-24B-Instruct-2512-UD-IQ2_M.gguf +2 -2
Devstral-Small-2-24B-Instruct-2512-UD-IQ2_XXS.gguf +2 -2
Devstral-Small-2-24B-Instruct-2512-UD-IQ3_XXS.gguf +2 -2
Devstral-Small-2-24B-Instruct-2512-UD-Q2_K_XL.gguf +2 -2
Devstral-Small-2-24B-Instruct-2512-UD-Q3_K_XL.gguf +2 -2
Devstral-Small-2-24B-Instruct-2512-UD-Q4_K_XL.gguf +2 -2
Devstral-Small-2-24B-Instruct-2512-UD-Q5_K_XL.gguf +2 -2
Devstral-Small-2-24B-Instruct-2512-UD-Q6_K_XL.gguf +2 -2
Devstral-Small-2-24B-Instruct-2512-UD-Q8_K_XL.gguf +2 -2
config.json +74 -0
imatrix_unsloth.gguf +3 -0
mmproj-BF16.gguf +2 -2
mmproj-F16.gguf +2 -2
mmproj-F32.gguf +2 -2

.gitattributes CHANGED Viewed

@@ -63,3 +63,4 @@ Devstral-Small-2-24B-Instruct-2512-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
 Devstral-Small-2-24B-Instruct-2512-UD-Q8_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
 imatrix_unsloth.gguf_file filter=lfs diff=lfs merge=lfs -text
 Devstral-Small-2-24B-Instruct-2512-BF16.gguf filter=lfs diff=lfs merge=lfs -text

 Devstral-Small-2-24B-Instruct-2512-UD-Q8_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
 imatrix_unsloth.gguf_file filter=lfs diff=lfs merge=lfs -text
 Devstral-Small-2-24B-Instruct-2512-BF16.gguf filter=lfs diff=lfs merge=lfs -text
+imatrix_unsloth.gguf filter=lfs diff=lfs merge=lfs -text

Devstral-Small-2-24B-Instruct-2512-BF16.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8285f809c440e2d508d3f4707da5189df617504c059769f7d4c62d5f0671a507
-size 47153531616

 version https://git-lfs.github.com/spec/v1
+oid sha256:6a86365cc26ec2e5ba1434aa85da15a7de28eb92015447623a780f1f86ab1d1b
+size 47154056032

Devstral-Small-2-24B-Instruct-2512-IQ4_NL.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:97f50bb06d6c5855b41531c6faff7c1bd1dd321e0cd34b08b0eb33cfe1fbb566
-size 13468027936

 version https://git-lfs.github.com/spec/v1
+oid sha256:455cc22c4d85e93acbb1df2760e4df5626dc86ca37b0bee39e479e560f09508f
+size 13468552352

Devstral-Small-2-24B-Instruct-2512-IQ4_XS.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:edd369b5e8556b1da7d72094509d7efaebd994dfabcbbec13f3b4893bfc80d15
-size 12779899936

 version https://git-lfs.github.com/spec/v1
+oid sha256:6b8270a839e7a1263f34a799c18fb9eb0ca6f1d039cdbfa4a11f9ac9552a118a
+size 12780424352

Devstral-Small-2-24B-Instruct-2512-Q2_K.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8480542b918780bb4c7651a5bb4ad5885ebfeea4f3187da9c25ca3e90cf0ad15
-size 8890338336

 version https://git-lfs.github.com/spec/v1
+oid sha256:408a61e7dcc0eb7fda7d70979395f974fe17310b4f38f977c95d9b98095fe1aa
+size 8890862752

Devstral-Small-2-24B-Instruct-2512-Q2_K_L.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c7c2464f80baa9c603ab364857154470211a21ed373eea54ab0366f2a813c5e6
-size 9047624736

 version https://git-lfs.github.com/spec/v1
+oid sha256:e32066271c0bbd03ce8a7648ebabf8335fbf16cbf16023389e987d7fc70111f7
+size 9048149152

Devstral-Small-2-24B-Instruct-2512-Q3_K_M.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1748f5f78d5b7a87a23d32d39ff3e1561c9962edbd5aff073ad60c9623686165
-size 11474095136

 version https://git-lfs.github.com/spec/v1
+oid sha256:53466aa3e486e6f0068c2593d12aeb15fb079c1dfe443ab922c468c39761bd16
+size 11474619552

Devstral-Small-2-24B-Instruct-2512-Q3_K_S.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f796975d2a90d5dc335632cafcfd622f64005900eb5338e8f6a7b32816c7f141
-size 10400287776

 version https://git-lfs.github.com/spec/v1
+oid sha256:2647a4944f3783e6744ab94a85d9d604ac6ea0f1bcbb78b90afbf823543bb501
+size 10400812192

Devstral-Small-2-24B-Instruct-2512-Q4_0.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c57da38d5c5712a951d462765531b55dec3643fed93d7c6b55e2f9d302437c96
-size 13494242336

 version https://git-lfs.github.com/spec/v1
+oid sha256:af9a7e072a101d0ee2befef444de1b78cdb4dce7f4515bcba7751ce78eecded5
+size 13494766752

Devstral-Small-2-24B-Instruct-2512-Q4_1.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1ffab801696e1bd5e21359d444899786e7b05e2f79e3ece9b95448596cc6340a
-size 14873119776

 version https://git-lfs.github.com/spec/v1
+oid sha256:0d075ca8b3614e9ef9597630c7d7b92816d31787f119ed81d306518d0e72d094
+size 14873644192

Devstral-Small-2-24B-Instruct-2512-Q4_K_M.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6592464426486e53f93a08aa9a713f560261c629b92c4c822343b97c3105c9ca
-size 14333922336

 version https://git-lfs.github.com/spec/v1
+oid sha256:d14ba9edee1bb4c4996a726deb81e49ae81800a3216f0774634238c380aee496
+size 14334446752

Devstral-Small-2-24B-Instruct-2512-Q4_K_S.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:878c12bd0ab31d95326723931daa6cbc246bbd41552db27f3beb95c98a2e5d01
-size 13549292576

 version https://git-lfs.github.com/spec/v1
+oid sha256:3a23a8cdce2b76b5f1338e5090be2068b3a9a1b62b8ad5277975f3e345220901
+size 13549816992

Devstral-Small-2-24B-Instruct-2512-Q5_K_M.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2dd491cedc5298dbb4b3f76548c97683546595d4dbae989fd3677bf041182d73
-size 16763997216

 version https://git-lfs.github.com/spec/v1
+oid sha256:2da6ca6c4ae387aa7f3f2f4a67bb3e1ca570ce0c69c21e26b8695e75172443b0
+size 16764521632

Devstral-Small-2-24B-Instruct-2512-Q5_K_S.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6ec97290753fa75adc73418c369aae00e1a4cfa3fe01723bf5a1b93ccfadface
-size 16304426016

 version https://git-lfs.github.com/spec/v1
+oid sha256:5acecb4247289f9b3d00307d94531936fe60a0664c8b841cf8f5b454c66ee757
+size 16304950432

Devstral-Small-2-24B-Instruct-2512-Q6_K.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f3e3b6fced15f29cc8eb34de10c331a39bc830367a20c5369dcb89e5a349a1a1
-size 19345951776

 version https://git-lfs.github.com/spec/v1
+oid sha256:b04b3e19730d7a1e19530f40b947b69c028b090cb7c58c1515cf1fc2ece5f821
+size 19346476192

Devstral-Small-2-24B-Instruct-2512-Q8_0.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2aef7c4d3a7b125d501c5b7f23d3744208758b1447dc509bd76dd8773484bf41
-size 25054792736

 version https://git-lfs.github.com/spec/v1
+oid sha256:0760502e9228234f6cfa843f8870b8fc91c46a13664cf766c639229cccc80866
+size 25055317152

Devstral-Small-2-24B-Instruct-2512-UD-IQ1_M.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4aa6b8352b060ec7b19c1c6aebac9798b8fcf7155aeee8b84dd1f3ea561bab4a
-size 6017322016

 version https://git-lfs.github.com/spec/v1
+oid sha256:e1a27d27597a32037e6796d3f94df7c17ff8454341d1661c1e99b7f6e54e0fba
+size 6017846432

Devstral-Small-2-24B-Instruct-2512-UD-IQ1_S.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:acc148b369debf7d7c5da915e71847cb2399a1d47b33d664d8f989fb5e75ef73
-size 5558570016

 version https://git-lfs.github.com/spec/v1
+oid sha256:a80ab11ca4af5af7fe42fd32c1a552e8eac4e4882be5646eac3273017a0e2a37
+size 5559094432

Devstral-Small-2-24B-Instruct-2512-UD-IQ2_M.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9a49b460b68c9a8c3201cad28e922c3a64ab1f05541350fe09981c0cbeb45b04
-size 8243334176

 version https://git-lfs.github.com/spec/v1
+oid sha256:669920de0afd6b7b5cf97da7779e7c63f8b57c9930b7669e9f23b1c46c2304a6
+size 8243858592

Devstral-Small-2-24B-Instruct-2512-UD-IQ2_XXS.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:82518b97eaaa8ff4defc182dea48ebd456a834eb4058a26bafe25e82ccc019b5
-size 6750915616

 version https://git-lfs.github.com/spec/v1
+oid sha256:40e93f2c700057860a4917552589d27119127070405cde3e7ed1c7f9ba4fc6db
+size 6751440032

Devstral-Small-2-24B-Instruct-2512-UD-IQ3_XXS.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1065dc7803d502e4cbd61b59cad95bc5dadf8caa87dc644c5606e23ea0cc3b2d
-size 9409219616

 version https://git-lfs.github.com/spec/v1
+oid sha256:6ab6264a46f5c530cee3a999a57e828ed4609770d01172e8124c1876e80e6415
+size 9409744032

Devstral-Small-2-24B-Instruct-2512-UD-Q2_K_XL.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:424f1202393d22f23215a2dced7fcd4bce6bc6d245a50161f98db99cffcfd55b
-size 9292155936

 version https://git-lfs.github.com/spec/v1
+oid sha256:d6e7b4d124338d00d698ce22432b43586941da5218d992952fb04bd8b8ae71f6
+size 9292680352

Devstral-Small-2-24B-Instruct-2512-UD-Q3_K_XL.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f614c18e7a690abb07a07d47ba139abfdfaaace6ecd470b0ada2a267f68a5d78
-size 11850886176

 version https://git-lfs.github.com/spec/v1
+oid sha256:613c44b12179f8380eb14f413784ad0412ee527e86d762a8d8816ac7ce0d42ca
+size 11851410592

Devstral-Small-2-24B-Instruct-2512-UD-Q4_K_XL.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3ab5391fa0ab00bc29602f01b77d0f55d90a6a950a679d037fd04381dc6bb244
-size 14505626656

 version https://git-lfs.github.com/spec/v1
+oid sha256:b44e34b78180fc3ab1abbe1edad9f1f3926fdca10eed3bfae168b065e683f6cd
+size 14506151072

Devstral-Small-2-24B-Instruct-2512-UD-Q5_K_XL.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5a39248c2155a84b378ee97caf5d41369413f6e69d358052b2227d294c19cbe7
-size 16765840416

 version https://git-lfs.github.com/spec/v1
+oid sha256:1bfce0c4ead4a46178bf42bb5b82ca7f6fb3b3126271d9f4c223c69fbc2c3ffe
+size 16766364832

Devstral-Small-2-24B-Instruct-2512-UD-Q6_K_XL.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:88cf41a250c53ffecf66eec29e473e1fe423aec52db245cd0e79d11a46beedc4
-size 20788399136

 version https://git-lfs.github.com/spec/v1
+oid sha256:10fa6b4b4b19f4dfc3c436dd28c1098aede6598b56e04a00c46c2eac98c16804
+size 20788923552

Devstral-Small-2-24B-Instruct-2512-UD-Q8_K_XL.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:898bea46e986e94336452c227d9078c0d70626b4e55b374e33fe4a2e09274c56
-size 28991867936

 version https://git-lfs.github.com/spec/v1
+oid sha256:4a50696a851571a53603b7496c444b30b6f5efa9b85c3d9d3e716db59985a06c
+size 28992392352

config.json ADDED Viewed

	@@ -0,0 +1,74 @@

+{
+  "architectures": [
+    "Mistral3ForConditionalGeneration"
+  ],
+  "torch_dtype": "bfloat16",
+  "image_token_index": 10,
+  "model_type": "mistral3",
+  "multimodal_projector_bias": false,
+  "pad_token_id": 11,
+  "projector_hidden_act": "gelu",
+  "quantization_config": {
+    "activation_scheme": "static",
+    "dequantize": false,
+    "modules_to_not_convert": [
+      "model.vision_tower",
+      "model.multi_modal_projector",
+      "lm_head"
+    ],
+    "quant_method": "fp8",
+    "weight_block_size": null
+  },
+  "spatial_merge_size": 2,
+  "text_config": {
+    "attention_dropout": 0.0,
+    "head_dim": 128,
+    "hidden_act": "silu",
+    "hidden_size": 5120,
+    "initializer_range": 0.02,
+    "intermediate_size": 32768,
+    "max_position_embeddings": 393216,
+    "model_type": "ministral3",
+    "num_attention_heads": 32,
+    "num_hidden_layers": 40,
+    "num_key_value_heads": 8,
+    "rms_norm_eps": 1e-05,
+    "rope_parameters": {
+      "beta_fast": 32.0,
+      "beta_slow": 1.0,
+      "factor": 48.0,
+      "llama_4_scaling_beta": 0.1,
+      "mscale": 1.0,
+      "mscale_all_dim": 1.0,
+      "original_max_position_embeddings": 8192,
+      "rope_theta": 100000000.0,
+      "rope_type": "yarn",
+      "type": "yarn"
+    },
+    "sliding_window": null,
+    "use_cache": true,
+    "vocab_size": 131072
+  },
+  "tie_word_embeddings": false,
+  "transformers_version": "5.0.0rc1",
+  "unsloth_fixed": true,
+  "vision_config": {
+    "attention_dropout": 0.0,
+    "head_dim": 64,
+    "hidden_act": "silu",
+    "hidden_size": 1024,
+    "image_size": 1540,
+    "initializer_range": 0.02,
+    "intermediate_size": 4096,
+    "model_type": "pixtral",
+    "num_attention_heads": 16,
+    "num_channels": 3,
+    "num_hidden_layers": 24,
+    "patch_size": 14,
+    "rope_parameters": {
+      "rope_theta": 10000.0,
+      "rope_type": "default"
+    }
+  },
+  "vision_feature_layer": -1
+}

imatrix_unsloth.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:57a041eab0edf63c5e00b86df544e4c213300ac0828e27858e4eaa433ba7e5f1
+size 10037344

mmproj-BF16.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ca1b45f9d039ffbea3cad4de65d0277b081601e0bc14af4271c6a8ab4d81ed93
-size 879258240

 version https://git-lfs.github.com/spec/v1
+oid sha256:7fbd63a14ed6642370f7a8c276fdb3c1061f3349468dc7bb3a0a9e803e70926b
+size 879258272

mmproj-F16.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5e2e687e715db43a59e1b7f532263f497aded771ee216ad416501c34f0b5c440
-size 878054016

 version https://git-lfs.github.com/spec/v1
+oid sha256:236bb4f0500620164917f26749cd402485b7d749034ea817c6a2d65253b126b9
+size 878054048

mmproj-F32.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:fee361ea1a4b2abc2b329e6872b0ab4dec700617a4a2ecc19598c3366ad58b8c
-size 1755867776

 version https://git-lfs.github.com/spec/v1
+oid sha256:f140d901ebc3d3e79dac0d6549592049d9cf411ef41e049e49afaaa72b5255ef
+size 1755867808