Wondernutts/MS3.1-24B-Magnum-Diamond-int8-ov

MS3.1 24B Magnum Diamond - merged OpenVINO INT8

This repository contains a fully merged, text-only OpenVINO INT8 build of DS-Archive/MS3.1-24B-Magnum-Diamond-LoRA on its exact ZeroAgency Mistral Small 3.1 text-only base.

The Magnum Diamond adapter was created by Doctor Shotgun/DS-Archive. ZeroAgency provided the Hugging Face text-only base conversion. Wondernutts performed the validated BF16 merge and OpenVINO INT8 conversion. Read and credit the adapter's original card for its training details, intended use, and original acknowledgements.

What it is

The source is an rsLoRA creative-writing and roleplay tune intended to bring the prose character of the Magnum/Claude-inspired family to Mistral Small 3.1 24B. The adapter is designed to work with or without prepended character names and with or without a response prefill.

This repository is not an adapter. It contains the merged weights as a self-contained OpenVINO model. PEFT and the source base are not required at inference time.

The source adapter card notes that a newer MS3.2 adapter exists. This artifact intentionally preserves the named MS3.1 adapter and should not be mistaken for that newer release.

Conversion details

Item	Value
Adapter	DS-Archive/MS3.1-24B-Magnum-Diamond-LoRA at 5a08273204fe16d5b3084f64f789d3c184cd8798
Exact base used	ZeroAgency/Mistral-Small-3.1-24B-Instruct-2503-hf at 8f787190232922ae904ec5f387f2347ad99e12be
Merge	BF16, rsLoRA merged with saved embed_tokens and untied lm_head preserved
Weight compression	INT8 asymmetric, per-channel, ratio 1.0
Packed constants	564 unsigned INT8 language-model weight constants
Source IR	Temporary FP16 OpenVINO IR
OpenVINO binary payload	approximately 21.96 GiB
Export	stateful text generation with tokenizer and detokenizer IRs

The BF16 step is the adapter merge. The FP16 IR was only a temporary export checkpoint used before compression. The final repository contains packed INT8 weights; it is not an FP16 model.

The exported configuration advertises 131,072 positions. The adapter was trained with a 32,768-token sequence length, so behavior beyond that training window has not been independently validated for this conversion.

Download

python -m pip install "openvino-genai==2026.2.1.0" "huggingface_hub>=1.0"
hf download Wondernutts/MS3.1-24B-Magnum-Diamond-int8-ov --local-dir ./ms31-24b-magnum-diamond-int8-ov

Run with OpenVINO GenAI

import openvino_genai as ov_genai

model_dir = "./ms31-24b-magnum-diamond-int8-ov"
pipe = ov_genai.LLMPipeline(
    model_dir,
    "GPU",
    CACHE_DIR="./ov_cache",
    DYNAMIC_QUANTIZATION_GROUP_SIZE=128,
)

config = ov_genai.GenerationConfig()
config.max_new_tokens = 768
config.do_sample = True
config.temperature = 0.9
config.top_p = 0.95
config.repetition_penalty = 1.1

pipe.start_chat("You are a perceptive fantasy character. Stay in character and write vivid prose.")
print(pipe.generate(
    "The campfire dies as footsteps circle beyond the trees.",
    generation_config=config,
))
pipe.finish_chat()

Use LLMPipeline; the selected base contains only the text component and does not support vision. start_chat() applies the bundled Mistral template and retains KV-cache state across turns. Change "GPU" to "CPU" for CPU inference.

Arc Pro B70 benchmark

Tested on one Intel Arc Pro B70 with prefix caching disabled. Each PP result is the mean of two exact-token runs after one same-shape warmup. Decode used 128 forced output tokens.

DQGS	512 PP tok/s	2K PP tok/s	6K PP tok/s	Decode tok/s
0	2,680	2,472	2,237	22.20
128	4,017	3,755	3,225	22.19

DQGS=128 improved PP by 49.9% at 512, 51.9% at 2K, and 44.2% at 6K. The generated-token hashes matched DQGS=0 at every measured shape, including the 128-token decode run.

Intended use and limitations

The source adapter is intended for fiction, creative writing, and roleplay, not factual advice. Its biases and failure modes carry into this conversion, and INT8 compression may introduce additional quality loss. Evaluate the output for your own application.

License and attribution

Apache-2.0, following the adapter and base repositories. Adapter and training credit: Doctor Shotgun / DS-Archive. Text-only base conversion: ZeroAgency. OpenVINO merge and conversion: Wondernutts.