How to use from the
Use from the
Transformers library
# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="aitf-its-tim3-dfk/ministral-8b-merged-ws3")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("aitf-its-tim3-dfk/ministral-8b-merged-ws3")
model = AutoModelForMultimodalLM.from_pretrained("aitf-its-tim3-dfk/ministral-8b-merged-ws3")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))
Quick Links
image

KomdigiITS-8B-DFK
Merged Classification

Ministral-3-8B · SLERP Merge · Multimodal + Text
SLERP · 0.30 multimodal · 0.70 text
✶
01 Overview

A SLERP-merged full model combining two specialized LoRA adapters into a single unified checkpoint. The merge blends the 8B Multimodal Multimodal classifier with the text-only DFK classifier, both built on KomdigiITS-8B-DFK-CPT (Ministral-3-8B-Base-2512).

The result is a model that operates in two modes: image-based multimodal classification and text-only classification — strong performance on both tasks from a single set of weights.

✶ Note: Best SLERP blend from a parameter sweep (w=0.30 multimodal, w=0.70 text). Parent adapters were merged into base model weights, producing a standalone model with no adapter overhead.
02 Model Details
Identity
DevelopedDFK Tim 1 & Tim 3 ITS
TypeMerged model (SLERP)
LanguageIndonesian
ModesMultimodal + Text-only
Architecture
ArchMistral3ForConditionalGeneration
Params8B (full merged weights)
Precisionfloat16

Parent Models

Multimodal Parent · w=0.30
TaskMultimodal — image + text
Datasetdfk_vlm_dataset_v3
Text Parent · w=0.70
TaskText-only DFK classification
Datasetdfk_text_dataset
03 Uses

Direct Use

This model supports two input modes from a single checkpoint:
Mode 1 · Multimodal
Image-based content moderation. Given a social media screenshot with contextual metadata, classifies into 4 labels: netral, disinformasi, fitnah, or ujaran kebencian.
Mode 2 · Text-Only
Text-based DFK detection using article references. Classifies into 5 labels: Fakta, Disinformasi, Fitnah, Ujaran Kebencian, or Non-DFK.

Out-of-Scope Use

Not intended for general-purpose vision-language or text generation tasks. Specialized for the DFK detection pipeline — should not be used for content moderation in other languages or domains without further fine-tuning.
04 Evaluation

Evaluated on held-out validation splits using greedy decoding (temperature=0.0) and BERTScore (bert-base-multilingual-cased).

Multimodal Task (Image + Text)

88.5
Accuracy
89.4
F1 Weighted
77.3
BERTScore F1

Text-Only Task

91.0
Accuracy
92.9
F1 Weighted
77.6
BERTScore F1
Parent Model Comparison

The merge trades a small amount of Multimodal accuracy for a massive improvement in text-only performance — turning a Multimodal-only model into a genuinely dual-mode classifier.

Model MM Acc MM F1w MM BERT MM ROUGE Text Acc Text F1w Text BERT Text ROUGE
Multimodal 92.5 92.3 80.0 38.7 77.5 70.8 73.1 19.0
Text Adapter 1.5 2.8 69.6 16.2 84.0 89.0 80.7 41.6
SLERP Merge 88.5 89.4 77.3 31.6 91.0 92.9 77.6 31.5
✶ Key takeaway: The merge surpasses the text adapter's own accuracy (91.0 vs 84.0) while retaining 95.7% of the Multimodal model's classification accuracy.
05 Merge Details

Method

MethodSLERP (Spherical Linear Interpolation)
Weight0.30 multimodal / 0.70 text
SelectionBest blend from WS3 sweep
OutputFull merged weights (no adapter)

Process

Both parent LoRA adapters (Multimodal + text) were first merged into the base model (KomdigiITS-8B-DFK-CPT) to produce two full-weight checkpoints. These were then interpolated via SLERP at various weight ratios, with each blend evaluated on both Multimodal and text-only benchmarks to find the optimal trade-off.

Label Classes

Multimodal Mode · 4 Classes
NetralFactual / non-DFK — no violation
DisinfoClaims contradicting established facts
FitnahFalse claims targeting an individual
Ujrn KbnciHate speech targeting SARA identity
Text Mode · 5 Classes
Non-DFKContent unrelated to DFK categories
FaktaFactual content, verified true
DisinfoClaims contradicting established facts
FitnahFalse claims targeting an individual
Ujrn KbnciHate speech targeting SARA identity
06 Input Formats

Both modes use the ministral_3 chat template ([INST] / [/INST] delimiters). The default Ministral system prompt is included when no explicit system message is provided.

Multimodal Mode · dfk_vlm_dataset_v3

Image-based classification with contextual metadata. Note: the instruction and context fields are directly concatenated with no separator.

<s>[SYSTEM_PROMPT]...default Ministral system prompt...[/SYSTEM_PROMPT][INST]Anda adalah seorang analis konten media sosial ahli. Diberikan tangkapan layar dari sebuah konten, tentukan label kategori pelanggaran dan berikan analisis detail mengenai pelanggaran yang ditemukan.Ringkasan: {ringkasan}
Klaim: {klaim}
Fakta: {fakta}[IMG][/INST]Label: {label}

Analisis: {analisis}</s>

Input Fields

RingkasanContent summary. Concatenation of image caption and user-provided text.
KlaimCore claim extracted from the content, used as a web search query for fact-checking.
FaktaVerification context from web search. Defaults to "Tidak ditemukan sumber yang valid." if none found.
[IMG]Screenshot of the social media post being analyzed.

Output Fields

LabelOne of netral, disinformasi, fitnah, or ujaran kebencian.
AnalisisFree-form Indonesian-language reasoning for the classification.
Text-Only Mode · dfk_text_dataset

Text-only classification with explicit system prompt and article references. Uses merge_labels=false (5 classes).

<s>[SYSTEM_PROMPT]Anda adalah sistem deteksi konten DFK berbasis artikel rujukan. Tugas Anda adalah membandingkan klaim dengan artikel rujukan, lalu mengklasifikasikan teks ke dalam salah satu label: Fakta, Disinformasi, Fitnah, Ujaran Kebencian, atau Non-DFK. Jawab dengan format: Label: **NamaLabel.** penjelasan: ...[/SYSTEM_PROMPT][INST]{klaim}

Artikel Rujukan: {fakta}[/INST]Label: {label}. penjelasan: {analisis}</s>

Input Fields

KlaimThe claim text to be verified, from the input column (before Artikel Rujukan:).
FaktaReference articles for fact-checking, from the input column (after Artikel Rujukan:).

Output Fields

LabelOne of Fakta, Disinformasi, Fitnah, Ujaran Kebencian, or Non-DFK.
PenjelasanIndonesian-language explanation for the classification decision.
08 Framework Versions
Transformers5.5.0
PyTorch2.11.0+cu128
Unsloth2026.5.5
Downloads last month
80
Safetensors
Model size
9B params
Tensor type
F16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for aitf-its-tim3-dfk/ministral-8b-merged-ws3