safouaneelg/Apertus-8B-Instruct-2509-AQUA-RAT-SFT

Apertus has released two models: 70B and 8B parameter multi-language model. Check out the model info here: Swiss-AI/LLM

Finetuned on AQUA-RAT

This repo contains the fine-tuned version of Apertus on AQuA-RAT dataset.

The fine-tuning was performed using Unsloth on one GPU (RTX A6000 48 GB) with the following parameters:

• per_device_train_batch_size: 8
• gradient_accumulation_steps: 4 (effective batch size: 32)
• warmup_steps: 10
• num_train_epochs: 1
• learning_rate: 5e-5
• fp16/bf16: Enabled based on hardware support
• logging_steps: 1
• optimizer: adamw_8bit
• weight_decay: 0.01
• lr_scheduler_type: linear
• seed: 3407
• eval_strategy: steps
• eval_steps: 150
• packing: True

How to use

You can run this fine-tuned version using the below instructions:

1. Transformers 4.56.0 are required to run the model.

pip install -U transformers Unsloth

2. I have personally managed to run it after setting the xiELU activation function which can theoretically be installed via the below command line.

pip install git+https://github.com/rubber-duck-debug/xielu

If you struggle, check the xiELU installation in my other tune model (safouaneelg/Apertus-8B-Instruct-2509-GSM8k-SFT).

3. Run inference using:

• Transformers pipeline
• Unsloth pipeline (This works better, if you have StaticLayer error, comment/uncomment the arg prompt_lookup_num_tokens=None)

from unsloth import FastLanguageModel
import torch
# Load the model and tokenizer
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="safouaneelg/Apertus-8B-Instruct-2509-AQUA-RAT-SFT",
    max_seq_length=2048,
    load_in_4bit=True,
)
# Move to device
device = "cuda" if torch.cuda.is_available() else "cpu"
# Example prompt from AQUA-RAT
prompt = """Question: A grocery sells a bag of ice for $1.25, and makes 20% profit. If it sells 500 bags of ice, how much total profit does it make?
Options: A)125
B)150
C)225
D)250
E)275
Rationale:"""
messages_think = [
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages_think,
    tokenize=False,
    add_generation_prompt=True,
)
model_inputs = tokenizer([text], return_tensors="pt", add_special_tokens=False).to(model.device)
outputs = model.generate(
    **model_inputs,
    max_new_tokens=256,
    temperature=0.8,
    top_p=0.9,
    use_cache=True,
    do_sample=True,
    prompt_lookup_num_tokens=None #for some reasoning this sometimes solve the inferencing errors
)
generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(generated_text)

import os
from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
model_name = "safouaneelg/Apertus-8B-Instruct-2509-AQUA-RAT-SFT"
device = "cuda" if torch.cuda.is_available() else "cpu"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
).to(device)
# prepare the model input
prompt = """Question: A grocery sells a bag of ice for $1.25, and makes 20% profit. If it sells 500 bags of ice, how much total profit does it make?
Options: A)125
B)150
C)225
D)250
E)275
Rationale:"""
messages_think = [
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages_think,
    tokenize=False,
    add_generation_prompt=True,
)
streamer = TextStreamer(tokenizer)
model_inputs = tokenizer([text], return_tensors="pt", add_special_tokens=False).to(model.device)
# Generate the output
generated_ids = model.generate(**model_inputs, streamer=streamer, max_new_tokens=2024)
# Get and decode the output
output_ids = generated_ids[0][len(model_inputs.input_ids[0]) :]
print(tokenizer.decode(output_ids, skip_special_tokens=True))

Citation

@misc{swissai2025apertus,
  title={{Apertus: Democratizing Open and Compliant LLMs for Global Language Environments}},
  author={Apertus Team},
  year={2025},
  howpublished={\url{https://huggingface.co/swiss-ai/Apertus-8B-Instruct-2509}}
}