RobbedoesHF/aya-23-dutch-definition-expansion-qlora

QLoRA Adapter for Dutch Definition Expansion (AYA 23)

This repository contains a QLoRA adapter fine-tuned on CohereLabs/aya-expanse-8b for the task of sense-preserving definitional expansion in Dutch.

This work was developed as part of the Master's thesis, "Transformer-based Expansion of Dutch Dictionary Definitions", submitted for the degree of Master of Science in Artificial Intelligence at KU Leuven.

About the Thesis

The research investigates the potential of transformer-based models to automate a significant bottleneck in contemporary lexicography: the manual expansion of concise, core-meaning definitions into comprehensive, formally structured dictionary entries. The study focuses on Dutch, a task requiring not only semantic accuracy but also strict adherence to lexicographical style and structure.

The thesis empirically compares two primary methodologies: in-context learning via few-shot prompting and adaptation via parameter-efficient fine-tuning (specifically, QLoRA). This comparison was conducted across a range of powerful multilingual and Dutch-specific models, including mT5-xl, GEITje Ultra, Aya-101, and Aya-23, to determine the most effective strategy for this high-precision domain.

This Model's Role and Performance

This fine-tuned Aya-23 model emerged as the top-performing and most reliable model in the study. The thesis concludes that for any practical application, "consistent reliability is far more valuable than sporadic excellence" and thus the overall winner is the fine-tuned Aya-23 model.

Its success is best understood in comparison to the best few-shot model, Aya-101. While the few-shot Aya-101 achieved the highest quantitative scores, its performance was found to be fundamentally unreliable, characterized by a "hit-or-miss" profile with outputs ranging from perfect replications to complete failures. This was because its success was entirely contingent on the structural quality of the few-shot examples, operating closer to "brittle pattern mimicry" than true generalization.

In stark contrast, this fine-tuned Aya-23 model demonstrated far greater reliability. Fine-tuning enabled it to internalize and robustly generalize lexicographical patterns, consistently producing structurally sound definitions and adhering to formal conventions. Its errors were typically minor and predictable, such as slight omissions, making it a trustworthy and practical tool. This consistency makes it the most suitable candidate for a "lexicographer's assistant," capable of generating high-quality first drafts that a human expert can trust and refine.

How to Use

To use this adapter you must first load the base model (`CohereLabs/aya-expanse-8b') in 4-bit and then apply this adapter on top of it.

import torch
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

base_model_id = "CohereLabs/aya-expanse-8b"
adapter_id = "RobbedoesHF/aya-23-8b-dutch-definition-expansion-qlora" # The repo ID of this adapter

# Load the base model with 4-bit quantization
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)

model = AutoModelForSeq2SeqLM.from_pretrained(
    base_model_id,
    quantization_config=bnb_config,
    device_map="auto",
    attn_implementation="sdpa", # Recommended
    trust_remote_code=True, # What was used in the thesis
)
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
tokenizer.pad_token = tokenizer.eos_token # Set pad token

# Apply the LoRA adapter
model = PeftModel.from_pretrained(model, adapter_id)

print("Model loaded successfully!")

Prompting Format

This adapter was fine-tuned on a specific instructional prompt. For best results, your input should match this structure.

# Define the lemma and short definition you want to expand
lemma = "ecoroman"
short_def = "roman over milieuproblematiek"


# Manually construct the prompt to match the fine-tuning format
system_prompt = "Je bent een expert-lexicograaf die definities schrijft voor een Nederlands woordenboek."
user_prompt = f"Breid de volgende korte definitie voor het woord '{lemma}' uit tot een volledige definitie: '{short_def}'"
full_user_prompt = f"{system_prompt}\n\n{user_prompt}"
prompt = f"<|START_OF_TURN_TOKEN|><|USER_TOKEN|>{full_user_prompt}<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>"

# Tokenize the prompt
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

# Generate the output tokens
print("
Generating definition...")
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=512, # Chosen based on the longest full definition's token length for this model
        num_beams=4, # What was used for the thesis
        early_stopping=True,
        pad_token_id=tokenizer.eos_token_id
    )

# Decode and clean the output
# The output includes the prompt so we split for the assistant's response
decoded_output = tokenizer.decode(outputs[0], skip_special_tokens=True)
assistant_response = decoded_output.split("<|assistant|>")[1].strip()

print("\n--- Prompt ---")
print(prompt)
print("\n--- Model Output ---")
print(decoded_output)