xxccho/EXAONE-3.5-7.8B-Instruct-ConvFinQA-SFT-GRPO

EXAONE-3.5-7.8B-Instruct ConvFinQA SFT+GRPO (LoRA Adapter)

This is a GRPO LoRA adapter for LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct, fine-tuned on the ConvFinQA dataset for financial reasoning.

Training Pipeline

This adapter is the second stage of a two-stage training pipeline:

1. SFT (Supervised Fine-Tuning): radishtiger/EXAONE-3.5-7.8B-Instruct_convfinqa_lora_r64_DR1.0
2. GRPO (Group Relative Policy Optimization): this adapter (xxccho/EXAONE-3.5-7.8B-Instruct-ConvFinQA-SFT-GRPO)

How to Use

This adapter must be applied on top of the SFT-merged model, not directly on the base model.

import contextlib
import sys as _sys
import types as _types

# Compatibility shim: EXAONE remote code was written for an older transformers version
import transformers.modeling_rope_utils as _rope_mod
if not hasattr(_rope_mod, "RopeParameters"):
    _rope_mod.RopeParameters = type("RopeParameters", (), {})

import transformers.integrations as _intgr
for _fn_name in ("use_kernel_func_from_hub", "use_kernelized_func"):
    if not hasattr(_intgr, _fn_name):
        setattr(_intgr, _fn_name, lambda *a, **kw: (lambda f: f))

import transformers.utils.generic as _gen
if not hasattr(_gen, "maybe_autocast"):
    @contextlib.contextmanager
    def _noop_autocast(*a, **kw): yield
    _gen.maybe_autocast = _noop_autocast
if hasattr(_gen, "check_model_inputs"):
    _gen.check_model_inputs = lambda f: f

import transformers.utils.auto_docstring as _
_auto_doc_mod = _sys.modules.get("transformers.utils.auto_docstring")
if _auto_doc_mod is not None and hasattr(_auto_doc_mod, "_process_parameter_type"):
    _orig_proc = _auto_doc_mod._process_parameter_type
    def _safe_proc(param, param_name, func):
        try:
            return _orig_proc(param, param_name, func)
        except AttributeError:
            return "", True
    _auto_doc_mod._process_parameter_type = _safe_proc

import transformers.modeling_utils as _mu_utils
if hasattr(_mu_utils, "AttentionInterface") and not hasattr(_mu_utils.AttentionInterface, "get_interface"):
    def _get_interface(self, attn_impl, default_fn):
        return self._global_mapping.get(attn_impl, default_fn)
    _mu_utils.AttentionInterface.get_interface = _get_interface

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, AutoConfig

# 1. Load base model
config = AutoConfig.from_pretrained("LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct", trust_remote_code=True)
if getattr(config, "rope_parameters", None) is None:
    config.rope_parameters = {
        "rope_type": "default",
        "rope_theta": getattr(config, "rope_theta", 10000.0),
    }

model = AutoModelForCausalLM.from_pretrained(
    "LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct",
    config=config,
    dtype="auto",
    trust_remote_code=True,
)
model.get_input_embeddings = _types.MethodType(lambda self: self.transformer.wte, model)
model.set_input_embeddings = _types.MethodType(lambda self, v: setattr(self.transformer, "wte", v), model)

tokenizer = AutoTokenizer.from_pretrained(
    "LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct",
    trust_remote_code=True,
)

# 2. Apply and merge SFT LoRA
model = PeftModel.from_pretrained(
    model,
    "radishtiger/EXAONE-3.5-7.8B-Instruct_convfinqa_lora_r64_DR1.0",
)
model = model.merge_and_unload()

# 3. Apply GRPO LoRA (this adapter)
model = PeftModel.from_pretrained(
    model,
    "xxccho/EXAONE-3.5-7.8B-Instruct-ConvFinQA-SFT-GRPO",
)

Training Details

• Base model: LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct
• SFT data: ConvFinQA (DR 1.0)
• GRPO data: ConvFinQA (ratio 1.0)
• LoRA rank: 64, alpha: 128, dropout: 0.05
• Reward: exact match (weight 1.0) + format reward (weight 0.1)
• Beta: 0.04
• Learning rate: 7e-6
• Precision: bf16

Framework versions

• PEFT 0.18.1