kuririrn/qwen3-4b-agent-trajectory-SFT_alfadm-prmcons_alformat3

AgentBench LoRA Adapter (Qwen/Qwen3-4B-Instruct-2507)

This repository provides a LoRA adapter fine-tuned from Qwen/Qwen3-4B-Instruct-2507 using LoRA + Unsloth.

This repository contains LoRA adapter weights only. The base model must be loaded separately.

Datasets

This adapter is trained on a single trajectory-style SFT dataset:

1. ALFWorld composite admissible-only dataset: kuririrn/sft_alfworld_trajectory_dataset_v3to5_admissible
   • Trajectories collected from multiple ALFWorld SFT sources (v3–v5).
   • Only trajectories with admissible actions text are kept.
   • User prompts include an admissible commands list (normalized from the original "admissible actions" field in ALFWorld) and the assistant outputs THOUGHT: / ACTION: style reasoning traces.

Training Objective

This adapter is trained to improve multi-turn agent task performance on ALFWorld (household tasks).

Loss is applied to all assistant turns in the trajectory, so the model learns environment observation, action selection, tool use, and recovery from errors in ALFWorld.

In addition, user prompts are augmented to:

• Re-state the task goal at each step (e.g., Your goal: ...),
• Provide the previous action (Previous Action: ...),
• Summarize the previous observation/result (Previous Result: ... when applicable),
• And include constraints to avoid repeating the same action and to prevent premature task succeeded outputs before the goal is reached.

Training Configuration

• Base model: Qwen/Qwen3-4B-Instruct-2507
• Method: LoRA (full precision base)
• Max sequence length: 2048
• Epochs: 2
• Learning rate: 2e-06
• LoRA r: 64
• LoRA alpha: 128

Constraint Prompting for Admissible Commands (ALFWorld)

For ALFWorld trajectories, user messages that already contain an admissible actions/commands list are augmented during training with an additional constraint prompt:

[Important Instruction]
You must choose exactly one action from the admissible commands list.
Do not generate any action that is not explicitly included in that list.
The action you output must exactly match one item in the
admissible commands list.

## Intended Use

This adapter is intended to be used as a tool-using agent
for ALFWorld and/or DBBench-style tasks, depending on the
training configuration:

- When trained only on ALFWorld, it focuses on household tasks.
- When trained only on DBBench, it focuses on database operations.
- When trained on both, it aims to balance both domains.

## How to Load

    from transformers import AutoModelForCausalLM, AutoTokenizer
    from peft import PeftModel

    base_model = AutoModelForCausalLM.from_pretrained(
        "Qwen/Qwen3-4B-Instruct-2507",
        torch_dtype="auto",
        device_map="auto",
    )
    tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-4B-Instruct-2507")

    model = PeftModel.from_pretrained(
        base_model,
        "/content/lora_agentbench_qwen3_4b",
    )

    model.eval()

## Limitations and Bias

This adapter is trained on synthetic and task-specific trajectories
for ALFWorld and/or DBBench. It may:

- Overfit to the style of prompts and environments in these datasets.
- Hallucinate actions or SQL queries outside the training distribution.
- Fail on tasks that are very different from ALFWorld/DBBench.

Always validate the model's outputs, especially when using it for
real-world decision making or database operations.