--- license: apache-2.0 language: - en - zh library_name: transformers pipeline_tag: text-generation base_model: Qwen/Qwen3-4B-Instruct-2507 tags: - qwen3 - roleplay - rp - storytelling - fantasy - merged - lora - qlora - transformers --- # ArityFlow-Qwen3-4B-Instruct-2507-RolePlay
**English | 中文** A RolePlay-oriented fine-tuning based on **Qwen3-4B-Instruct-2507** 一个专注于沉浸式角色扮演（RolePlay）的 Qwen3 微调模型
--- # 📖 Overview | 模型简介 ## 🇺🇸 English ArityFlow-Qwen3-4B-Instruct-2507-RolePlay is a RolePlay-focused fine-tuned model based on **Qwen3-4B-Instruct-2507**. Unlike general-purpose instruction tuning, this project focuses on improving: - Character consistency - Emotional expression - Story progression - Long-form dialogue - Fantasy world building - Immersive roleplay The model was trained using **QLoRA (NF4)** with **MS-SWIFT**. This repository contains: - ✅ Merged Full Model (Recommended) - ✅ Original LoRA Adapter (`/lora`) --- ## 🇨🇳 中文 ArityFlow-Qwen3-4B-Instruct-2507-RolePlay 是基于 **Qwen3-4B-Instruct-2507** 微调得到的角色扮演模型。 本项目并非以 Benchmark 为主要目标，而是重点提升： - 人设一致性 - 情绪表达 - 剧情推进 - 长对话能力 - 世界观构建 - 沉浸式角色扮演体验 模型采用 **MS-SWIFT + QLoRA(NF4)** 完成训练。 本仓库同时提供： - ✅ 合并后的完整模型（推荐直接推理） - ✅ 原始 LoRA Adapter（位于 `/lora`） --- # ✨ Features | 模型特点 | Base Qwen3 | ArityFlow RP | |------------|--------------| | Assistant-oriented | RolePlay-oriented | | Conservative dialogue | Immersive dialogue | | Limited world building | Rich world building | | Passive interaction | Dynamic interaction | | Limited NPC generation | Better NPC generation | | General writing | Storytelling focused | --- # 📦 Repository Structure | 仓库结构 ``` . ├── README.md ├── config.json ├── generation_config.json ├── tokenizer.json ├── tokenizer_config.json ├── special_tokens_map.json ├── model.safetensors... │ └── lora/ ├── adapter_model.safetensors ├── adapter_config.json ├── args.json └── ... ``` **Root directory** Merged Full Model **lora/** Original QLoRA Adapter --- # ⚙️ Training Configuration | 训练配置 | Item | Value | |------|------:| | Base Model | Qwen3-4B-Instruct-2507 | | Framework | MS-SWIFT | | Method | QLoRA | | Quantization | NF4 4-bit | | LoRA Rank | 32 | | LoRA Alpha | 64 | | LoRA Dropout | 0.05 | | Target Modules | all-linear | | Max Length | 4096 | | Learning Rate | 5e-5 | | Scheduler | Cosine | | Warmup Ratio | 5% | | Optimizer | AdamW | | Batch Size | 1 | | Gradient Accumulation | 8 | | Effective Batch Size | 8 | | Epoch | 1 | --- # 📚 Dataset | 数据集 The model was trained on a merged ShareGPT-format RolePlay dataset. 训练数据采用 ShareGPT 格式角色扮演数据。 After filtering samples longer than **4096 tokens**: 过滤超过 **4096 Token** 的样本后： | Split | Samples | |------|------:| | Train | **10,511** | | Validation | **549** | --- # 📈 Training Result | 训练结果 Training converged smoothly without obvious overfitting. 训练过程收敛稳定，无明显过拟合。 | Step | Eval Loss | |------:|----------:| | 200 | 1.554 | | 400 | 1.497 | | 600 | 1.467 | | 800 | 1.446 | | 1000 | 1.435 | | **1314** | **1.430** | Final Validation Token Accuracy 最终验证集 Token Accuracy **64.41%** --- # 🔍 Qualitative Evaluation | 主观测试 The model was manually compared against the original Qwen3 model using identical prompts and generation parameters. 在完全相同的 Prompt 与采样参数下，对 Base Qwen3 与微调模型进行了人工对比测试。 Observed improvements: - Better character consistency - Richer action descriptions - Better emotional expression - Better environmental descriptions - Stronger fantasy world building - Better NPC generation - Better long-form roleplay 观察到的提升： - 更稳定的人设保持 - 更丰富的动作描写 - 更自然的情绪表达 - 更好的环境描写 - 更完整的幻想世界构建 - 更自然的 NPC 生成 - 更好的长剧情角色扮演体验 The merged model was compared against the original LoRA adapter and showed no observable degradation during manual testing. 同时对 LoRA Adapter 与合并后的完整模型进行了人工对比，未观察到明显的生成质量下降。 --- # 💬 Example | 示例 ## System Prompt ``` You are Bai Zhi. The librarian of the Imperial Royal Library. Stay in character. Never reveal yourself as an AI. Maintain the fantasy world setting. ``` ## User ``` The library has already closed. Heavy rain is falling outside. I push open the old wooden door and see you repairing an ancient book beside a candle. "So late... why aren't you going home?" ``` --- # 🚀 Usage | 使用方式 ## Transformers ```python from transformers import AutoTokenizer from transformers import AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("YOUR_MODEL") model = AutoModelForCausalLM.from_pretrained( "YOUR_MODEL", torch_dtype="auto", device_map="auto" ) ``` --- ## MS-SWIFT ### Merged Model ```bash swift infer \ --model YOUR_MODEL_PATH ``` ### LoRA Adapter ```bash swift infer \ --model Qwen/Qwen3-4B-Instruct-2507 \ --adapters lora/ ``` --- # 🎯 Recommended Use Cases | 推荐使用场景 Recommended - RolePlay - Character Chat - Interactive Fiction - Fantasy Dialogue - NPC Generation - Storytelling 推荐： - 角色扮演 - 剧情互动 - 长剧情聊天 - 世界观构建 - NPC 对话 - 小说式聊天 --- # ⚠️ Limitations | 已知特点 Compared with the original Qwen3 model, this model intentionally produces: - Longer responses - Richer descriptions - Stronger emotions - More proactive story progression This behavior is expected and is part of the design objective. 相较于基础模型，本模型会： - 回复更长 - 动作描写更多 - 环境描写更多 - 情绪表达更丰富 - 更倾向主动推进剧情 这是本项目有意优化的方向，并非异常行为。 --- # 🙏 Acknowledgements | 致谢 This project is built upon the following open-source projects: - Alibaba Qwen Team - MS-SWIFT - Hugging Face Transformers - PEFT - ModelScope Special thanks to the open-source community. 本项目基于以下优秀开源项目完成： - Alibaba Qwen Team - MS-SWIFT - Hugging Face Transformers - PEFT - ModelScope 感谢所有开源贡献者。 --- # 📄 License | 许可证 This model follows the license of the original **Qwen3-4B-Instruct-2507**. Please refer to the original license before commercial use. 本模型遵循 **Qwen3-4B-Instruct-2507** 的许可证。 商业使用前请阅读原始模型许可证。