Root-fix Qwen3 Unsloth notebook with clean install and current SFTConfig

EthicalHacking_Qwen3-4B_Ultimate_Colab.ipynb

@@ -1,548 +1,26 @@
 {
  "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# \ud83d\udd10 Ultimate LLM Fine-Tuning \u2013 Qwen3-4B (Colab Free Tier T4)\n",
-    "\n",
-    "**\ud83e\udd47 Model:** [Qwen3-4B-Instruct-2507](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507) via Unsloth 4-bit  \n",
-    "**\ud83c\udfc6 Why this model?** Highest coding/reasoning scores among sub-10B models (LiveCodeBench 35.1, MMLU-Pro 69.6). Only **3.3 GB** in 4-bit.  \n",
-    "**\ud83d\udcca Datasets:** Your choice \u2014 cybersecurity, general chat, multilingual, coding, or mix them!  \n",
-    "**\u26a1 Framework:** Unsloth + TRL SFTTrainer \u2014 2\u00d7 faster, 70% less VRAM  \n",
-    "\n",
-    "> \u26a0\ufe0f Default is cybersecurity. Pick general-purpose datasets for other domains.\n",
-    "\n",
-    "---\n",
-    "\n",
-    "## \ud83d\udccb T4 VRAM Cheat-Sheet\n",
-    "\n",
-    "| Setting | Value | Why |\n",
-    "|---------|-------|-----|\n",
-    "| `MAX_SEQ_LENGTH` | 4096 | Huge headroom on T4 |\n",
-    "| `LORA_R` | 64 | Higher rank = more capacity |\n",
-    "| `BATCH_SIZE` | 4 | You have ~11GB free VRAM |\n",
-    "| `GRAD_ACCUM` | 2 | Effective batch = 8 |\n",
-    "| `PACKING` | **False** | Avoids Unsloth `num_items_in_batch` bug (stable) |\n",
-    "| `optim` | `adamw_8bit` | Massive VRAM saver |\n",
-    "\n",
-    "If you still hit OOM \u2192 lower `MAX_SEQ_LENGTH` to 3072 or set `use_rslora=True`."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## 1\ufe0f\u20e3 Install Dependencies"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "%%capture\n",
-    "!pip install -q unsloth trl datasets accelerate transformers bitsandbytes huggingface_hub"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## 2\ufe0f\u20e3 (Optional) Login to HuggingFace Hub"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from huggingface_hub import login\n",
-    "# login(token=\"hf_YOUR_TOKEN\")   # \u2190 uncomment and paste your token"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## 3\ufe0f\u20e3 Load Qwen3-4B-Instruct-2507 in 4-bit via Unsloth\n",
-    "\n",
-    "**\u26a0\ufe0f IMPORTANT:** We add `device_map={\"\": torch.cuda.current_device()}` to force the model onto the correct GPU.\n",
-    "Without this, `accelerate` may place the model on CPU and throw a `ValueError` during training on Kaggle/Colab."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from unsloth import FastLanguageModel\n",
-    "import torch\n",
-    "\n",
-    "# ==================== T4-COLAB HYPERPARAMETERS ====================\n",
-    "MAX_SEQ_LENGTH = 4096\n",
-    "LORA_R = 64\n",
-    "LORA_ALPHA = 64\n",
-    "BATCH_SIZE = 4\n",
-    "GRAD_ACCUM = 2\n",
-    "LEARNING_RATE = 2e-4\n",
-    "MAX_STEPS = 4000\n",
-    "WARMUP_STEPS = 200\n",
-    "LOGGING_STEPS = 50\n",
-    "SAVE_STEPS = 500\n",
-    "PACKING = False  # \u2190 MUST be False to avoid Unsloth num_items_in_batch bug\n",
-    "SAMPLE_SIZE = 50000\n",
-    "HUB_MODEL_ID = \"your-username/cyber-qwen3-4b-lora\"\n",
-    "# ==================================================================\n",
-    "\n",
-    "model, tokenizer = FastLanguageModel.from_pretrained(\n",
-    "    model_name=\"unsloth/Qwen3-4B-Instruct-2507-unsloth-bnb-4bit\",\n",
-    "    max_seq_length=MAX_SEQ_LENGTH,\n",
-    "    dtype=None,\n",
-    "    load_in_4bit=True,\n",
-    "    device_map={\"\": torch.cuda.current_device()},  # \u2190 FORCE GPU: fixes Kaggle/Colab device placement bug\n",
-    ")\n",
-    "\n",
-    "model = FastLanguageModel.get_peft_model(\n",
-    "    model,\n",
-    "    r=LORA_R,\n",
-    "    target_modules=[\"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n",
-    "                   \"gate_proj\", \"up_proj\", \"down_proj\"],\n",
-    "    lora_alpha=LORA_ALPHA,\n",
-    "    lora_dropout=0,\n",
-    "    bias=\"none\",\n",
-    "    use_gradient_checkpointing=\"unsloth\",\n",
-    "    random_state=3407,\n",
-    "    use_rslora=False,\n",
-    "    loftq_config=None,\n",
-    ")\n",
-    "\n",
-    "trainable = sum(p.numel() for p in model.parameters() if p.requires_grad)\n",
-    "total     = sum(p.numel() for p in model.parameters())\n",
-    "print(f\"\u2705 Qwen3-4B loaded. Trainable params: {trainable:,} / {total:,} ({100*trainable/total:.2f}%)\")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## 4\ufe0f\u20e3 \ud83c\udfaf CHOOSE YOUR DATASET(S)\n",
-    "\n",
-    "Uncomment **ONE** `DATASET_CHOICE` line to select your training data.\n",
-    "\n",
-    "| Choice | Dataset | Rows | Format | Best For |\n",
-    "|--------|---------|------|--------|----------|\n",
-    "| `\"cybersecurity\"` | Fenrir v2.1 + Trendyol | 153K\u219250K | system/user/assistant | Ethical hacking education |\n",
-    "| `\"ultrachat\"` | UltraChat 200K SFT | 200K\u219250K | messages (user/assistant) | General conversation |\n",
-    "| `\"openhermes\"` | OpenHermes 2.5 | 1M+\u219250K | conversations (human/gpt) | Reasoning, coding |\n",
-    "| `\"sharegpt_en\"` | ShareGPT English | ~90K\u219250K | conversations (human/gpt) | Multi-turn dialogue |\n",
-    "| `\"sharegpt_de\"` | ShareGPT German | ~104K\u219250K | conversations (human/gpt) | German fine-tuning |\n",
-    "| `\"sharegpt_hi\"` | ShareGPT Hindi | ~153K\u219250K | conversations (human/gpt) | Hindi fine-tuning |\n",
-    "| `\"code_corpus\"` | [Code Corpus LLM Training](https://huggingface.co/datasets/krystv/code-corpus-llm-training) | 240K\u219250K | text (code files) | **Code completion, coding assistant** |\n",
-    "| `\"custom_mix\"` | Mix of your choice | \u2014 | varies | Combine datasets |"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from datasets import load_dataset, concatenate_datasets\n",
-    "\n",
-    "# \u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\n",
-    "#   SELECT YOUR DATASET \u2014 UNCOMMENT ONE LINE\n",
-    "# \u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\n",
-    "\n",
-    "DATASET_CHOICE = \"cybersecurity\"\n",
-    "\n",
-    "# DATASET_CHOICE = \"ultrachat\"\n",
-    "# DATASET_CHOICE = \"openhermes\"\n",
-    "# DATASET_CHOICE = \"sharegpt_en\"\n",
-    "# DATASET_CHOICE = \"sharegpt_de\"\n",
-    "# DATASET_CHOICE = \"sharegpt_hi\"\n",
-    "# DATASET_CHOICE = \"code_corpus\"\n",
-    "# DATASET_CHOICE = \"custom_mix\"\n",
-    "\n",
-    "CUSTOM_DATASETS = [\n",
-    "    # (\"dataset_name_or_id\", \"split\", rows_to_take, \"format_type\")\n",
-    "    # format_type: \"messages\" | \"conversations\" | \"text\"\n",
-    "    (\"AlicanKiraz0/Cybersecurity-Dataset-Fenrir-v2.1\", \"train\", 10000, \"messages\"),\n",
-    "    (\"HuggingFaceH4/ultrachat_200k\", \"train_sft\", 20000, \"messages\"),\n",
-    "    (\"teknium/OpenHermes-2.5\", \"train\", 20000, \"conversations\"),\n",
-    "]\n",
-    "\n",
-    "print(f\"\ud83c\udfaf DATASET_CHOICE = {DATASET_CHOICE}\")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## 5\ufe0f\u20e3 Load, Convert & Pre-process Selected Dataset\n",
-    "\n",
-    "Auto-detects dataset format and converts everything to standard `messages` \u2192 `text`."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "import random\n",
-    "\n",
-    "def _convert_fenrir(example):\n",
-    "    return {\"messages\": [\n",
-    "        {\"role\": \"system\",    \"content\": example[\"system\"]},\n",
-    "        {\"role\": \"user\",      \"content\": example[\"user\"]},\n",
-    "        {\"role\": \"assistant\", \"content\": example[\"assistant\"]},\n",
-    "    ]}\n",
-    "\n",
-    "def _convert_trendyol(example):\n",
-    "    return {\"messages\": [\n",
-    "        {\"role\": \"system\",    \"content\": example[\"system\"]},\n",
-    "        {\"role\": \"user\",      \"content\": example[\"user\"]},\n",
-    "        {\"role\": \"assistant\", \"content\": example[\"assistant\"]},\n",
-    "    ]}\n",
-    "\n",
-    "def _convert_ultrachat(example):\n",
-    "    return {\"messages\": example[\"messages\"]}\n",
-    "\n",
-    "def _convert_conversations(example):\n",
-    "    msgs = []\n",
-    "    system_prompt = example.get(\"system_prompt\", \"\") or example.get(\"system\", \"\")\n",
-    "    if system_prompt:\n",
-    "        msgs.append({\"role\": \"system\", \"content\": system_prompt})\n",
-    "    for turn in example[\"conversations\"]:\n",
-    "        role = \"user\" if turn[\"from\"] in (\"human\", \"user\") else \"assistant\"\n",
-    "        msgs.append({\"role\": role, \"content\": turn[\"value\"]})\n",
-    "    return {\"messages\": msgs}\n",
-    "\n",
-    "def _convert_code_corpus(example):\n",
-    "    code_text = example[\"text\"]\n",
-    "    domain = example.get(\"domain\", \"code\")\n",
-    "    repo = example.get(\"repo\", \"unknown\")\n",
-    "    lang = example.get(\"language\", \"\")\n",
-    "    user_prompt = f\"Here is a code snippet from the {domain} domain (repo: {repo}, language: {lang}). Please explain or improve it.\"\n",
-    "    return {\"messages\": [\n",
-    "        {\"role\": \"user\",      \"content\": user_prompt},\n",
-    "        {\"role\": \"assistant\", \"content\": code_text},\n",
-    "    ]}\n",
-    "\n",
-    "all_datasets = []\n",
-    "\n",
-    "if DATASET_CHOICE == \"cybersecurity\":\n",
-    "    ds1 = load_dataset(\"AlicanKiraz0/Cybersecurity-Dataset-Fenrir-v2.1\", split=\"train\")\n",
-    "    ds1 = ds1.map(_convert_fenrir, remove_columns=ds1.column_names, batched=False)\n",
-    "    all_datasets.append(ds1)\n",
-    "    ds2 = load_dataset(\"Trendyol/Trendyol-Cybersecurity-Instruction-Tuning-Dataset\", split=\"train\")\n",
-    "    ds2 = ds2.map(_convert_trendyol, remove_columns=ds2.column_names, batched=False)\n",
-    "    all_datasets.append(ds2)\n",
-    "\n",
-    "elif DATASET_CHOICE == \"ultrachat\":\n",
-    "    ds = load_dataset(\"HuggingFaceH4/ultrachat_200k\", split=\"train_sft\")\n",
-    "    ds = ds.map(_convert_ultrachat, remove_columns=ds.column_names, batched=False)\n",
-    "    all_datasets.append(ds)\n",
-    "\n",
-    "elif DATASET_CHOICE == \"openhermes\":\n",
-    "    ds = load_dataset(\"teknium/OpenHermes-2.5\", split=\"train\")\n",
-    "    ds = ds.map(_convert_conversations, remove_columns=ds.column_names, batched=False)\n",
-    "    all_datasets.append(ds)\n",
-    "\n",
-    "elif DATASET_CHOICE.startswith(\"sharegpt_\"):\n",
-    "    split_map = {\"sharegpt_en\": \"english\", \"sharegpt_de\": \"german_4b_translated\", \"sharegpt_hi\": \"hindi_27b_translated\"}\n",
-    "    ds = load_dataset(\"deepmage121/ShareGPT_multilingual\", split=split_map[DATASET_CHOICE])\n",
-    "    ds = ds.map(_convert_conversations, remove_columns=ds.column_names, batched=False)\n",
-    "    all_datasets.append(ds)\n",
-    "\n",
-    "elif DATASET_CHOICE == \"code_corpus\":\n",
-    "    print(\"\ud83d\udce5 Loading Code Corpus LLM Training (krystv)...\")\n",
-    "    ds = load_dataset(\"krystv/code-corpus-llm-training\", split=\"train\")\n",
-    "    ds = ds.map(_convert_code_corpus, remove_columns=ds.column_names, batched=False)\n",
-    "    all_datasets.append(ds)\n",
-    "\n",
-    "elif DATASET_CHOICE == \"custom_mix\":\n",
-    "    for ds_id, split, n_rows, fmt in CUSTOM_DATASETS:\n",
-    "        print(f\"\ud83d\udce5 Loading {ds_id} ({split}, {n_rows} rows)...\")\n",
-    "        ds = load_dataset(ds_id, split=split)\n",
-    "        if n_rows and len(ds) > n_rows:\n",
-    "            ds = ds.shuffle(seed=3407).select(range(n_rows))\n",
-    "        if fmt == \"messages\":\n",
-    "            ds = ds.map(_convert_ultrachat, remove_columns=ds.column_names, batched=False)\n",
-    "        elif fmt == \"conversations\":\n",
-    "            ds = ds.map(_convert_conversations, remove_columns=ds.column_names, batched=False)\n",
-    "        elif fmt == \"text\":\n",
-    "            ds = ds.map(_convert_code_corpus, remove_columns=ds.column_names, batched=False)\n",
-    "        else:\n",
-    "            raise ValueError(f\"Unknown format: {fmt}\")\n",
-    "        all_datasets.append(ds)\n",
-    "\n",
-    "else:\n",
-    "    raise ValueError(f\"Unknown DATASET_CHOICE: {DATASET_CHOICE}\")\n",
-    "\n",
-    "train_dataset = concatenate_datasets(all_datasets) if len(all_datasets) > 1 else all_datasets[0]\n",
-    "print(f\"\\n\ud83d\udcca COMBINED DATASET: {len(train_dataset)} rows\")\n",
-    "\n",
-    "sample = train_dataset[random.randint(0, len(train_dataset)-1)]\n",
-    "print(f\"Sample roles: {[m['role'] for m in sample['messages']]}\")\n",
-    "for m in sample[\"messages\"]: print(f\"  {m['role']}: {m['content'][:80]}...\")\n",
-    "\n",
-    "if len(train_dataset) > SAMPLE_SIZE:\n",
-    "    train_dataset = train_dataset.shuffle(seed=3407).select(range(SAMPLE_SIZE))\n",
-    "    print(f\"\\n\ud83d\ude80 SUBSAMPLED to {len(train_dataset)} rows\")\n",
-    "\n",
-    "print(f\"   Effective batch size: {BATCH_SIZE * GRAD_ACCUM}\")\n",
-    "print(f\"   Steps per epoch: ~{len(train_dataset) // (BATCH_SIZE * GRAD_ACCUM)}\")\n",
-    "print(f\"   Capped to MAX_STEPS: {MAX_STEPS}\")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## 6\ufe0f\u20e3 Convert Messages \u2192 Text (Chat Template)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "def convert_messages_to_text(examples):\n",
-    "    texts = []\n",
-    "    for msgs in examples[\"messages\"]:\n",
-    "        text = tokenizer.apply_chat_template(msgs, tokenize=False, add_generation_prompt=False)\n",
-    "        texts.append(text)\n",
-    "    return {\"text\": texts}\n",
-    "\n",
-    "print(\"\ud83d\udd04 Converting messages to text...\")\n",
-    "train_dataset = train_dataset.map(convert_messages_to_text, batched=True, remove_columns=[\"messages\"], batch_size=100)\n",
-    "print(f\"\u2705 Dataset pre-processed. Columns: {train_dataset.column_names}\")\n",
-    "print(f\"\ud83d\udcc4 Sample text length: {len(train_dataset[0]['text'])} chars\")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## 7\ufe0f\u20e3 Configure SFT Trainer"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from trl import SFTTrainer\n",
-    "from transformers import TrainingArguments\n",
-    "\n",
-    "trainer = SFTTrainer(\n",
-    "    model=model,\n",
-    "    tokenizer=tokenizer,\n",
-    "    train_dataset=train_dataset,\n",
-    "    dataset_text_field=\"text\",\n",
-    "    max_seq_length=MAX_SEQ_LENGTH,\n",
-    "    dataset_num_proc=2,\n",
-    "    packing=PACKING,\n",
-    "    args=TrainingArguments(\n",
-    "        per_device_train_batch_size=BATCH_SIZE,\n",
-    "        gradient_accumulation_steps=GRAD_ACCUM,\n",
-    "        warmup_steps=WARMUP_STEPS,\n",
-    "        max_steps=MAX_STEPS,\n",
-    "        learning_rate=LEARNING_RATE,\n",
-    "        fp16=True,\n",
-    "        logging_steps=LOGGING_STEPS,\n",
-    "        optim=\"adamw_8bit\",\n",
-    "        weight_decay=0.01,\n",
-    "        lr_scheduler_type=\"linear\",\n",
-    "        seed=3407,\n",
-    "        output_dir=\"./outputs\",\n",
-    "        save_strategy=\"steps\",\n",
-    "        save_steps=SAVE_STEPS,\n",
-    "        save_total_limit=2,\n",
-    "        report_to=\"none\",\n",
-    "    ),\n",
-    ")\n",
-    "\n",
-    "print(f\"\u2705 Trainer ready. Dataset: {DATASET_CHOICE} | Steps: {MAX_STEPS}\")\n",
-    "print(f\"   Effective batch size: {BATCH_SIZE * GRAD_ACCUM}\")\n",
-    "print(f\"   Packing enabled: {PACKING}\")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## 8\ufe0f\u20e3 Train \ud83d\ude80"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# \u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\n",
-    "#  RUNTIME PATCH: Fix num_items_in_batch int bug\n",
-    "#  Must run AFTER trainer = SFTTrainer(...) \n",
-    "#  and BEFORE trainer.train()\n",
-    "# \u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\n",
-    "import types\n",
-    "\n",
-    "# Walk the trainer's class hierarchy and patch every training_step\n",
-    "trainer_cls = type(trainer)\n",
-    "\n",
-    "for cls in trainer_cls.__mro__:\n",
-    "    if 'training_step' in cls.__dict__:\n",
-    "        _orig_step = cls.__dict__['training_step']\n",
-    "        \n",
-    "        def _make_safe_step(original):\n",
-    "            def _safe_training_step(self, model, inputs, num_items_in_batch=None, **kwargs):\n",
-    "                if isinstance(num_items_in_batch, int):\n",
-    "                    num_items_in_batch = None\n",
-    "                return original(self, model, inputs, num_items_in_batch=num_items_in_batch, **kwargs)\n",
-    "            return _safe_training_step\n",
-    "        \n",
-    "        cls.training_step = _make_safe_step(_orig_step)\n",
-    "        print(f\"Patched {cls.__name__}.training_step\")\n",
-    "\n",
-    "print(\"Ready to run trainer.train()\")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "if torch.cuda.is_available():\n",
-    "    print(f\"VRAM before train: {torch.cuda.memory_allocated()/1e9:.2f} GB / {torch.cuda.get_device_properties(0).total_memory/1e9:.2f} GB\")\n",
-    "\n",
-    "trainer_stats = trainer.train()\n",
-    "\n",
-    "print(\"\\n\ud83c\udf89 Training complete!\")\n",
-    "print(trainer_stats)\n",
-    "\n",
-    "if torch.cuda.is_available():\n",
-    "    print(f\"VRAM after train:  {torch.cuda.memory_allocated()/1e9:.2f} GB\")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## 9\ufe0f\u20e3 Save & Push to HuggingFace Hub"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "model.save_pretrained(\"./lora-adapter\")\n",
-    "tokenizer.save_pretrained(\"./lora-adapter\")\n",
-    "print(\"\u2705 LoRA adapter saved\")\n",
-    "\n",
-    "print(\"\\n\ud83d\udd04 Merging LoRA into base model...\")\n",
-    "merged_model = model.merge_and_unload()\n",
-    "merged_model.save_pretrained(\"./merged-model\")\n",
-    "tokenizer.save_pretrained(\"./merged-model\")\n",
-    "print(\"\u2705 Merged model saved\")\n",
-    "\n",
-    "# model.push_to_hub(HUB_MODEL_ID)\n",
-    "# tokenizer.push_to_hub(HUB_MODEL_ID)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## \ud83d\udd1f Inference Demo \u2013 Qwen3 Thinking Toggle\n",
-    "\n",
-    "| Mode | Use Case | Speed |\n",
-    "|------|----------|-------|\n",
-    "| `enable_thinking=True` | Deep reasoning, analysis | Slower, thorough |\n",
-    "| `enable_thinking=False` | Quick answers, coding | Fast, direct |"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "FastLanguageModel.for_inference(model)\n",
-    "\n",
-    "test_prompt = \"Explain how parameterized queries prevent SQL injection, with a Python example.\"\n",
-    "\n",
-    "messages = [\n",
-    "    {\"role\": \"system\", \"content\": \"You are a helpful and knowledgeable assistant.\"},\n",
-    "    {\"role\": \"user\",     \"content\": test_prompt},\n",
-    "]\n",
-    "\n",
-    "for think_mode in [True, False]:\n",
-    "    label = \"\ud83e\udde0 THINKING=ON\" if think_mode else \"\u26a1 THINKING=OFF\"\n",
-    "    print(f\"\\n{'='*60}\")\n",
-    "    print(label)\n",
-    "    print(f\"{'='*60}\")\n",
-    "\n",
-    "    inputs = tokenizer.apply_chat_template(\n",
-    "        messages, tokenize=True, add_generation_prompt=True,\n",
-    "        enable_thinking=think_mode, return_tensors=\"pt\",\n",
-    "    ).to(model.device)\n",
-    "\n",
-    "    outputs = model.generate(\n",
-    "        input_ids=inputs, max_new_tokens=512, temperature=0.7,\n",
-    "        top_p=0.9, do_sample=True,\n",
-    "        pad_token_id=tokenizer.pad_token_id,\n",
-    "        eos_token_id=tokenizer.eos_token_id,\n",
-    "    )\n",
-    "    reply = tokenizer.decode(outputs[0], skip_special_tokens=True)\n",
-    "    print(reply.split(\"assistant\")[-1].strip()[:800])\n",
-    "    print(f\"\\n[Tokens: {len(outputs[0]) - len(inputs[0])}]\")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "---\n",
-    "## \ud83d\udcda Dataset & Model References\n",
-    "\n",
-    "| Resource | Link |\n",
-    "|----------|------|\n",
-    "| **Qwen3-4B-Instruct-2507** | https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507 |\n",
-    "| **UltraChat 200K** | https://huggingface.co/datasets/HuggingFaceH4/ultrachat_200k |\n",
-    "| **OpenHermes 2.5** | https://huggingface.co/datasets/teknium/OpenHermes-2.5 |\n",
-    "| **ShareGPT Multilingual** | https://huggingface.co/datasets/deepmage121/ShareGPT_multilingual |\n",
-    "| **Code Corpus LLM Training** | https://huggingface.co/datasets/krystv/code-corpus-llm-training |\n",
-    "| **Fenrir Cybersecurity** | https://huggingface.co/datasets/AlicanKiraz0/Cybersecurity-Dataset-Fenrir-v2.1 |\n",
-    "| **Trendyol Cybersecurity** | https://huggingface.co/datasets/Trendyol/Trendyol-Cybersecurity-Instruction-Tuning-Dataset |\n",
-    "| **Unsloth Docs** | https://unsloth.ai/docs |\n",
-    "\n",
-    "---\n",
-    "*Pick any dataset. Train anything. Use responsibly.*"
-   ]
-  }
+  {"cell_type":"markdown","metadata":{},"source":["# 🔐 Qwen3-4B Unsloth QLoRA — Root-Fixed Kaggle/Colab Notebook\n","\n","Keeps **Unsloth** for low VRAM, but fixes the repeated `AttributeError: 'int' object has no attribute 'mean'` from the root.\n","\n","Root fixes:\n","1. Deletes `/kaggle/working/unsloth_compiled_cache`.\n","2. Updates `unsloth` + `unsloth_zoo` together.\n","3. Forces one kernel restart after install.\n","4. Uses current TRL `SFTConfig` + `processing_class=tokenizer`.\n","5. Sets `trainer.model_accepts_loss_kwargs = False`.\n","\n","> Qwen3-4B is stronger than LFM2.5 but tighter on T4. Defaults are conservative.\n"]},
+  {"cell_type":"markdown","metadata":{},"source":["## 1. Clean install Unsloth stack — run first\n"]},
+  {"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["import os, sys, shutil, subprocess, pathlib\n","work_dir = pathlib.Path('/kaggle/working') if pathlib.Path('/kaggle/working').exists() else pathlib.Path('/content')\n","marker = work_dir / '.bex_unsloth_env_ready_v3'\n","shutil.rmtree(str(work_dir / 'unsloth_compiled_cache'), ignore_errors=True)\n","print('✅ Removed stale unsloth_compiled_cache')\n","if not marker.exists():\n","    print('Installing/updating Unsloth stack. Kernel will restart after this cell. Run all cells again after restart.')\n","    subprocess.check_call([sys.executable, '-m', 'pip', 'uninstall', '-y', 'unsloth', 'unsloth_zoo'])\n","    subprocess.check_call([sys.executable, '-m', 'pip', 'install', '-U', '--no-cache-dir', 'unsloth', 'unsloth_zoo'])\n","    marker.write_text('ready')\n","    print('✅ Installed Unsloth. Restarting kernel now...')\n","    os.kill(os.getpid(), 9)\n","else:\n","    print('✅ Unsloth stack already prepared for this session')\n"]},
+  {"cell_type":"markdown","metadata":{},"source":["## 2. Optional HF login\n"]},
+  {"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["from huggingface_hub import login\n","# login(token='hf_YOUR_WRITE_TOKEN')\n"]},
+  {"cell_type":"markdown","metadata":{},"source":["## 3. Imports and version check\n"]},
+  {"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["from unsloth import FastLanguageModel, is_bfloat16_supported\n","from trl import SFTTrainer, SFTConfig\n","from datasets import load_dataset, concatenate_datasets\n","import torch, random, os, time\n","import transformers, trl, peft, accelerate\n","try:\n","    import unsloth\n","    print('unsloth', getattr(unsloth, '__version__', 'unknown'))\n","except Exception as e:\n","    print('unsloth version unavailable', e)\n","print('transformers', transformers.__version__)\n","print('trl', trl.__version__)\n","print('peft', peft.__version__)\n","print('accelerate', accelerate.__version__)\n","if torch.cuda.is_available():\n","    print('GPU:', torch.cuda.get_device_name(0))\n","    print(f'VRAM: {torch.cuda.get_device_properties(0).total_memory/1e9:.2f} GB')\n","else:\n","    print('⚠️ No GPU found. Enable GPU runtime.')\n"]},
+  {"cell_type":"markdown","metadata":{},"source":["## 4. Configuration\n"]},
+  {"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["MODEL_ID = 'unsloth/Qwen3-4B-Instruct-2507-unsloth-bnb-4bit'\n","RUN_NAME = 'qwen3-4b-cyber-unsloth-rootfixed'\n","DATASET_CHOICE = 'cybersecurity'  # cybersecurity | ultrachat | openhermes | code_corpus\n","SAMPLE_SIZE = 30000\n","MAX_SEQ_LENGTH = 2048\n","LORA_R = 32\n","LORA_ALPHA = 64\n","BATCH_SIZE = 1\n","GRAD_ACCUM = 8\n","MAX_STEPS = 1500\n","LEARNING_RATE = 1.5e-4\n","WARMUP_STEPS = 100\n","SAVE_STEPS = 500\n","LOGGING_STEPS = 10\n","PACKING = False\n","SEED = 3407\n","OUTPUT_DIR = './outputs_qwen3_rootfixed'\n","HUB_MODEL_ID = 'your-username/qwen3-4b-cyber-unsloth-rootfixed'\n","PUSH_TO_HUB = False\n","random.seed(SEED)\n","torch.manual_seed(SEED)\n","if torch.cuda.is_available(): torch.cuda.manual_seed_all(SEED)\n"]},
+  {"cell_type":"markdown","metadata":{},"source":["## 5. Load model with Unsloth 4-bit QLoRA\n"]},
+  {"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["model, tokenizer = FastLanguageModel.from_pretrained(\n","    model_name=MODEL_ID,\n","    max_seq_length=MAX_SEQ_LENGTH,\n","    dtype=None,\n","    load_in_4bit=True,\n","    device_map={'': torch.cuda.current_device()} if torch.cuda.is_available() else None,\n",")\n","if tokenizer.pad_token is None:\n","    tokenizer.pad_token = tokenizer.eos_token\n","tokenizer.padding_side = 'right'\n","model = FastLanguageModel.get_peft_model(\n","    model,\n","    r=LORA_R,\n","    target_modules=['q_proj','k_proj','v_proj','o_proj','gate_proj','up_proj','down_proj'],\n","    lora_alpha=LORA_ALPHA,\n","    lora_dropout=0,\n","    bias='none',\n","    use_gradient_checkpointing='unsloth',\n","    random_state=SEED,\n","    use_rslora=True,\n","    loftq_config=None,\n",")\n","trainable = sum(p.numel() for p in model.parameters() if p.requires_grad)\n","total = sum(p.numel() for p in model.parameters())\n","print(f'Trainable params: {trainable:,} / {total:,} ({100*trainable/total:.2f}%)')\n","if torch.cuda.is_available():\n","    print(f'VRAM after model load: {torch.cuda.memory_allocated()/1e9:.2f} GB')\n"]},
+  {"cell_type":"markdown","metadata":{},"source":["## 6. Load and format dataset\n"]},
+  {"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["SYSTEM_SAFE = 'You are a cybersecurity education assistant. Provide defensive, ethical, and authorized security guidance only. Refuse harmful or unauthorized requests.'\n","def convert_sua(example):\n","    return {'messages': [{'role':'system','content':example.get('system') or SYSTEM_SAFE},{'role':'user','content':example['user']},{'role':'assistant','content':example['assistant']}]}\n","def convert_ultrachat(example): return {'messages': example['messages']}\n","def convert_conversations(example):\n","    msgs=[]\n","    sys_prompt=example.get('system_prompt','') or example.get('system','')\n","    if sys_prompt: msgs.append({'role':'system','content':sys_prompt})\n","    for turn in example['conversations']:\n","        role='user' if turn.get('from') in ('human','user') else 'assistant'\n","        msgs.append({'role':role,'content':turn.get('value','')})\n","    return {'messages':msgs}\n","def convert_code(example):\n","    return {'messages':[{'role':'user','content':'Explain and improve this code with attention to safety and correctness.'},{'role':'assistant','content':example['text']}]}\n","if DATASET_CHOICE == 'cybersecurity':\n","    ds1=load_dataset('AlicanKiraz0/Cybersecurity-Dataset-Fenrir-v2.1', split='train').map(convert_sua, remove_columns=['system','user','assistant'])\n","    ds2=load_dataset('Trendyol/Trendyol-Cybersecurity-Instruction-Tuning-Dataset', split='train').map(convert_sua, remove_columns=['system','user','assistant'])\n","    dataset=concatenate_datasets([ds1,ds2])\n","elif DATASET_CHOICE == 'ultrachat':\n","    dataset=load_dataset('HuggingFaceH4/ultrachat_200k', split='train_sft')\n","    dataset=dataset.map(convert_ultrachat, remove_columns=dataset.column_names)\n","elif DATASET_CHOICE == 'openhermes':\n","    dataset=load_dataset('teknium/OpenHermes-2.5', split='train')\n","    dataset=dataset.map(convert_conversations, remove_columns=dataset.column_names)\n","elif DATASET_CHOICE == 'code_corpus':\n","    dataset=load_dataset('krystv/code-corpus-llm-training', split='train')\n","    dataset=dataset.map(convert_code, remove_columns=dataset.column_names)\n","else: raise ValueError(DATASET_CHOICE)\n","print('Rows:', len(dataset))\n","if SAMPLE_SIZE and len(dataset)>SAMPLE_SIZE:\n","    dataset=dataset.shuffle(seed=SEED).select(range(SAMPLE_SIZE))\n","    print('Subsampled:', len(dataset))\n","print(dataset[0]['messages'])\n"]},
+  {"cell_type":"markdown","metadata":{},"source":["## 7. Convert messages to text\n"]},
+  {"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["def to_text(example):\n","    try:\n","        text=tokenizer.apply_chat_template(example['messages'], tokenize=False, add_generation_prompt=False)\n","    except Exception:\n","        text='\\n'.join([f\"<{m['role']}>\\n{m['content']}\\n</{m['role']}>\" for m in example['messages']]) + tokenizer.eos_token\n","    return {'text': text}\n","dataset=dataset.map(to_text, remove_columns=['messages'])\n","print(dataset[0]['text'][:500])\n"]},
+  {"cell_type":"markdown","metadata":{},"source":["## 8. Train with current SFTConfig API\n"]},
+  {"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["args = SFTConfig(\n","    output_dir=OUTPUT_DIR, dataset_text_field='text', max_length=MAX_SEQ_LENGTH, packing=PACKING,\n","    per_device_train_batch_size=BATCH_SIZE, gradient_accumulation_steps=GRAD_ACCUM,\n","    warmup_steps=WARMUP_STEPS, max_steps=MAX_STEPS, learning_rate=LEARNING_RATE,\n","    fp16=not is_bfloat16_supported(), bf16=is_bfloat16_supported(), logging_steps=LOGGING_STEPS,\n","    optim='adamw_8bit', weight_decay=0.01, lr_scheduler_type='linear', seed=SEED,\n","    save_strategy='steps', save_steps=SAVE_STEPS, save_total_limit=2, report_to='none',\n","    disable_tqdm=True, logging_first_step=True,\n",")\n","trainer = SFTTrainer(model=model, processing_class=tokenizer, train_dataset=dataset, args=args)\n","trainer.model_accepts_loss_kwargs = False\n","if hasattr(trainer, 'model'): trainer.model.config.use_cache = False\n","if torch.cuda.is_available(): print(f'VRAM before train: {torch.cuda.memory_allocated()/1e9:.2f} GB / {torch.cuda.get_device_properties(0).total_memory/1e9:.2f} GB')\n","trainer_stats = trainer.train()\n","print(trainer_stats)\n"]},
+  {"cell_type":"markdown","metadata":{},"source":["## 9. Save adapter and test\n"]},
+  {"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["model.save_pretrained('./qwen3-lora-adapter')\n","tokenizer.save_pretrained('./qwen3-lora-adapter')\n","print('Saved adapter to ./qwen3-lora-adapter')\n","if PUSH_TO_HUB:\n","    model.push_to_hub(HUB_MODEL_ID); tokenizer.push_to_hub(HUB_MODEL_ID)\n","FastLanguageModel.for_inference(model)\n","messages=[{'role':'system','content':SYSTEM_SAFE},{'role':'user','content':'Explain parameterized queries for preventing SQL injection with safe Python.'}]\n","inputs=tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors='pt').to(model.device)\n","with torch.no_grad():\n","    outputs=model.generate(input_ids=inputs, max_new_tokens=384, temperature=0.7, top_p=0.9, do_sample=True, pad_token_id=tokenizer.pad_token_id, eos_token_id=tokenizer.eos_token_id)\n","print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))\n"]}
  ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "name": "python",
-   "version": "3.10.12"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 4
-}
+ "metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.10"}},
+ "nbformat":4,
+ "nbformat_minor":5
+}