Upload Choonsik Stage 3 checkpoint

README.md

@@ -1,58 +1,84 @@
 ---
+license: mit
 base_model: Qwen/Qwen3.5-9B
-library_name: transformers
-model_name: Choonsik-Qwen3.5-9B
 tags:
-- generated_from_trainer
-- sft
-- trl
-licence: license
+  - choonsik
+  - VLA
+  - Minecraft
+  - vision-language-action
+  - qwen3.5
+  - image-text-to-text
+datasets:
+  - CraftJarvis/minecraft-vla-sft
+library_name: transformers
+language:
+  - en
+pipeline_tag: image-text-to-text
 ---
 
-# Model Card for Choonsik-Qwen3.5-9B
+# Choonsik — Minecraft Vision-Language-Action Model
+
+Choonsik is a **Vision-Language-Action (VLA)** model for Minecraft, built on
+[Qwen/Qwen3.5-9B](<https://huggingface.co/Qwen/Qwen3.5-9B>) and trained with the
+three-stage **ActVLP** pipeline from
+[JARVIS-VLA](https://arxiv.org/abs/2503.16365).
+
+Given a Minecraft observation frame and a natural-language task instruction,
+Choonsik outputs keyboard + mouse action tokens that can be executed directly
+in the game — covering 1,000+ atomic tasks (crafting, mining, smelting, combat,
+navigation, etc.).
 
-This model is a fine-tuned version of [Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B).
-It has been trained using [TRL](https://github.com/huggingface/trl).
+| | |
+|---|---|
+| **Base model** | [Qwen/Qwen3.5-9B](<https://huggingface.co/Qwen/Qwen3.5-9B>) |
+| **Training data** | [CraftJarvis/minecraft-vla-sft](<https://huggingface.co/datasets/CraftJarvis/minecraft-vla-sft>) (3.78M examples) |
+| **Training stages** | Language → Vision-Language → Imitation Learning |
+| **License** | MIT |
 
-## Quick start
+## Usage
 
 ```python
-from transformers import pipeline
+from choonsik.inference import ChoonsikInferenceRunner
+from PIL import Image
 
-question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
-generator = pipeline("text-generation", model="Infinity08/Choonsik-Qwen3.5-9B", device="cuda")
-output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
-print(output["generated_text"])
-```
+runner = ChoonsikInferenceRunner("Infinity08/Choonsik-Qwen3.5-9B")
+frame  = Image.open("minecraft_frame.png")
 
-## Training procedure
+action = runner.predict(frame, task="craft a wooden pickaxe")
+# action = {"forward": 0, "attack": 1, ..., "camera": [0.0, 0.3]}
+```
 
-[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/infinity08-choonsik/huggingface/runs/n0avpowm) 
+## Action Space
 
+Choonsik predicts actions using **mu-law discretized tokens**:
 
+| Token type | Count | Description |
+|---|---|---|
+| Keyboard | 29 | `forward`, `attack`, `use`, `jump`, hotbar 1–9, … |
+| Mouse X | 21 | Horizontal camera rotation (mu-law bins) |
+| Mouse Y | 21 | Vertical camera rotation (mu-law bins) |
 
-This model was trained with SFT.
+## Training
 
-### Framework versions
+Three-stage ActVLP pipeline (following JARVIS-VLA):
 
-- TRL: 1.3.0
-- Transformers: 5.7.0
-- Pytorch: 2.4.1+cu124
-- Datasets: 4.8.5
-- Tokenizers: 0.22.2
+1. **Stage 1 — Language post-training**: Minecraft world knowledge (text-only SFT)
+2. **Stage 2 — Vision-language alignment**: Image captioning and VQA on gameplay frames
+3. **Stage 3 — Imitation learning**: Action prediction on 3.78M trajectory examples
 
-## Citations
+Training hardware: L40S (48 GB VRAM). Inference: RTX 5080 with 4-bit NF4 quantization.
 
+## Citation
 
+If you use Choonsik or the underlying JARVIS-VLA methodology, please cite:
 
-Cite TRL as:
-    
 ```bibtex
-@software{vonwerra2020trl,
-  title   = {{TRL: Transformers Reinforcement Learning}},
-  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
-  license = {Apache-2.0},
-  url     = {https://github.com/huggingface/trl},
-  year    = {2020}
+@article{li2025jarvisvla,
+  title   = {JARVIS-VLA: Post-Training Large-Scale Vision Language Models
+             to Play Visual Games with Keyboards and Mouse},
+  author  = {Muyao Li and Zihao Wang and Kaichen He and others},
+  journal = {arXiv preprint arXiv:2503.16365},
+  year    = {2025}
 }
-```
+```
+

config.json

@@ -0,0 +1,83 @@
+{
+  "architectures": [
+    "Qwen3_5ForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "attn_output_gate": true,
+  "bos_token_id": null,
+  "dtype": "bfloat16",
+  "eos_token_id": 248044,
+  "full_attention_interval": 4,
+  "head_dim": 256,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 12288,
+  "layer_types": [
+    "linear_attention",
+    "linear_attention",
+    "linear_attention",
+    "full_attention",
+    "linear_attention",
+    "linear_attention",
+    "linear_attention",
+    "full_attention",
+    "linear_attention",
+    "linear_attention",
+    "linear_attention",
+    "full_attention",
+    "linear_attention",
+    "linear_attention",
+    "linear_attention",
+    "full_attention",
+    "linear_attention",
+    "linear_attention",
+    "linear_attention",
+    "full_attention",
+    "linear_attention",
+    "linear_attention",
+    "linear_attention",
+    "full_attention",
+    "linear_attention",
+    "linear_attention",
+    "linear_attention",
+    "full_attention",
+    "linear_attention",
+    "linear_attention",
+    "linear_attention",
+    "full_attention"
+  ],
+  "linear_conv_kernel_dim": 4,
+  "linear_key_head_dim": 128,
+  "linear_num_key_heads": 16,
+  "linear_num_value_heads": 32,
+  "linear_value_head_dim": 128,
+  "mamba_ssm_dtype": "float32",
+  "max_position_embeddings": 262144,
+  "mlp_only_layers": [],
+  "model_type": "qwen3_5_text",
+  "mtp_num_hidden_layers": 1,
+  "mtp_use_dedicated_embeddings": false,
+  "num_attention_heads": 16,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 4,
+  "pad_token_id": null,
+  "partial_rotary_factor": 0.25,
+  "rms_norm_eps": 1e-06,
+  "rope_parameters": {
+    "mrope_interleaved": true,
+    "mrope_section": [
+      11,
+      11,
+      10
+    ],
+    "partial_rotary_factor": 0.25,
+    "rope_theta": 10000000,
+    "rope_type": "default"
+  },
+  "tie_word_embeddings": false,
+  "transformers_version": "5.7.0",
+  "use_cache": true,
+  "vocab_size": 248320
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "eos_token_id": 248044,
+  "transformers_version": "5.7.0",
+  "use_cache": true
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61b7d5f795d3f7504705a0881f2b7262ed2084f7e5d099d93f4a854a76514482
+size 17907663008

processor_config.json

@@ -20,8 +20,8 @@
     "resample": 3,
     "rescale_factor": 0.00392156862745098,
     "size": {
-      "longest_edge": 16777216,
-      "shortest_edge": 65536
+      "longest_edge": 200704,
+      "shortest_edge": 50176
     },
     "temporal_patch_size": 2
   },

tokenizer_config.json

@@ -9,8 +9,10 @@
   "eos_token": "<|im_end|>",
   "errors": "replace",
   "image_token": "<|image_pad|>",
-  "is_local": false,
+  "is_local": true,
   "local_files_only": false,
+  "max_pixels": 200704,
+  "min_pixels": 50176,
   "model_max_length": 262144,
   "model_specific_special_tokens": {
     "audio_bos_token": "<|audio_start|>",