victor70 commited on
Commit
06bafc4
ยท
verified ยท
1 Parent(s): f1cb8ec

Upload folder using huggingface_hub

Browse files
Files changed (5) hide show
  1. HybriKo_tok.model +3 -0
  2. HybriKo_tok.vocab +0 -0
  3. README.md +88 -0
  4. config.yaml +31 -0
  5. pytorch_model.pt +3 -0
HybriKo_tok.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8a9651005063f8bf9efc66d7333da8e99f72dba48791e35d57429159c2f891bb
3
+ size 805880
HybriKo_tok.vocab ADDED
The diff for this file is too large to render. See raw diff
 
README.md ADDED
@@ -0,0 +1,88 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # HybriKo-117M-LinuxFC-SFT-v2
2
+
3
+ Korean Hybrid LLM fine-tuned for Linux Command Function Calling.
4
+
5
+ ## Model Description
6
+
7
+ - **Architecture**: Griffin-style Hybrid (RNN + Attention, 2:1 ratio)
8
+ - **Parameters**: 117.8M
9
+ - **Base Model**: HybriKo-117M (exp7_phase1)
10
+ - **Fine-tuning**: Linux Function Calling SFT
11
+ - **Training Data**: 4,725 samples (21 Linux commands)
12
+
13
+ ## Performance
14
+
15
+ | Metric | Value |
16
+ |--------|-------|
17
+ | **Action Name Accuracy** | **100%** (100/100) |
18
+ | Eval Loss | 0.0039 |
19
+ | Training Epochs | 15 |
20
+
21
+ ## Supported Commands (21)
22
+
23
+ `ls`, `cd`, `mkdir`, `rm`, `cp`, `mv`, `find`, `cat`, `grep`, `head`, `tail`, `wc`, `ps`, `df`, `du`, `top`, `ping`, `curl`, `chmod`, `tar`, `Finish`
24
+
25
+ ## Usage
26
+
27
+ ```python
28
+ import torch
29
+ import sentencepiece as spm
30
+ from hybridko.model import HybriKoModel, HybriKoConfig
31
+
32
+ # Load tokenizer
33
+ sp = spm.SentencePieceProcessor()
34
+ sp.Load("HybriKo_tok.model")
35
+
36
+ # Load model
37
+ config = HybriKoConfig(
38
+ d_model=768, n_layers=12, vocab_size=32000,
39
+ n_heads=12, n_kv_heads=3, ff_mult=3,
40
+ max_seq_len=6144, dropout=0.0
41
+ )
42
+ model = HybriKoModel(config)
43
+ checkpoint = torch.load("pytorch_model.pt", map_location="cpu")
44
+ model.load_state_dict(checkpoint["model_state_dict"])
45
+ model.eval()
46
+
47
+ # Inference
48
+ prompt = """<|im_start|>system
49
+ You are a Linux command assistant.
50
+ <|im_end|>
51
+ <|im_start|>user
52
+ ํ˜„์žฌ ํด๋”์˜ ํŒŒ์ผ ๋ชฉ๋ก์„ ๋ณด์—ฌ์ค˜
53
+ <|im_end|>
54
+ <|im_start|>assistant
55
+ """
56
+ # Generate response...
57
+ ```
58
+
59
+ ## Output Format
60
+
61
+ ```
62
+ Thought: ๋””๋ ‰ํ† ๋ฆฌ ๋‚ด์šฉ์„ ํ™•์ธํ•ฉ๋‹ˆ๋‹ค.
63
+ Action: ls_command
64
+ Action Input: {"path": ".", "options": "-l"}
65
+ ```
66
+
67
+ ## Training Details
68
+
69
+ - **Hardware**: A100 x 8 (DDP)
70
+ - **Batch Size**: 32 (1 per GPU x 8 GPUs x 4 grad accum)
71
+ - **Learning Rate**: 5e-5
72
+ - **Warmup Steps**: 100
73
+ - **Epochs**: 15 (converged)
74
+
75
+ ## License
76
+
77
+ Apache 2.0
78
+
79
+ ## Citation
80
+
81
+ ```bibtex
82
+ @misc{hybridko-linuxfc-2026,
83
+ title={HybriKo-117M-LinuxFC-SFT-v2},
84
+ author={Yaongi},
85
+ year={2026},
86
+ publisher={HuggingFace}
87
+ }
88
+ ```
config.yaml ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # HybriKo Default Configuration
2
+ # ~117.8M parameters, optimized for Colab T4 (16GB VRAM)
3
+
4
+ model:
5
+ d_model: 768 # Hidden dimension
6
+ n_layers: 12 # Number of hybrid layers
7
+ vocab_size: 32000 # Vocabulary size
8
+ n_heads: 12 # Attention heads
9
+ n_kv_heads: 3 # KV heads for GQA (1:4 ratio)
10
+ ff_mult: 3 # Feed-forward multiplier
11
+ max_seq_len: 1024 # Maximum sequence length
12
+
13
+ training:
14
+ learning_rate: 3.0e-4
15
+ weight_decay: 0.1
16
+ warmup_steps: 20
17
+ max_steps: 1000
18
+ grad_accum_steps: 1
19
+ save_steps: 500
20
+ batch_size: 16
21
+ max_length: 512 # Training sequence length
22
+
23
+ data:
24
+ num_samples: 30000
25
+ min_length: 50
26
+ tokenizer_samples: 100000
27
+
28
+ tokenizer:
29
+ vocab_size: 32000
30
+ model_type: unigram
31
+ character_coverage: 0.9995
pytorch_model.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:222a76320a48bfdb5b9673ba5926cb6be7274b3a294db50bc52577caa5cda6f7
3
+ size 1414061818