Hugging Face's logo

Padu98
/
ampazephyr-2-prompt-2-versuch-2

PEFT
TensorBoard
Safetensors
trl
sft
Generated from Trainer
Model card Files
xet
 Metrics Community
Padu98 commited on
Commit
4af6939
·
verified ·
1 Parent(s): cf57d29

Padu98/ampazephyr-2-prompt-2-versuch-2

Browse files
Files changed (8) hide show
  1. README.md +5 -5
  2. adapter_config.json +6 -1
  3. adapter_model.safetensors +2 -2
  4. runs/Mar05_18-15-01_b84c459f25d9/events.out.tfevents.1709662503.b84c459f25d9.174.0 +3 -0
  5. runs/Mar05_23-53-34_fd8102db6ce4/events.out.tfevents.1709682816.fd8102db6ce4.229.0 +3 -0
  6. runs/Mar06_00-12-58_fd8102db6ce4/events.out.tfevents.1709683980.fd8102db6ce4.229.1 +3 -0
  7. runs/Mar06_00-12-58_fd8102db6ce4/events.out.tfevents.1709701064.fd8102db6ce4.229.2 +3 -0
  8. training_args.bin +2 -2
README.md CHANGED
@@ -18,7 +18,7 @@ should probably proofread and complete it, then remove this comment. -->
18
 
19
  This model is a fine-tuned version of [HuggingFaceH4/zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta) on the None dataset.
20
  It achieves the following results on the evaluation set:
21
- - Loss: 0.7187
22
 
23
  ## Model description
24
 
@@ -49,15 +49,15 @@ The following hyperparameters were used during training:
49
 
50
  | Training Loss | Epoch | Step | Validation Loss |
51
  |:-------------:|:-----:|:----:|:---------------:|
52
- | No log | 1.0 | 143 | 0.8529 |
53
- | No log | 2.0 | 286 | 0.7502 |
54
- | No log | 3.0 | 429 | 0.7187 |
55
 
56
 
57
  ### Framework versions
58
 
59
  - PEFT 0.9.0
60
  - Transformers 4.39.0.dev0
61
- - Pytorch 2.0.0
62
  - Datasets 2.16.0
63
  - Tokenizers 0.15.2
 
18
 
19
  This model is a fine-tuned version of [HuggingFaceH4/zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta) on the None dataset.
20
  It achieves the following results on the evaluation set:
21
+ - Loss: 0.3778
22
 
23
  ## Model description
24
 
 
49
 
50
  | Training Loss | Epoch | Step | Validation Loss |
51
  |:-------------:|:-----:|:----:|:---------------:|
52
+ | No log | 1.0 | 136 | 0.4528 |
53
+ | No log | 2.0 | 272 | 0.3838 |
54
+ | No log | 3.0 | 408 | 0.3778 |
55
 
56
 
57
  ### Framework versions
58
 
59
  - PEFT 0.9.0
60
  - Transformers 4.39.0.dev0
61
+ - Pytorch 2.1.2
62
  - Datasets 2.16.0
63
  - Tokenizers 0.15.2
adapter_config.json CHANGED
@@ -19,8 +19,13 @@
19
  "rank_pattern": {},
20
  "revision": null,
21
  "target_modules": [
 
 
 
 
22
  "q_proj",
23
- "v_proj"
 
24
  ],
25
  "task_type": "CAUSAL_LM",
26
  "use_dora": false,
 
19
  "rank_pattern": {},
20
  "revision": null,
21
  "target_modules": [
22
+ "up_proj",
23
+ "down_proj",
24
+ "v_proj",
25
+ "o_proj",
26
  "q_proj",
27
+ "k_proj",
28
+ "gate_proj"
29
  ],
30
  "task_type": "CAUSAL_LM",
31
  "use_dora": false,
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:cae5160df8f0391d6488d4fc476d53fe5e7957b5ee68fc680ff6904775e535b7
3
- size 54543184
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0260ed0775dd5385b8bf9dbfefb99357c632fa8c01c36564e134015ff5a65a78
3
+ size 335604696
runs/Mar05_18-15-01_b84c459f25d9/events.out.tfevents.1709662503.b84c459f25d9.174.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a20f2c8e2432d9cc8a995b4fa9d464bf473531cfd6973749ca39c4334557ed95
3
+ size 5360
runs/Mar05_23-53-34_fd8102db6ce4/events.out.tfevents.1709682816.fd8102db6ce4.229.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:107ce9fa3b4e2de4f8c61452443ffc959a97bc94018ae66154f907dd5d4486b3
3
+ size 5089
runs/Mar06_00-12-58_fd8102db6ce4/events.out.tfevents.1709683980.fd8102db6ce4.229.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6803aab8789c3aaf5d835b3782f079249c4b97768eccf60aab1b2e3b8f68df84
3
+ size 6256
runs/Mar06_00-12-58_fd8102db6ce4/events.out.tfevents.1709701064.fd8102db6ce4.229.2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4983b37de9193dbe9e1635b3bd4c7477d01bcdd29c2b3754cb42b07c5a1d944a
3
+ size 359
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:679d7825894ae33e64f8cd8a5037030aacb1c76a0ca0539bbb176d1e0bb3e3f1
3
- size 4475
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eda54c08c2be8b54edd5c6d9ec5d8435a7476629a1f39c458d52f66064591088
3
+ size 4920