End of training
Browse files- README.md +9 -15
- adapter_model.bin +1 -1
README.md
CHANGED
|
@@ -64,8 +64,8 @@ lora_model_dir: null
|
|
| 64 |
lora_r: 32
|
| 65 |
lora_target_linear: true
|
| 66 |
lr_scheduler: cosine
|
| 67 |
-
max_steps:
|
| 68 |
-
micro_batch_size:
|
| 69 |
mlflow_experiment_name: /tmp/f4249ae0a9430740_train_data.json
|
| 70 |
model_type: AutoModelForCausalLM
|
| 71 |
num_epochs: 10
|
|
@@ -78,20 +78,14 @@ sample_packing: false
|
|
| 78 |
save_steps: 0
|
| 79 |
saves_per_epoch: null
|
| 80 |
seed: 20
|
| 81 |
-
sequence_len:
|
| 82 |
strict: false
|
| 83 |
tf32: false
|
| 84 |
tokenizer_type: AutoTokenizer
|
| 85 |
train_on_inputs: false
|
| 86 |
trust_remote_code: true
|
| 87 |
-
wandb_entity: null
|
| 88 |
-
wandb_mode: online
|
| 89 |
-
wandb_name: def0065d-f6b6-49e0-85c3-e5fce04c5505
|
| 90 |
-
wandb_project: Gradients-On-Demand
|
| 91 |
-
wandb_run: your_name
|
| 92 |
-
wandb_runid: def0065d-f6b6-49e0-85c3-e5fce04c5505
|
| 93 |
warmup_steps: 100
|
| 94 |
-
weight_decay: 0.
|
| 95 |
xformers_attention: null
|
| 96 |
|
| 97 |
```
|
|
@@ -120,18 +114,18 @@ More information needed
|
|
| 120 |
|
| 121 |
The following hyperparameters were used during training:
|
| 122 |
- learning_rate: 0.000202
|
| 123 |
-
- train_batch_size:
|
| 124 |
-
- eval_batch_size:
|
| 125 |
- seed: 20
|
| 126 |
- distributed_type: multi-GPU
|
| 127 |
- num_devices: 8
|
| 128 |
- gradient_accumulation_steps: 4
|
| 129 |
-
- total_train_batch_size:
|
| 130 |
-
- total_eval_batch_size:
|
| 131 |
- optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
|
| 132 |
- lr_scheduler_type: cosine
|
| 133 |
- lr_scheduler_warmup_steps: 100
|
| 134 |
-
- training_steps:
|
| 135 |
|
| 136 |
### Training results
|
| 137 |
|
|
|
|
| 64 |
lora_r: 32
|
| 65 |
lora_target_linear: true
|
| 66 |
lr_scheduler: cosine
|
| 67 |
+
max_steps: 700
|
| 68 |
+
micro_batch_size: 4
|
| 69 |
mlflow_experiment_name: /tmp/f4249ae0a9430740_train_data.json
|
| 70 |
model_type: AutoModelForCausalLM
|
| 71 |
num_epochs: 10
|
|
|
|
| 78 |
save_steps: 0
|
| 79 |
saves_per_epoch: null
|
| 80 |
seed: 20
|
| 81 |
+
sequence_len: 2048
|
| 82 |
strict: false
|
| 83 |
tf32: false
|
| 84 |
tokenizer_type: AutoTokenizer
|
| 85 |
train_on_inputs: false
|
| 86 |
trust_remote_code: true
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 87 |
warmup_steps: 100
|
| 88 |
+
weight_decay: 0.01
|
| 89 |
xformers_attention: null
|
| 90 |
|
| 91 |
```
|
|
|
|
| 114 |
|
| 115 |
The following hyperparameters were used during training:
|
| 116 |
- learning_rate: 0.000202
|
| 117 |
+
- train_batch_size: 4
|
| 118 |
+
- eval_batch_size: 4
|
| 119 |
- seed: 20
|
| 120 |
- distributed_type: multi-GPU
|
| 121 |
- num_devices: 8
|
| 122 |
- gradient_accumulation_steps: 4
|
| 123 |
+
- total_train_batch_size: 128
|
| 124 |
+
- total_eval_batch_size: 32
|
| 125 |
- optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
|
| 126 |
- lr_scheduler_type: cosine
|
| 127 |
- lr_scheduler_warmup_steps: 100
|
| 128 |
+
- training_steps: 700
|
| 129 |
|
| 130 |
### Training results
|
| 131 |
|
adapter_model.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 60676170
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:23ee88fbcd6d9bb5210e489229ad05a072823a21a1a14ca44a09695b12ba8d29
|
| 3 |
size 60676170
|