Hugging Face's logo

Svetlana0303
/
Regression_bert_NOaug_CustomLoss

Text Classification
Transformers
TensorFlow
distilbert
generated_from_keras_callback
Model card Files
xet
 Community
Svetlana0303 commited on
Commit
424fa63
·
1 Parent(s): bd0adbd

Upload TFDistilBertForSequenceClassification

Browse files
Files changed (3) hide show
  1. README.md +73 -0
  2. config.json +30 -0
  3. tf_model.h5 +3 -0
README.md ADDED
@@ -0,0 +1,73 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ tags:
4
+ - generated_from_keras_callback
5
+ model-index:
6
+ - name: Regression_bert_NOaug_CustomLoss
7
+ results: []
8
+ ---
9
+
10
+ <!-- This model card has been generated automatically according to the information Keras had access to. You should
11
+ probably proofread and complete it, then remove this comment. -->
12
+
13
+ # Regression_bert_NOaug_CustomLoss
14
+
15
+ This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on an unknown dataset.
16
+ It achieves the following results on the evaluation set:
17
+ - Train Loss: 0.0264
18
+ - Train Mae: 0.1981
19
+ - Train Mse: 0.0536
20
+ - Train R2-score: 0.9557
21
+ - Validation Loss: 0.1484
22
+ - Validation Mae: 0.3703
23
+ - Validation Mse: 0.2656
24
+ - Validation R2-score: 0.8862
25
+ - Epoch: 14
26
+
27
+ ## Model description
28
+
29
+ More information needed
30
+
31
+ ## Intended uses & limitations
32
+
33
+ More information needed
34
+
35
+ ## Training and evaluation data
36
+
37
+ More information needed
38
+
39
+ ## Training procedure
40
+
41
+ ### Training hyperparameters
42
+
43
+ The following hyperparameters were used during training:
44
+ - optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': 1e-04, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
45
+ - training_precision: float32
46
+
47
+ ### Training results
48
+
49
+ | Train Loss | Train Mae | Train Mse | Train R2-score | Validation Loss | Validation Mae | Validation Mse | Validation R2-score | Epoch |
50
+ |:----------:|:---------:|:---------:|:--------------:|:---------------:|:--------------:|:--------------:|:-------------------:|:-----:|
51
+ | 0.1477 | 0.5158 | 0.3587 | 0.8489 | 0.1118 | 0.5348 | 0.3366 | 0.8997 | 0 |
52
+ | 0.1280 | 0.4634 | 0.2930 | 0.8414 | 0.1375 | 0.4847 | 0.3121 | 0.8873 | 1 |
53
+ | 0.1232 | 0.4331 | 0.2728 | -0.3855 | 0.1453 | 0.5454 | 0.4140 | 0.8773 | 2 |
54
+ | 0.0862 | 0.3752 | 0.2042 | 0.8843 | 0.1683 | 0.4117 | 0.2940 | 0.8728 | 3 |
55
+ | 0.0827 | 0.3573 | 0.1824 | 0.9046 | 0.1383 | 0.3792 | 0.2434 | 0.8940 | 4 |
56
+ | 0.0701 | 0.4034 | 0.2084 | 0.8164 | 0.1313 | 0.4766 | 0.3297 | 0.8879 | 5 |
57
+ | 0.0473 | 0.2988 | 0.1245 | 0.8744 | 0.1544 | 0.4001 | 0.2930 | 0.8780 | 6 |
58
+ | 0.0370 | 0.2501 | 0.0887 | 0.8672 | 0.1464 | 0.4236 | 0.3019 | 0.8809 | 7 |
59
+ | 0.0346 | 0.3122 | 0.1224 | 0.9196 | 0.1296 | 0.4837 | 0.3147 | 0.8885 | 8 |
60
+ | 0.0303 | 0.2493 | 0.0864 | 0.9624 | 0.1399 | 0.4292 | 0.2975 | 0.8876 | 9 |
61
+ | 0.0312 | 0.2527 | 0.0862 | 0.9426 | 0.1436 | 0.3984 | 0.2722 | 0.8876 | 10 |
62
+ | 0.0301 | 0.2160 | 0.0657 | 0.6312 | 0.1479 | 0.3819 | 0.2836 | 0.8849 | 11 |
63
+ | 0.0275 | 0.2286 | 0.0712 | 0.9543 | 0.1473 | 0.3770 | 0.2634 | 0.8851 | 12 |
64
+ | 0.0272 | 0.2209 | 0.0656 | 0.9691 | 0.1372 | 0.4141 | 0.2886 | 0.8899 | 13 |
65
+ | 0.0264 | 0.1981 | 0.0536 | 0.9557 | 0.1484 | 0.3703 | 0.2656 | 0.8862 | 14 |
66
+
67
+
68
+ ### Framework versions
69
+
70
+ - Transformers 4.28.1
71
+ - TensorFlow 2.12.0
72
+ - Datasets 2.12.0
73
+ - Tokenizers 0.13.3
config.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "distilbert-base-uncased",
3
+ "activation": "gelu",
4
+ "architectures": [
5
+ "DistilBertForSequenceClassification"
6
+ ],
7
+ "attention_dropout": 0.1,
8
+ "dim": 768,
9
+ "dropout": 0.1,
10
+ "hidden_dim": 3072,
11
+ "id2label": {
12
+ "0": "LABEL_0"
13
+ },
14
+ "initializer_range": 0.02,
15
+ "label2id": {
16
+ "LABEL_0": 0
17
+ },
18
+ "max_position_embeddings": 512,
19
+ "model_type": "distilbert",
20
+ "n_heads": 12,
21
+ "n_layers": 6,
22
+ "pad_token_id": 0,
23
+ "problem_type": "regression",
24
+ "qa_dropout": 0.1,
25
+ "seq_classif_dropout": 0.2,
26
+ "sinusoidal_pos_embds": false,
27
+ "tie_weights_": true,
28
+ "transformers_version": "4.28.1",
29
+ "vocab_size": 30522
30
+ }
tf_model.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:30e8c05722b3865bc1d610d2a739994f637710c5b4596f6d091155149702706a
3
+ size 267948736