README.md

metadata

license: apache-2.0
tags:
  - generated_from_keras_callback
model-index:
  - name: Regression_bert_NOaug_CustomLoss
    results: []

Regression_bert_NOaug_CustomLoss

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.0264
• Train Mae: 0.1981
• Train Mse: 0.0536
• Train R2-score: 0.9557
• Validation Loss: 0.1484
• Validation Mae: 0.3703
• Validation Mse: 0.2656
• Validation R2-score: 0.8862
• Epoch: 14

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': 1e-04, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
• training_precision: float32

Training results

Train Loss	Train Mae	Train Mse	Train R2-score	Validation Loss	Validation Mae	Validation Mse	Validation R2-score	Epoch
0.1477	0.5158	0.3587	0.8489	0.1118	0.5348	0.3366	0.8997	0
0.1280	0.4634	0.2930	0.8414	0.1375	0.4847	0.3121	0.8873	1
0.1232	0.4331	0.2728	-0.3855	0.1453	0.5454	0.4140	0.8773	2
0.0862	0.3752	0.2042	0.8843	0.1683	0.4117	0.2940	0.8728	3
0.0827	0.3573	0.1824	0.9046	0.1383	0.3792	0.2434	0.8940	4
0.0701	0.4034	0.2084	0.8164	0.1313	0.4766	0.3297	0.8879	5
0.0473	0.2988	0.1245	0.8744	0.1544	0.4001	0.2930	0.8780	6
0.0370	0.2501	0.0887	0.8672	0.1464	0.4236	0.3019	0.8809	7
0.0346	0.3122	0.1224	0.9196	0.1296	0.4837	0.3147	0.8885	8
0.0303	0.2493	0.0864	0.9624	0.1399	0.4292	0.2975	0.8876	9
0.0312	0.2527	0.0862	0.9426	0.1436	0.3984	0.2722	0.8876	10
0.0301	0.2160	0.0657	0.6312	0.1479	0.3819	0.2836	0.8849	11
0.0275	0.2286	0.0712	0.9543	0.1473	0.3770	0.2634	0.8851	12
0.0272	0.2209	0.0656	0.9691	0.1372	0.4141	0.2886	0.8899	13
0.0264	0.1981	0.0536	0.9557	0.1484	0.3703	0.2656	0.8862	14

Framework versions

• Transformers 4.28.1
• TensorFlow 2.12.0
• Datasets 2.12.0
• Tokenizers 0.13.3