---
license: apache-2.0
tags:
- generated_from_keras_callback
model-index:
- name: Regression_bert_NOaug_CustomLoss
  results: []
---

<!-- This model card has been generated automatically according to the information Keras had access to. You should
probably proofread and complete it, then remove this comment. -->

# Regression_bert_NOaug_CustomLoss

This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on an unknown dataset.
It achieves the following results on the evaluation set:
- Train Loss: 0.0264
- Train Mae: 0.1981
- Train Mse: 0.0536
- Train R2-score: 0.9557
- Validation Loss: 0.1484
- Validation Mae: 0.3703
- Validation Mse: 0.2656
- Validation R2-score: 0.8862
- Epoch: 14

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': 1e-04, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
- training_precision: float32

### Training results

| Train Loss | Train Mae | Train Mse | Train R2-score | Validation Loss | Validation Mae | Validation Mse | Validation R2-score | Epoch |
|:----------:|:---------:|:---------:|:--------------:|:---------------:|:--------------:|:--------------:|:-------------------:|:-----:|
| 0.1477     | 0.5158    | 0.3587    | 0.8489         | 0.1118          | 0.5348         | 0.3366         | 0.8997              | 0     |
| 0.1280     | 0.4634    | 0.2930    | 0.8414         | 0.1375          | 0.4847         | 0.3121         | 0.8873              | 1     |
| 0.1232     | 0.4331    | 0.2728    | -0.3855        | 0.1453          | 0.5454         | 0.4140         | 0.8773              | 2     |
| 0.0862     | 0.3752    | 0.2042    | 0.8843         | 0.1683          | 0.4117         | 0.2940         | 0.8728              | 3     |
| 0.0827     | 0.3573    | 0.1824    | 0.9046         | 0.1383          | 0.3792         | 0.2434         | 0.8940              | 4     |
| 0.0701     | 0.4034    | 0.2084    | 0.8164         | 0.1313          | 0.4766         | 0.3297         | 0.8879              | 5     |
| 0.0473     | 0.2988    | 0.1245    | 0.8744         | 0.1544          | 0.4001         | 0.2930         | 0.8780              | 6     |
| 0.0370     | 0.2501    | 0.0887    | 0.8672         | 0.1464          | 0.4236         | 0.3019         | 0.8809              | 7     |
| 0.0346     | 0.3122    | 0.1224    | 0.9196         | 0.1296          | 0.4837         | 0.3147         | 0.8885              | 8     |
| 0.0303     | 0.2493    | 0.0864    | 0.9624         | 0.1399          | 0.4292         | 0.2975         | 0.8876              | 9     |
| 0.0312     | 0.2527    | 0.0862    | 0.9426         | 0.1436          | 0.3984         | 0.2722         | 0.8876              | 10    |
| 0.0301     | 0.2160    | 0.0657    | 0.6312         | 0.1479          | 0.3819         | 0.2836         | 0.8849              | 11    |
| 0.0275     | 0.2286    | 0.0712    | 0.9543         | 0.1473          | 0.3770         | 0.2634         | 0.8851              | 12    |
| 0.0272     | 0.2209    | 0.0656    | 0.9691         | 0.1372          | 0.4141         | 0.2886         | 0.8899              | 13    |
| 0.0264     | 0.1981    | 0.0536    | 0.9557         | 0.1484          | 0.3703         | 0.2656         | 0.8862              | 14    |


### Framework versions

- Transformers 4.28.1
- TensorFlow 2.12.0
- Datasets 2.12.0
- Tokenizers 0.13.3