finetuning-twitter-sentiment-distilbert

This model is a fine-tuned version of distilbert-base-uncased on the zeroshot/twitter-financial-news-sentiment dataset. It achieves the following results on the evaluation set:

• Loss: 0.6031
• Accuracy: 0.8591
• F1: 0.8578

Model description

Finetuned Distilbert on Twitter Finance Sentiment Dataset (zeroshot/twitter-financial-news-sentiment)

Intended uses & limitations

Predict sentiment in financial reporting

Training and evaluation data

Training procedure

Trained on colab using a custom notebook and dataset modifications (multi-class and f1 score adjusted)

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	1.0	478	0.4212	0.8444	0.8434
0.5568	2.0	956	0.4151	0.8565	0.8537
0.3143	3.0	1434	0.4950	0.8601	0.8572
0.1866	4.0	1912	0.5650	0.8586	0.8573
0.1144	5.0	2390	0.6031	0.8591	0.8578

Framework versions

• Transformers 5.0.0
• Pytorch 2.10.0+cu128
• Datasets 4.0.0
• Tokenizers 0.22.2