distilbert-conll2003-ner

This model is a fine-tuned version of distilbert/distilbert-base-uncased on the conll2003 dataset. It achieves the following results on the evaluation set:

• Loss: 0.1074
• Loc: {'precision': 0.9324827219564061, 'recall': 0.954817637452368, 'f1': 0.9435180204410974, 'number': 1837}
• Misc: {'precision': 0.8313120176405733, 'recall': 0.8177874186550976, 'f1': 0.8244942591580099, 'number': 922}
• Org: {'precision': 0.8582848837209303, 'recall': 0.8806860551826995, 'f1': 0.8693411851306589, 'number': 1341}
• Per: {'precision': 0.9718766901027582, 'recall': 0.9755700325732899, 'f1': 0.9737198591167705, 'number': 1842}
• Overall Precision: 0.9124
• Overall Recall: 0.9233
• Overall F1: 0.9178
• Overall Accuracy: 0.9841

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 3
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Loc	Misc	Org	Per	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
0.8142	0.2278	100	0.4868	{'precision': 0.5252085600290171, 'recall': 0.7882416984213392, 'f1': 0.630387461906835, 'number': 1837}	{'precision': 0.5973154362416108, 'recall': 0.19305856832971802, 'f1': 0.2918032786885246, 'number': 922}	{'precision': 0.44214876033057854, 'recall': 0.47874720357941836, 'f1': 0.4597207303974221, 'number': 1341}	{'precision': 0.8619422572178478, 'recall': 0.8914223669923995, 'f1': 0.8764344809180677, 'number': 1842}	0.6098	0.6580	0.6330	0.9365
0.2651	0.4556	200	0.2400	{'precision': 0.80419921875, 'recall': 0.8965704953728906, 'f1': 0.8478764478764479, 'number': 1837}	{'precision': 0.664488017429194, 'recall': 0.6616052060737527, 'f1': 0.6630434782608696, 'number': 922}	{'precision': 0.6728081321473952, 'recall': 0.7897091722595079, 'f1': 0.7265866209262436, 'number': 1341}	{'precision': 0.9495060373216246, 'recall': 0.9391965255157437, 'f1': 0.9443231441048034, 'number': 1842}	0.7931	0.8492	0.8202	0.9685
0.1980	0.6834	300	0.1735	{'precision': 0.8909378292939937, 'recall': 0.9205225911812738, 'f1': 0.9054886211512718, 'number': 1837}	{'precision': 0.711211778029445, 'recall': 0.6811279826464208, 'f1': 0.6958448753462604, 'number': 922}	{'precision': 0.7452645329849772, 'recall': 0.8508575689783744, 'f1': 0.7945682451253483, 'number': 1341}	{'precision': 0.9570585077831455, 'recall': 0.9679695982627579, 'f1': 0.9624831309041835, 'number': 1842}	0.8491	0.8824	0.8654	0.9750
0.1872	0.9112	400	0.1484	{'precision': 0.8834605597964377, 'recall': 0.945019052803484, 'f1': 0.9132035770647028, 'number': 1837}	{'precision': 0.7883040935672515, 'recall': 0.7310195227765727, 'f1': 0.7585818795723129, 'number': 922}	{'precision': 0.8026874115983027, 'recall': 0.8463832960477256, 'f1': 0.8239564428312159, 'number': 1341}	{'precision': 0.953797132235794, 'recall': 0.9750271444082519, 'f1': 0.9642953020134228, 'number': 1842}	0.8731	0.8989	0.8858	0.9785
0.1424	1.1390	500	0.1365	{'precision': 0.9009336099585062, 'recall': 0.9455634186173109, 'f1': 0.9227091633466136, 'number': 1837}	{'precision': 0.8020477815699659, 'recall': 0.764642082429501, 'f1': 0.7828983897834537, 'number': 922}	{'precision': 0.832, 'recall': 0.8530947054436987, 'f1': 0.8424153166421208, 'number': 1341}	{'precision': 0.9575371549893843, 'recall': 0.9793702497285559, 'f1': 0.9683306494900698, 'number': 1842}	0.8886	0.9071	0.8977	0.9812
0.1372	1.3667	600	0.1303	{'precision': 0.9034090909090909, 'recall': 0.9520958083832335, 'f1': 0.9271137026239067, 'number': 1837}	{'precision': 0.7838427947598253, 'recall': 0.7787418655097614, 'f1': 0.7812840043525571, 'number': 922}	{'precision': 0.8535849056603774, 'recall': 0.843400447427293, 'f1': 0.8484621155288823, 'number': 1341}	{'precision': 0.964516129032258, 'recall': 0.9739413680781759, 'f1': 0.9692058346839546, 'number': 1842}	0.8932	0.9074	0.9002	0.9810
0.1547	1.5945	700	0.1210	{'precision': 0.9248523886205046, 'recall': 0.9379422972237343, 'f1': 0.9313513513513514, 'number': 1837}	{'precision': 0.8255269320843092, 'recall': 0.764642082429501, 'f1': 0.793918918918919, 'number': 922}	{'precision': 0.8238596491228071, 'recall': 0.8754660700969426, 'f1': 0.8488792480115691, 'number': 1341}	{'precision': 0.9722826086956522, 'recall': 0.9712269272529859, 'f1': 0.9717544812601847, 'number': 1842}	0.9012	0.9073	0.9042	0.9820
0.1349	1.8223	800	0.1143	{'precision': 0.9209277807063785, 'recall': 0.9510070767555797, 'f1': 0.9357257632565613, 'number': 1837}	{'precision': 0.7986798679867987, 'recall': 0.7874186550976139, 'f1': 0.7930092845439651, 'number': 922}	{'precision': 0.8309859154929577, 'recall': 0.8799403430275914, 'f1': 0.8547627671133647, 'number': 1341}	{'precision': 0.9675500270416442, 'recall': 0.9712269272529859, 'f1': 0.969384990517475, 'number': 1842}	0.8958	0.9159	0.9057	0.9824
0.1095	2.0501	900	0.1131	{'precision': 0.9215376513954713, 'recall': 0.9526401741970604, 'f1': 0.936830835117773, 'number': 1837}	{'precision': 0.8379310344827586, 'recall': 0.7906724511930586, 'f1': 0.8136160714285714, 'number': 922}	{'precision': 0.835799859055673, 'recall': 0.8844146159582401, 'f1': 0.8594202898550725, 'number': 1341}	{'precision': 0.970173535791757, 'recall': 0.9712269272529859, 'f1': 0.9706999457406402, 'number': 1842}	0.9042	0.9179	0.9110	0.9829
0.1190	2.2779	1000	0.1131	{'precision': 0.926610348468849, 'recall': 0.9553620032661949, 'f1': 0.9407665505226481, 'number': 1837}	{'precision': 0.8135223555070883, 'recall': 0.8091106290672451, 'f1': 0.8113104948341489, 'number': 922}	{'precision': 0.843772498200144, 'recall': 0.8739746457867263, 'f1': 0.8586080586080587, 'number': 1341}	{'precision': 0.9682454251883746, 'recall': 0.9766558089033659, 'f1': 0.9724324324324325, 'number': 1842}	0.9033	0.9209	0.912	0.9833
0.1058	2.5057	1100	0.1097	{'precision': 0.9318786588610963, 'recall': 0.9531845400108874, 'f1': 0.942411194833154, 'number': 1837}	{'precision': 0.8092672413793104, 'recall': 0.8145336225596529, 'f1': 0.8118918918918919, 'number': 922}	{'precision': 0.8534798534798534, 'recall': 0.8687546607009694, 'f1': 0.861049519586105, 'number': 1341}	{'precision': 0.9697950377562028, 'recall': 0.9761129207383279, 'f1': 0.9729437229437229, 'number': 1842}	0.9069	0.9197	0.9133	0.9837
0.0992	2.7335	1200	0.1090	{'precision': 0.9292128895932382, 'recall': 0.9575394665215025, 'f1': 0.9431635388739946, 'number': 1837}	{'precision': 0.8337004405286343, 'recall': 0.8210412147505423, 'f1': 0.8273224043715847, 'number': 922}	{'precision': 0.8557971014492753, 'recall': 0.8806860551826995, 'f1': 0.8680632120543919, 'number': 1341}	{'precision': 0.9728703201302225, 'recall': 0.9733984799131379, 'f1': 0.973134328358209, 'number': 1842}	0.9114	0.9239	0.9176	0.9842
0.1005	2.9613	1300	0.1074	{'precision': 0.9324827219564061, 'recall': 0.954817637452368, 'f1': 0.9435180204410974, 'number': 1837}	{'precision': 0.8313120176405733, 'recall': 0.8177874186550976, 'f1': 0.8244942591580099, 'number': 922}	{'precision': 0.8582848837209303, 'recall': 0.8806860551826995, 'f1': 0.8693411851306589, 'number': 1341}	{'precision': 0.9718766901027582, 'recall': 0.9755700325732899, 'f1': 0.9737198591167705, 'number': 1842}	0.9124	0.9233	0.9178	0.9841

Framework versions

• PEFT 0.18.1
• Transformers 5.2.0
• Pytorch 2.9.0+cu126
• Datasets 3.6.0
• Tokenizers 0.22.2