pretrained-hist-l2_tenKQ_finetune-itemseg_v12

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.8729
• Accuracy: 0.9413
• Macro F1: 0.8360

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 3365
• training_steps: 67312

Training results

Training Loss	Epoch	Step	Accuracy	Validation Loss	Macro F1
9.4894	2.0010	201	0.4265	82.2341	0.1031
6.7449	5.0009	402	0.5825	102.5144	0.1528
5.4965	8.0009	603	0.6267	107.0084	0.1853
4.2875	11.0008	804	0.6563	73.1985	0.2102
3.3496	14.0008	1005	0.6989	36.2252	0.2567
2.779	17.0007	1206	0.7334	19.4525	0.2885
2.4203	20.0007	1407	0.7516	11.0852	0.3233
2.2108	23.0007	1608	0.7236	9.4808	0.3318
2.0646	26.0006	1809	0.7676	6.1216	0.3933
1.9279	29.0006	2010	0.7791	5.7069	0.4219
1.7764	32.0005	2211	0.7934	5.7469	0.4651
1.6468	35.0005	2412	0.8065	4.0662	0.5038
1.5504	38.0004	2613	0.8191	3.9270	0.5150
1.4653	41.0004	2814	0.8218	3.7084	0.5407
1.3526	44.0003	3015	0.8426	3.4484	0.5808
1.2435	47.0003	3216	0.8489	3.2652	0.6123
1.1545	50.0003	3417	0.8612	3.4536	0.6279
1.104	53.0002	3618	0.8696	3.7528	0.6489
1.0201	56.0002	3819	0.8789	3.5046	0.6692
0.9694	59.0001	4020	0.8816	3.8398	0.6839
0.9406	62.0001	4221	0.8892	4.4960	0.6946
0.907	65.0000	4422	0.8817	4.5369	0.6938
0.8771	67.0010	4623	0.8912	5.3581	0.7163
0.8431	70.0010	4824	0.9007	5.0785	0.7283
0.8201	73.0009	5025	0.8999	5.8392	0.7279
0.8044	76.0009	5226	0.9036	6.3780	0.7407
0.7838	79.0008	5427	0.9035	6.7336	0.7401
0.7701	82.0008	5628	0.9061	7.0209	0.7442
0.7588	85.0007	5829	0.9099	6.8126	0.7545
0.7485	88.0007	6030	0.9080	8.0571	0.7498
0.7313	91.0006	6231	0.9102	7.7641	0.7590
0.7253	94.0006	6432	0.9125	7.0026	0.7584
0.7207	97.0005	6633	0.9109	7.4268	0.7692
0.7119	100.0005	6834	0.9149	7.4075	0.7705
0.7091	103.0005	7035	0.9174	7.1209	0.7766
0.698	106.0004	7236	0.9178	7.1546	0.7784
0.6912	109.0004	7437	0.9168	6.4869	0.7815
0.6859	112.0003	7638	0.9176	6.0039	0.7791
0.6779	115.0003	7839	0.9171	6.5499	0.7831
0.6794	118.0002	8040	0.9187	5.7540	0.7860
0.6675	121.0002	8241	0.9209	5.9967	0.7884
0.6642	124.0001	8442	0.9229	6.1720	0.7908
0.6629	127.0001	8643	0.9219	5.4910	0.7916
0.6584	130.0001	8844	0.9204	5.0177	0.7890
0.6582	133.0000	9045	0.9212	4.4445	0.7906
0.649	135.0010	9246	0.9221	5.1284	0.7947
0.6477	138.0009	9447	0.9237	4.3223	0.7975
0.649	141.0009	9648	0.9267	4.3668	0.7997
0.6419	144.0008	9849	0.9242	3.7235	0.7965
0.6423	147.0008	10050	0.9248	3.5874	0.7969
0.6394	150.0008	10251	0.9290	3.6674	0.8045
0.6356	153.0007	10452	0.9276	3.2794	0.8036
0.6351	156.0007	10653	0.9279	3.5941	0.8043
0.6391	159.0006	10854	0.9275	3.3168	0.8016
0.629	162.0006	11055	0.9282	3.2130	0.8022
0.6333	165.0005	11256	0.9276	3.2266	0.8003
0.6321	168.0005	11457	0.9255	3.0698	0.8032
0.6344	171.0004	11658	0.9229	2.9339	0.8013
0.6277	174.0004	11859	0.9290	2.8489	0.8068
0.6186	177.0004	12060	0.9293	2.7256	0.8061
0.6242	180.0003	12261	0.9286	2.7066	0.8070
0.6198	183.0003	12462	0.9308	2.6861	0.8097
0.6155	186.0002	12663	0.9293	2.6359	0.8076
0.6164	189.0002	12864	0.9277	2.5540	0.8092
0.6153	192.0001	13065	0.9296	2.6405	0.8102
0.6104	195.0001	13266	0.9299	2.3697	0.8097
0.6108	198.0000	13467	0.9308	2.2934	0.8124
0.6102	200.0010	13668	0.9302	2.4024	0.8100
0.6085	203.0010	13869	0.9313	2.3044	0.8141
0.6069	206.0009	14070	0.9314	2.3014	0.8118
0.6042	209.0009	14271	0.9285	2.1750	0.8108
0.6062	212.0008	14472	0.9296	2.2470	0.8098
0.6012	215.0008	14673	0.9305	2.1518	0.8156
0.6038	218.0007	14874	0.9300	2.1946	0.8150
0.6051	221.0007	15075	0.9300	2.2586	0.8145
0.6014	224.0007	15276	0.9289	2.1255	0.8138
0.605	227.0006	15477	0.9333	2.0546	0.8150
0.5963	230.0006	15678	0.9316	2.1129	0.8139
0.5986	233.0005	15879	0.9318	2.1905	0.8171
0.5969	236.0005	16080	0.9316	2.1857	0.8119
0.5932	239.0004	16281	0.9321	2.2104	0.8163
0.5958	242.0004	16482	0.9338	2.1522	0.8190
0.5887	245.0003	16683	0.9332	2.0534	0.8168
0.5926	248.0003	16884	0.9327	2.1169	0.8139
0.59	251.0003	17085	0.9307	2.1269	0.8175
0.592	254.0002	17286	0.9294	2.2844	0.8158
0.5867	257.0002	17487	0.9350	2.0603	0.8225
0.5889	260.0001	17688	0.9316	2.0984	0.8169
0.589	263.0001	17889	0.9330	2.0921	0.8199
0.5837	266.0000	18090	0.9334	2.1851	0.8186
0.5833	268.0010	18291	0.9338	1.9966	0.8171
0.5921	271.0010	18492	0.9338	1.9955	0.8179
0.5851	274.0009	18693	0.9337	1.9206	0.8222
0.5793	277.0009	18894	0.9343	2.1252	0.8210
0.5775	280.0008	19095	0.9351	1.9636	0.8242
0.5821	283.0008	19296	0.9339	1.9675	0.8231
0.5755	286.0007	19497	0.9363	2.0011	0.8225
0.5779	289.0007	19698	0.9356	1.8349	0.8243
0.5778	292.0006	19899	0.9368	1.8548	0.8253
0.577	295.0006	20100	0.9362	1.9164	0.8252
0.5768	298.0005	20301	0.9335	1.9570	0.8206
0.5753	301.0005	20502	0.9366	1.8765	0.8258
0.5792	304.0005	20703	0.9364	1.8633	0.8254
0.5734	307.0004	20904	0.9368	1.9099	0.8258
0.5781	310.0004	21105	0.9364	1.9035	0.8255
0.5733	313.0003	21306	0.9368	1.9031	0.8250
0.5738	316.0003	21507	0.9362	1.8097	0.8259
0.5721	319.0002	21708	0.9356	1.8207	0.8244
0.5721	322.0002	21909	0.9358	1.9438	0.8253
0.5682	325.0001	22110	0.9362	1.9100	0.8244
0.5726	328.0001	22311	0.9349	1.9880	0.8227
0.5716	331.0001	22512	0.9333	1.9597	0.8213
0.5716	334.0000	22713	0.9294	1.9030	0.8198
0.571	336.0010	22914	0.9350	1.8403	0.8252
0.5692	339.0009	23115	0.9362	1.8682	0.8268
0.5652	342.0009	23316	0.9371	1.8518	0.8271
0.561	345.0008	23517	0.9368	1.8311	0.8244
0.5652	348.0008	23718	0.9376	1.9008	0.8262
0.5671	351.0008	23919	0.9385	1.8283	0.8274
0.5685	354.0007	24120	0.9347	1.9261	0.8247
0.5683	357.0007	24321	0.9356	1.9199	0.8238
0.5637	360.0006	24522	0.9375	1.8717	0.8283
0.5638	363.0006	24723	0.9373	1.9691	0.8255
0.5625	366.0005	24924	0.9364	1.8239	0.8263
0.5581	369.0005	25125	0.9370	1.8235	0.8270
0.5609	372.0004	25326	0.9377	1.9154	0.8287
0.5616	375.0004	25527	0.9364	1.8724	0.8290
0.5595	378.0004	25728	0.9375	1.8221	0.8275
0.5561	381.0003	25929	0.9366	1.9112	0.8250
0.5566	384.0003	26130	0.9386	1.8305	0.8283
0.5595	387.0002	26331	0.9371	1.9234	0.8292
0.5583	390.0002	26532	0.9367	1.8729	0.8250
0.5575	393.0001	26733	0.9368	1.8734	0.8253
0.559	396.0001	26934	0.9365	1.8494	0.8267
0.5581	399.0000	27135	0.9366	1.9040	0.8256
0.5568	401.0010	27336	0.9364	1.9088	0.8265
0.5558	404.0010	27537	0.9365	1.9019	0.8250
0.554	407.0009	27738	0.9364	1.8305	0.8261
0.5545	410.0009	27939	0.9377	1.8043	0.8286
0.5499	413.0008	28140	0.9381	1.8281	0.8276
0.5529	416.0008	28341	0.9365	1.8258	0.8275
0.5508	419.0007	28542	0.9377	1.8154	0.8290
0.5517	422.0007	28743	0.9356	1.9012	0.8269
0.5539	425.0007	28944	0.9360	1.8213	0.8262
0.5524	428.0006	29145	0.9394	1.7988	0.8316
0.5492	431.0006	29346	0.9385	1.8877	0.8283
0.5542	434.0005	29547	0.9394	1.8565	0.8298
0.5462	437.0005	29748	0.9396	1.9118	0.8310
0.5501	440.0004	29949	0.9390	2.0187	0.8308
0.5496	443.0004	30150	0.9383	1.8465	0.8272
0.5508	446.0003	30351	0.9391	1.8768	0.8296
0.5454	449.0003	30552	0.9376	1.8931	0.8294
0.5423	452.0003	30753	0.9372	1.8294	0.8268
0.5496	455.0002	30954	0.9378	1.8447	0.8307
0.547	458.0002	31155	0.9402	1.8675	0.8317
0.5471	461.0001	31356	0.9364	1.9534	0.8267
0.5483	464.0001	31557	0.9390	1.8403	0.8315
0.5468	467.0000	31758	0.9382	1.8295	0.8289
0.5437	469.0010	31959	0.9386	1.8962	0.8308
0.5475	472.0010	32160	0.9386	1.8455	0.8298
0.5438	475.0009	32361	0.9381	1.8167	0.8301
0.5373	478.0009	32562	0.9393	1.8011	0.8320
0.5443	481.0008	32763	0.9390	1.8344	0.8309
0.5451	484.0008	32964	0.9402	1.8497	0.8332
0.5435	487.0007	33165	0.9391	1.8581	0.8319
0.5432	490.0007	33366	0.9397	1.8560	0.8309
0.5454	493.0006	33567	0.9372	1.9058	0.8273
0.545	496.0006	33768	0.9398	1.8449	0.8331
0.5379	499.0005	33969	0.9397	1.9129	0.8332
0.5416	502.0005	34170	0.9391	1.8154	0.8296
0.5405	505.0005	34371	0.9398	1.8035	0.8308
0.5367	508.0004	34572	0.9362	1.8193	0.8259
0.542	511.0004	34773	0.9403	1.8016	0.8314
0.541	514.0003	34974	0.9403	1.8085	0.8318
0.5388	517.0003	35175	0.9391	1.8389	0.8301
0.5353	520.0002	35376	0.9390	1.8476	0.8311
0.5403	523.0002	35577	0.9397	1.8621	0.8324
0.538	526.0001	35778	0.9389	1.8458	0.8298
0.537	529.0001	35979	0.9395	1.8377	0.8299
0.5401	532.0001	36180	0.9398	1.8880	0.8320
0.5382	535.0000	36381	0.9389	1.8733	0.8312
0.5388	537.0010	36582	0.9402	1.8899	0.8333
0.5332	540.0009	36783	0.9396	1.9142	0.8333
0.5384	543.0009	36984	0.9395	1.8428	0.8319
0.5426	546.0008	37185	0.9384	1.8582	0.8290
0.5423	549.0008	37386	0.9380	1.9310	0.8305
0.5329	552.0008	37587	0.9389	1.8910	0.8317
0.536	555.0007	37788	0.9401	1.8495	0.8325
0.5324	558.0007	37989	0.9398	1.7763	0.8340
0.5366	561.0006	38190	0.9394	1.9208	0.8318
0.5328	564.0006	38391	0.9394	1.8634	0.8308
0.5307	567.0005	38592	0.9391	1.8508	0.8310
0.5374	570.0005	38793	0.9392	1.8415	0.8308
0.5333	573.0004	38994	0.9390	1.8571	0.8324
0.5315	576.0004	39195	0.9400	1.8383	0.8334
0.5321	579.0004	39396	0.9398	1.8877	0.8311
0.5333	582.0003	39597	0.9377	1.9407	0.8294
0.5316	585.0003	39798	0.9395	1.9262	0.8320
0.5328	588.0002	39999	0.9390	1.8173	0.8298
0.5322	591.0002	40200	0.9396	1.9507	0.8341
0.532	594.0001	40401	0.9386	1.8563	0.8325
0.5245	597.0001	40602	0.9401	1.8680	0.8343
0.5298	600.0000	40803	0.9392	1.8437	0.8323
0.527	602.0010	41004	0.9398	1.8172	0.8340
0.5284	605.0010	41205	0.9392	1.8983	0.8326
0.5283	608.0009	41406	0.9392	1.8673	0.8319
0.5284	611.0009	41607	0.9393	1.9442	0.8319
0.5317	614.0008	41808	0.9397	1.8397	0.8332
0.5277	617.0008	42009	0.9400	1.9985	0.8340
0.5304	620.0007	42210	0.9391	1.8331	0.8335
0.5309	623.0007	42411	0.9396	1.8566	0.8336
0.5317	626.0007	42612	0.9394	1.8307	0.8333
0.5221	629.0006	42813	0.9403	1.8332	0.8344
0.523	632.0006	43014	0.9406	1.8888	0.8326
0.53	635.0005	43215	0.9395	1.7987	0.8321
0.5238	638.0005	43416	0.9413	1.7754	0.8360
0.5284	641.0004	43617	0.9406	1.8406	0.8344
0.5245	644.0004	43818	0.9404	1.9149	0.8343
0.5251	647.0003	44019	0.9389	1.8008	0.8327
0.5272	650.0003	44220	0.9400	1.8639	0.8342
0.5245	653.0003	44421	0.9410	1.8402	0.8354
0.5263	656.0002	44622	0.9393	1.8685	0.8333
0.5227	659.0002	44823	0.9404	2.0006	0.8332
0.5237	662.0001	45024	0.9386	1.9078	0.8320
0.5226	665.0001	45225	0.9387	1.8725	0.8323
0.5218	668.0000	45426	0.9393	1.8364	0.8331
0.5224	670.0010	45627	0.9390	1.8631	0.8304
0.5255	673.0010	45828	0.9405	1.8857	0.8336
0.5213	676.0009	46029	0.9391	1.8577	0.8333
0.5235	679.0009	46230	0.9391	1.8360	0.8320
0.5245	682.0008	46431	0.9389	1.8353	0.8326
0.5187	685.0008	46632	0.9397	1.8014	0.8348
0.5212	688.0007	46833	0.9409	1.8572	0.8353
0.5186	691.0007	47034	0.9410	1.8790	0.8351
0.521	694.0006	47235	0.9407	1.8966	0.8346
0.5228	697.0006	47436	0.9410	1.8963	0.8351

Framework versions

• Transformers 4.46.0
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.20.1