--- library_name: transformers base_model: Arthur-Tsai/histv4_pretrain_tssp-smlm tags: - generated_from_trainer metrics: - accuracy model-index: - name: histv4_ftis_pretrain_tssp-smlm_0329 results: [] --- # histv4_ftis_pretrain_tssp-smlm_0329 This model is a fine-tuned version of [Arthur-Tsai/histv4_pretrain_tssp-smlm](https://huggingface.co/Arthur-Tsai/histv4_pretrain_tssp-smlm) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 0.7599 - Accuracy: 0.9610 - Macro F1: 0.9058 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 0.0001 - train_batch_size: 8 - eval_batch_size: 4 - seed: 42 - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments - lr_scheduler_type: linear - lr_scheduler_warmup_steps: 6732 - training_steps: 134650 ### Training results | Training Loss | Epoch | Step | Validation Loss | Accuracy | Macro F1 | |:-------------:|:--------:|:-----:|:---------------:|:--------:|:--------:| | 11.3916 | 0.0025 | 336 | 8.0277 | 0.4182 | 0.1156 | | 6.0125 | 1.0025 | 672 | 5.2700 | 0.5958 | 0.1641 | | 4.7491 | 2.0025 | 1008 | 4.0342 | 0.6281 | 0.1748 | | 3.7009 | 3.0025 | 1344 | 2.8428 | 0.6588 | 0.2100 | | 2.9014 | 4.0025 | 1680 | 2.2087 | 0.7034 | 0.2419 | | 2.3003 | 5.0025 | 2016 | 1.9890 | 0.7256 | 0.2790 | | 2.0895 | 6.0025 | 2352 | 1.8024 | 0.7479 | 0.3347 | | 1.8415 | 7.0024 | 2688 | 1.6778 | 0.7750 | 0.3759 | | 1.7152 | 8.0024 | 3024 | 1.5370 | 0.7951 | 0.4240 | | 1.5497 | 9.0024 | 3360 | 1.4729 | 0.8037 | 0.4553 | | 1.5593 | 10.0024 | 3696 | 1.3168 | 0.8210 | 0.5071 | | 1.3738 | 11.0024 | 4032 | 1.2857 | 0.8339 | 0.5285 | | 1.1876 | 12.0024 | 4368 | 1.1716 | 0.8400 | 0.5568 | | 1.1746 | 13.0024 | 4704 | 1.1193 | 0.8558 | 0.5784 | | 1.1063 | 14.0024 | 5040 | 1.0744 | 0.8567 | 0.5968 | | 1.0336 | 15.0024 | 5376 | 1.0388 | 0.8662 | 0.6259 | | 0.9161 | 16.0024 | 5712 | 1.1186 | 0.8622 | 0.6175 | | 0.8923 | 17.0024 | 6048 | 0.9439 | 0.8857 | 0.6524 | | 0.8547 | 18.0024 | 6384 | 0.9416 | 0.8893 | 0.6759 | | 0.8217 | 19.0024 | 6720 | 0.8847 | 0.8918 | 0.6874 | | 0.777 | 20.0023 | 7056 | 0.9041 | 0.8950 | 0.6807 | | 0.7134 | 21.0023 | 7392 | 0.8724 | 0.9032 | 0.7039 | | 0.7108 | 22.0023 | 7728 | 0.8556 | 0.9046 | 0.7303 | | 0.7189 | 23.0023 | 8064 | 0.8142 | 0.9060 | 0.7210 | | 0.6324 | 24.0023 | 8400 | 0.7806 | 0.9187 | 0.7417 | | 0.5949 | 25.0023 | 8736 | 0.7460 | 0.9238 | 0.7552 | | 0.5285 | 26.0023 | 9072 | 0.7482 | 0.9219 | 0.7618 | | 0.5477 | 27.0023 | 9408 | 0.7793 | 0.9268 | 0.7690 | | 0.5164 | 28.0023 | 9744 | 0.7374 | 0.9291 | 0.7781 | | 0.5476 | 29.0023 | 10080 | 0.7050 | 0.9315 | 0.7851 | | 0.5009 | 30.0023 | 10416 | 0.7018 | 0.9326 | 0.7882 | | 0.4908 | 31.0023 | 10752 | 0.7106 | 0.9344 | 0.7893 | | 0.462 | 32.0023 | 11088 | 0.7520 | 0.9336 | 0.7728 | | 0.4704 | 33.0023 | 11424 | 0.6871 | 0.9346 | 0.7913 | | 0.4612 | 34.0022 | 11760 | 0.6826 | 0.9387 | 0.8070 | | 0.4707 | 35.0022 | 12096 | 0.7139 | 0.9403 | 0.7983 | | 0.414 | 36.0022 | 12432 | 0.7585 | 0.9409 | 0.8161 | | 0.4068 | 37.0022 | 12768 | 0.6720 | 0.9427 | 0.8141 | | 0.4481 | 38.0022 | 13104 | 0.7381 | 0.9403 | 0.8195 | | 0.3791 | 39.0022 | 13440 | 0.7072 | 0.9415 | 0.8226 | | 0.3824 | 40.0022 | 13776 | 0.6873 | 0.9467 | 0.8287 | | 0.3624 | 41.0022 | 14112 | 0.7320 | 0.9439 | 0.8107 | | 0.372 | 42.0022 | 14448 | 0.6873 | 0.9440 | 0.8333 | | 0.3576 | 43.0022 | 14784 | 0.6846 | 0.9461 | 0.8338 | | 0.3481 | 44.0022 | 15120 | 0.7048 | 0.9496 | 0.8377 | | 0.3567 | 45.0022 | 15456 | 0.7339 | 0.9465 | 0.8435 | | 0.3651 | 46.0022 | 15792 | 0.7016 | 0.9414 | 0.8398 | | 0.3462 | 47.0021 | 16128 | 0.6781 | 0.9453 | 0.8254 | | 0.3427 | 48.0021 | 16464 | 0.7162 | 0.9485 | 0.8491 | | 0.3447 | 49.0021 | 16800 | 0.6853 | 0.9491 | 0.8474 | | 0.3354 | 50.0021 | 17136 | 0.6949 | 0.9520 | 0.8491 | | 0.3327 | 51.0021 | 17472 | 0.6741 | 0.9482 | 0.8461 | | 0.3232 | 52.0021 | 17808 | 0.6951 | 0.9501 | 0.8520 | | 0.3226 | 53.0021 | 18144 | 0.7612 | 0.9483 | 0.8510 | | 0.335 | 54.0021 | 18480 | 0.7019 | 0.9519 | 0.8601 | | 0.3211 | 55.0021 | 18816 | 0.7500 | 0.9499 | 0.8546 | | 0.3124 | 56.0021 | 19152 | 0.7240 | 0.9523 | 0.8643 | | 0.3093 | 57.0021 | 19488 | 0.6935 | 0.9520 | 0.8588 | | 0.3138 | 58.0021 | 19824 | 0.7082 | 0.9479 | 0.8654 | | 0.3058 | 59.0021 | 20160 | 0.7292 | 0.9521 | 0.8591 | | 0.3047 | 60.0020 | 20496 | 0.7327 | 0.9541 | 0.8659 | | 0.3055 | 61.0020 | 20832 | 0.7506 | 0.9511 | 0.8668 | | 0.3057 | 62.0020 | 21168 | 0.7143 | 0.9551 | 0.8493 | | 0.3029 | 63.0020 | 21504 | 0.6925 | 0.9518 | 0.8617 | | 0.2922 | 64.0020 | 21840 | 0.7074 | 0.9536 | 0.8654 | | 0.295 | 65.0020 | 22176 | 0.7020 | 0.9532 | 0.8706 | | 0.2983 | 66.0020 | 22512 | 0.6974 | 0.9521 | 0.8713 | | 0.3077 | 67.0020 | 22848 | 0.7091 | 0.9527 | 0.8680 | | 0.2905 | 68.0020 | 23184 | 0.7175 | 0.9540 | 0.8747 | | 0.2805 | 69.0020 | 23520 | 0.7342 | 0.9523 | 0.8737 | | 0.2837 | 70.0020 | 23856 | 0.7966 | 0.9533 | 0.8517 | | 0.2874 | 71.0020 | 24192 | 0.6759 | 0.9556 | 0.8596 | | 0.296 | 72.0020 | 24528 | 0.6929 | 0.9527 | 0.8558 | | 0.2879 | 73.0020 | 24864 | 0.7687 | 0.9525 | 0.8740 | | 0.2921 | 74.0019 | 25200 | 0.7167 | 0.9552 | 0.8817 | | 0.2835 | 75.0019 | 25536 | 0.6643 | 0.9552 | 0.8625 | | 0.2813 | 76.0019 | 25872 | 0.7766 | 0.9546 | 0.8592 | | 0.2855 | 77.0019 | 26208 | 0.7541 | 0.9571 | 0.8624 | | 0.2746 | 78.0019 | 26544 | 0.7661 | 0.9555 | 0.8847 | | 0.2856 | 79.0019 | 26880 | 0.7422 | 0.9564 | 0.8833 | | 0.2703 | 80.0019 | 27216 | 0.7156 | 0.9553 | 0.8862 | | 0.2776 | 81.0019 | 27552 | 0.6946 | 0.9554 | 0.8670 | | 0.27 | 82.0019 | 27888 | 0.7123 | 0.9566 | 0.8786 | | 0.275 | 83.0019 | 28224 | 0.7023 | 0.9535 | 0.8836 | | 0.269 | 84.0019 | 28560 | 0.7575 | 0.9556 | 0.8879 | | 0.2704 | 85.0019 | 28896 | 0.7044 | 0.9569 | 0.8888 | | 0.2663 | 86.0019 | 29232 | 0.8086 | 0.9527 | 0.8824 | | 0.2707 | 87.0018 | 29568 | 0.7517 | 0.9564 | 0.8633 | | 0.2622 | 88.0018 | 29904 | 0.7146 | 0.9564 | 0.8857 | | 0.2692 | 89.0018 | 30240 | 0.7903 | 0.9572 | 0.8702 | | 0.2635 | 90.0018 | 30576 | 0.7233 | 0.9564 | 0.8858 | | 0.2644 | 91.0018 | 30912 | 0.6877 | 0.9581 | 0.8903 | | 0.2671 | 92.0018 | 31248 | 0.7469 | 0.9570 | 0.8895 | | 0.2574 | 93.0018 | 31584 | 0.7492 | 0.9548 | 0.8669 | | 0.2587 | 94.0018 | 31920 | 0.6922 | 0.9586 | 0.8713 | | 0.2652 | 95.0018 | 32256 | 0.8032 | 0.9582 | 0.8919 | | 0.2626 | 96.0018 | 32592 | 0.7381 | 0.9558 | 0.8698 | | 0.2622 | 97.0018 | 32928 | 0.7241 | 0.9550 | 0.8862 | | 0.2541 | 98.0018 | 33264 | 0.6927 | 0.9563 | 0.8896 | | 0.2579 | 99.0018 | 33600 | 0.7792 | 0.9584 | 0.8922 | | 0.2547 | 100.0018 | 33936 | 0.7461 | 0.9592 | 0.8918 | | 0.258 | 101.0017 | 34272 | 0.7134 | 0.9575 | 0.8749 | | 0.2628 | 102.0017 | 34608 | 0.7115 | 0.9573 | 0.8941 | | 0.2588 | 103.0017 | 34944 | 0.8139 | 0.9551 | 0.8923 | | 0.2642 | 104.0017 | 35280 | 0.7333 | 0.9591 | 0.8942 | | 0.2536 | 105.0017 | 35616 | 0.6827 | 0.9611 | 0.8727 | | 0.2494 | 106.0017 | 35952 | 0.8377 | 0.9568 | 0.8926 | | 0.2546 | 107.0017 | 36288 | 0.7951 | 0.9560 | 0.8927 | | 0.2536 | 108.0017 | 36624 | 0.6921 | 0.9571 | 0.8748 | | 0.2494 | 109.0017 | 36960 | 0.7702 | 0.9591 | 0.8930 | | 0.2495 | 110.0017 | 37296 | 0.7993 | 0.9604 | 0.8771 | | 0.2456 | 111.0017 | 37632 | 0.7948 | 0.9574 | 0.8733 | | 0.2566 | 112.0017 | 37968 | 0.7221 | 0.9603 | 0.8976 | | 0.2502 | 113.0017 | 38304 | 0.7206 | 0.9573 | 0.8689 | | 0.2504 | 114.0016 | 38640 | 0.8227 | 0.9573 | 0.8968 | | 0.2542 | 115.0016 | 38976 | 0.7220 | 0.9557 | 0.8720 | | 0.2499 | 116.0016 | 39312 | 0.7095 | 0.9572 | 0.8756 | | 0.2448 | 117.0016 | 39648 | 0.7044 | 0.9591 | 0.8936 | | 0.2463 | 118.0016 | 39984 | 0.7497 | 0.9578 | 0.8770 | | 0.2439 | 119.0016 | 40320 | 0.7419 | 0.9579 | 0.8951 | | 0.2444 | 120.0016 | 40656 | 0.7349 | 0.9597 | 0.9009 | | 0.2423 | 121.0016 | 40992 | 0.6960 | 0.9591 | 0.8816 | | 0.2443 | 122.0016 | 41328 | 0.8190 | 0.9590 | 0.8935 | | 0.2483 | 123.0016 | 41664 | 0.7469 | 0.9581 | 0.8959 | | 0.2509 | 124.0016 | 42000 | 0.6861 | 0.9598 | 0.9000 | | 0.2447 | 125.0016 | 42336 | 0.8074 | 0.9567 | 0.8811 | | 0.2457 | 126.0016 | 42672 | 0.7880 | 0.9556 | 0.8759 | | 0.245 | 127.0016 | 43008 | 0.6982 | 0.9594 | 0.8958 | | 0.2459 | 128.0015 | 43344 | 0.7395 | 0.9580 | 0.8965 | | 0.2404 | 129.0015 | 43680 | 0.8232 | 0.9601 | 0.8959 | | 0.2441 | 130.0015 | 44016 | 0.7924 | 0.9587 | 0.8979 | | 0.2385 | 131.0015 | 44352 | 0.7226 | 0.9619 | 0.8999 | | 0.2428 | 132.0015 | 44688 | 0.7610 | 0.9554 | 0.8997 | | 0.2415 | 133.0015 | 45024 | 0.7271 | 0.9573 | 0.8989 | | 0.2384 | 134.0015 | 45360 | 0.7247 | 0.9590 | 0.9012 | | 0.2401 | 135.0015 | 45696 | 0.7528 | 0.9566 | 0.8738 | | 0.2376 | 136.0015 | 46032 | 0.7993 | 0.9546 | 0.8944 | | 0.2399 | 137.0015 | 46368 | 0.6989 | 0.9579 | 0.8974 | | 0.2359 | 138.0015 | 46704 | 0.7358 | 0.9586 | 0.8973 | | 0.2416 | 139.0015 | 47040 | 0.7672 | 0.9535 | 0.8961 | | 0.2394 | 140.0015 | 47376 | 0.8060 | 0.9580 | 0.8959 | | 0.2393 | 141.0014 | 47712 | 0.8144 | 0.9575 | 0.8815 | | 0.2348 | 142.0014 | 48048 | 0.7318 | 0.9578 | 0.8808 | | 0.2387 | 143.0014 | 48384 | 0.7526 | 0.9598 | 0.8820 | | 0.2411 | 144.0014 | 48720 | 0.7875 | 0.9579 | 0.8975 | | 0.2354 | 145.0014 | 49056 | 0.8195 | 0.9596 | 0.8986 | | 0.2375 | 146.0014 | 49392 | 0.8084 | 0.9568 | 0.8810 | | 0.241 | 147.0014 | 49728 | 0.8042 | 0.9567 | 0.8821 | | 0.2397 | 148.0014 | 50064 | 0.7167 | 0.9590 | 0.9023 | | 0.2339 | 149.0014 | 50400 | 0.8129 | 0.9592 | 0.8864 | | 0.2363 | 150.0014 | 50736 | 0.7552 | 0.9583 | 0.8782 | | 0.2349 | 151.0014 | 51072 | 0.7979 | 0.9564 | 0.8792 | | 0.2365 | 152.0014 | 51408 | 0.7781 | 0.9603 | 0.8811 | | 0.2354 | 153.0014 | 51744 | 0.7425 | 0.9582 | 0.8784 | | 0.2383 | 154.0014 | 52080 | 0.7508 | 0.9600 | 0.8983 | | 0.2337 | 155.0013 | 52416 | 0.7715 | 0.9588 | 0.8813 | | 0.2392 | 156.0013 | 52752 | 0.7476 | 0.9594 | 0.9030 | | 0.2323 | 157.0013 | 53088 | 0.8146 | 0.9541 | 0.9010 | | 0.2323 | 158.0013 | 53424 | 0.7687 | 0.9605 | 0.8828 | | 0.2321 | 159.0013 | 53760 | 0.7765 | 0.9564 | 0.8793 | | 0.2279 | 160.0013 | 54096 | 0.6969 | 0.9581 | 0.8969 | | 0.2318 | 161.0013 | 54432 | 0.8148 | 0.9583 | 0.8784 | | 0.2337 | 162.0013 | 54768 | 0.7911 | 0.9560 | 0.9038 | | 0.2328 | 163.0013 | 55104 | 0.7445 | 0.9586 | 0.9003 | | 0.2327 | 164.0013 | 55440 | 0.7848 | 0.9596 | 0.8844 | | 0.2341 | 165.0013 | 55776 | 0.8535 | 0.9590 | 0.8811 | | 0.2341 | 166.0013 | 56112 | 0.7657 | 0.9573 | 0.8992 | | 0.232 | 167.0013 | 56448 | 0.7364 | 0.9587 | 0.8803 | | 0.2354 | 168.0012 | 56784 | 0.7325 | 0.9609 | 0.9045 | | 0.2362 | 169.0012 | 57120 | 0.7578 | 0.9575 | 0.9021 | | 0.2292 | 170.0012 | 57456 | 0.8224 | 0.9590 | 0.8789 | | 0.2315 | 171.0012 | 57792 | 0.7437 | 0.9586 | 0.8831 | | 0.2284 | 172.0012 | 58128 | 0.7082 | 0.9611 | 0.8855 | | 0.2315 | 173.0012 | 58464 | 0.7512 | 0.9605 | 0.8986 | | 0.2273 | 174.0012 | 58800 | 0.8463 | 0.9580 | 0.8999 | | 0.2259 | 175.0012 | 59136 | 0.7478 | 0.9608 | 0.8798 | | 0.2272 | 176.0012 | 59472 | 0.6928 | 0.9619 | 0.9088 | | 0.2301 | 177.0012 | 59808 | 0.7414 | 0.9586 | 0.9035 | | 0.2283 | 178.0012 | 60144 | 0.6533 | 0.9603 | 0.8823 | | 0.2269 | 179.0012 | 60480 | 0.7978 | 0.9593 | 0.8845 | | 0.2296 | 180.0012 | 60816 | 0.8178 | 0.9589 | 0.8991 | | 0.2241 | 181.0012 | 61152 | 0.7398 | 0.9589 | 0.8841 | | 0.2276 | 182.0011 | 61488 | 0.7210 | 0.9627 | 0.8855 | | 0.2243 | 183.0011 | 61824 | 0.8306 | 0.9551 | 0.8780 | | 0.2252 | 184.0011 | 62160 | 0.8134 | 0.9612 | 0.9048 | | 0.2279 | 185.0011 | 62496 | 0.7744 | 0.9609 | 0.9071 | | 0.2276 | 186.0011 | 62832 | 0.7818 | 0.9606 | 0.9054 | | 0.2269 | 187.0011 | 63168 | 0.8698 | 0.9601 | 0.9051 | | 0.2255 | 188.0011 | 63504 | 0.8384 | 0.9583 | 0.9028 | | 0.2245 | 189.0011 | 63840 | 0.8141 | 0.9588 | 0.9056 | | 0.2247 | 190.0011 | 64176 | 0.7958 | 0.9603 | 0.9050 | | 0.2227 | 191.0011 | 64512 | 0.7830 | 0.9594 | 0.8918 | | 0.2244 | 192.0011 | 64848 | 0.7641 | 0.9610 | 0.9086 | | 0.2279 | 193.0011 | 65184 | 0.7270 | 0.9619 | 0.9076 | | 0.2267 | 194.0011 | 65520 | 0.7470 | 0.9601 | 0.9037 | | 0.2261 | 195.0010 | 65856 | 0.8915 | 0.9593 | 0.9044 | | 0.2226 | 196.0010 | 66192 | 0.8146 | 0.9585 | 0.8828 | ### Framework versions - Transformers 4.46.0 - Pytorch 2.3.1+cu121 - Datasets 2.20.0 - Tokenizers 0.20.1