Update README.md

README.md

@@ -30,6 +30,31 @@ Finetune on [mesolitica/Malaysian-SFT](https://huggingface.co/datasets/mesolitic
 
 Source code at https://github.com/mesolitica/malaya/tree/master/session/gemma3
 
+## Benchmark
+
+### MalayMMLU
+
+Based on 0-shot first token accuracy,
+
+```
+                     Model   Accuracy   shot by_letter        category
+0  Malaysian-gemma-3-1b-it  48.096603  0shot      True            STEM
+1  Malaysian-gemma-3-1b-it  47.423664  0shot      True        Language
+2  Malaysian-gemma-3-1b-it  47.210176  0shot      True  Social science
+3  Malaysian-gemma-3-1b-it  47.709283  0shot      True          Others
+4  Malaysian-gemma-3-1b-it  51.786121  0shot      True      Humanities
+{'Social science': 6918, 'Language': 6288, 'Humanities': 4395, 'Others': 4169, 'STEM': 2443}
+Model : Malaysian-gemma-3-1b-it
+Metric : first
+Shot : 0shot
+average accuracy 48.27158964192789
+accuracy for STEM 48.09660253786328
+accuracy for Language 47.4236641221374
+accuracy for Social science 47.21017635154669
+accuracy for Others 47.70928280163108
+accuracy for Humanities 51.786120591581344
+```
+
 ## Acknowledgement
 
 Special thanks to https://www.sns.com.my for 8x H100 node!