KLD sample size
#8
by cpral - opened
Hi @mratsim and thanks for your new Qwen 397B tuned quant
I saw a new note about KLD measurements
For speed, this was measured with only 10 lines of 2048 tokens from wikitext2. The default is 100 lines, and according to my benchmarks for Qwen3.5-397B the KL-div can be much lower with 100. If you compare this to other quants, make sure you use the same number of rows.
Do you plan to measure them with 100 lines soon? If no, let me know and I'll do that since I'd want to have this data for completeness.
Do you plan to measure them with 100 lines soon? If no, let me know and I'll do that since I'd want to have this data for completeness.
No I don't, feel free to go ahead!