====== Perplexity statistics ======
Mean PPL(Q)                   :   8.681563 ±   0.061366
Mean PPL(base)                :   7.669212 ±   0.052592
Cor(ln(PPL(Q)), ln(PPL(base))):  97.26%
Mean ln(PPL(Q)/PPL(base))     :   0.123988 ±   0.001643
Mean PPL(Q)/PPL(base)         :   1.132002 ±   0.001860
Mean PPL(Q)-PPL(base)         :   1.012351 ±   0.015930

====== KL divergence statistics ======
Mean    KLD:   0.138030 ±   0.000557
Maximum KLD:   7.658795
99.9%   KLD:   2.405652
99.0%   KLD:   0.951313
99.0%   KLD:   0.951313
Median  KLD:   0.084501
10.0%   KLD:   0.002096
 5.0%   KLD:   0.000461
 1.0%   KLD:   0.000051
Minimum KLD:  -0.000001

====== Token probability statistics ======
Mean    Δp: -1.839 ± 0.027 %
Maximum Δp: 83.515%
99.9%   Δp: 47.100%
99.0%   Δp: 26.306%
95.0%   Δp: 12.637%
90.0%   Δp:  6.803%
75.0%   Δp:  0.792%
Median  Δp: -0.100%
25.0%   Δp: -3.802%
10.0%   Δp: -12.897%
 5.0%   Δp: -20.264%
 1.0%   Δp: -38.820%
 0.1%   Δp: -70.608%
Minimum Δp: -99.528%
RMS Δp    : 10.731 ± 0.047 %
Same top p: 82.171 ± 0.098 %