Qwen3.8-27B-BF16-INT4-W4A16-G32-AutoRound / quantization_logs /Qwen3.8-27B-INT4-W4A16-AutoRound_LightRAG_Ingestion_bench_20260814.log
| SETUP | |
| profile: Kabbala_Shaddow_patched.yml guidance 8173c · 2 examples · 1 declared types | |
| prompts: prompt.py [JSON mode] system 5514c user 983c | |
| input: bench_sample.txt 1864267c → 25 chunks | |
| prompt: ~6752 tokens · mode json · temp 0.4 · limits 120/260 | |
| models: 2 → LOCAL, deepseek | |
| NOISE FLOOR — LOCAL against itself | |
| two identical passes; anything smaller than the gap is not a finding | |
| ▸ LOCAL · pass 1 [LOCAL] temp=0.4 | |
| 531s · unparseable 3/25 · 188 entities · 138 relations · ~25486 out-tokens | |
| ▸ LOCAL · pass 2 [LOCAL] temp=0.4 | |
| 512s · unparseable 1/25 · 228 entities · 161 relations · ~27093 out-tokens | |
| spread: entities ±40 · Rel/Ent ±0.03 · dangling ±1.8pp · types ±0 | |
| self-overlap of entity names: 63% ← ceiling of reproducibility | |
| RUNS | |
| ▸ deepseek [deepseek/deepseek-v4-flash-0731] temp=0.4 | |
| 856s · unparseable 12/25 · 139 entities · 92 relations · ~164294 out-tokens | |
| ════════════════════════════════════════════════════════════════ | |
| LOCAL deepseek noise | |
| ════════════════════════════════════════════════════════════════ | |
| unparseable replies 3 12 ±2 | |
| entities total 188 139 ±40 | |
| entities per chunk 8.5 10.7 ±0.95 | |
| relations total 138 92 ±23 | |
| Rel/Ent ratio 0.73 0.66 ±0.03 | |
| dangling refs % 3.6 0.0 ±1.76 | |
| invented types % 97.3 93.5 ±0.85 | |
| names w/ drifting type % 4.1 5.4 ±0.63 | |
| synthetic names 0 0 ±0 | |
| distinct declared types 1 1 ±0 | |
| top-2 type share % 43.6 62.6 ±7.26 | |
| empty descriptions % 0.0 0.0 ±0 | |
| duplicate names % 0.0 0.0 ±0 | |
| wall time, s 531 856 ±19.34 | |
| seconds per entity 2.82 6.16 ±0.58 | |
| tokens per entity 136 1182 ±16.73 | |
| ──────────────────────────────────────────────────────────────── | |
| name overlap vs ref % 100 24 63% | |
| INVENTED TYPES — not present in your profile | |
| LOCAL: 97.3% of entities | |
| Ангельская_Сущность ×9 invented from scratch | |
| Аנגельская_Сущность ×1 invented from scratch | |
| Божественное_Имя ×19 invented from scratch | |
| Дибур_Заголовок ×2 invented from scratch | |
| Ивритская_Буква ×23 invented from scratch | |
| Ивритская_Лемма ×24 invented from scratch | |
| deepseek: 93.5% of entities | |
| Ангельская_Сущность ×1 invented from scratch | |
| Божественное_Имя ×5 invented from scratch | |
| Гематрия ×1 invented from scratch | |
| Ивритская_ Лемма ×1 invented from scratch | |
| Ивритская_Буква ×17 invented from scratch | |
| Ивритская_Лемма ×18 invented from scratch | |
| type LOCAL deepseek | |
| ──────────────────────────────────────────────────────── | |
| Каббалистический_Концепт 58 69 | |
| Ивритская_Лемма 24 18 | |
| Ивритская_Буква 23 17 | |
| Божественное_Имя 19 5 | |
| Комментатор 20 0 | |
| Other 5 9 | |
| Ангельская_Сущность 9 1 | |
| Каббалист_Мудрец 2 8 | |
| Каббалистический_Труд 6 1 | |
| Кабבалистический_Концепт 6 0 | |
| Раввинистический_Источник 1 4 | |
| Ивритская_Цитата 0 5 | |
| Парцуф 4 0 | |
| Сфира 3 0 | |
| ════════════════════════════════════════════════════════════════ | |
| VERDICT | |
| weighted score — points for leading a metric (within measured | |
| noise). Rel/Ent and dangling refs carry 3, they decide whether | |
| the graph is usable at all. Maximum possible: 17 | |
| LOCAL 12/17 ██████████████░░░░░░ 531s · 2.82s/ent | |
| deepseek 10/17 ████████████░░░░░░░░ 856s · 6.16s/ent | |
| deepseek: loses relations — sparse graph; shreds text into two types instead of extracting meaning; NO speed advantage: 6.16 vs 2.82 s/entity | |
| ════════════════════════════════════════════════════════════════ | |