| 0.00.396.812 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg) |
| 0.00.396.817 I device_info: |
| 0.00.793.035 I - CUDA0 : NVIDIA GeForce RTX 3080 (20052 MiB, 19814 MiB free) |
| 0.00.921.578 I - CUDA1 : NVIDIA GeForce RTX 3080 (20052 MiB, 19814 MiB free) |
| 0.01.055.161 I - CUDA2 : NVIDIA GeForce RTX 3080 (20052 MiB, 19814 MiB free) |
| 0.01.055.182 I - CPU : AMD EPYC 7F32 8-Core Processor (193092 MiB, 193092 MiB free) |
| 0.01.055.312 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CUDA : ARCHS = 860 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | |
| 0.01.055.442 I srv init: using 15 threads for HTTP server |
| 0.01.055.930 I srv start: binding port with default address family |
| 0.01.057.281 I srv llama_server: loading model |
| 0.01.057.290 I srv load_model: loading model '/mnt/models/PropellerA-models/qwen3.8-27b-Q8_0.gguf' |
| 0.01.686.307 I srv load_model: [spec] estimated memory usage of draft model is 2048.78 MiB |
| 0.01.686.341 I common_init_result: fitting params to device memory ... |
| 0.01.686.342 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on) |
| 0.08.261.671 W llama_context: n_ctx_seq (16384) < n_ctx_train (262144) -- the full capacity of the model will not be utilized |
| 0.08.361.129 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable) |
| 0.08.677.269 I srv load_model: loading draft model '/mnt/models/PropellerA-models/qwen3.8-27b-MTP-Q8_0.gguf' |
| 0.09.782.538 W llama_context: n_ctx_seq (16384) < n_ctx_train (262144) -- the full capacity of the model will not be utilized |
| 0.09.882.967 I srv load_model: initializing slots, n_slots = 1 |
| 0.09.900.540 I common_context_can_seq_rm: the context supports bounded partial sequence removal |
| 0.09.935.909 I common_speculative_impl_draft_mtp: adding speculative implementation 'draft-mtp' |
| 0.09.935.915 I common_speculative_impl_draft_mtp: - n_max=3, n_min=0, p_min=0.00, n_embd=5120, backend_sampling=1 |
| 0.09.935.917 I common_speculative_impl_draft_mtp: - gpu_layers=99, cache_k=f16, cache_v=f16, ctx_tgt=yes, ctx_dft=yes, devices=[default] |
| 0.10.076.423 I srv load_model: speculative decoding context initialized |
| 0.10.076.433 I slot load_model: id 0 | task -1 | new slot, n_ctx = 16384 |
| 0.10.076.474 I srv load_model: prompt cache is enabled, size limit: 8192 MiB |
| 0.10.076.474 I srv load_model: use `--cache-ram 0` to disable the prompt cache |
| 0.10.076.475 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391 |
| 0.10.076.475 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256 |
| 0.10.076.499 I srv init: idle slots will be saved to prompt cache upon starting a new task |
| 0.10.097.760 I init: chat template, example_format: '<|im_start|>system |
| Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer. |
|
|
| You are a helpful assistant<|im_end|> |
| <|im_start|>user |
| Hello<|im_end|> |
| <|im_start|>assistant |
| <think> |
|
|
| </think> |
|
|
| Hi there<|im_end|> |
| <|im_start|>user |
| How are you?<|im_end|> |
| <|im_start|>assistant |
| <think> |
| ' |
| 0.10.113.455 I srv init: init: chat template, thinking = 1 |
| 0.10.113.518 I srv llama_server: model loaded |
| 0.10.113.525 I srv llama_server: server is listening on http://127.0.0.1:8291 |
| 0.10.113.533 I srv update_slots: all slots are idle |
| 0.11.041.687 I srv params_from_: Chat format: peg-native |
| 0.11.042.051 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = -1 |
| 0.11.042.056 I srv get_availabl: updating prompt cache |
| 0.11.042.067 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000 |
| 0.11.042.077 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 16384 tokens, 8589934592 est) |
| 0.11.042.084 I srv get_availabl: prompt cache update took 0.03 ms |
| 0.11.042.208 I slot launch_slot_: id 0 | task 0 | processing task, is_child = 0 |
| 0.11.258.356 I slot create_check: id 0 | task 0 | created context checkpoint 1 of 32 (pos_min = 41, pos_max = 41, n_tokens = 42, size = 149.791 MiB) |
| 0.13.546.242 I slot print_timing: id 0 | task 0 | n_decoded = 100, tg = 46.63 t/s |
| 0.16.677.902 I slot print_timing: id 0 | task 0 | n_decoded = 219, tg = 41.51 t/s |
| 0.18.625.541 I slot print_timing: id 0 | task 0 | prompt eval time = 359.38 ms / 107 tokens ( 3.36 ms per token, 297.74 tokens per second) |
| 0.18.625.552 I slot print_timing: id 0 | task 0 | eval time = 7223.89 ms / 293 tokens ( 24.65 ms per token, 40.56 tokens per second) |
| 0.18.625.552 I slot print_timing: id 0 | task 0 | total time = 7583.26 ms / 400 tokens |
| 0.18.625.560 I slot print_timing: id 0 | task 0 | graphs reused = 98 |
| 0.18.625.564 I slot print_timing: id 0 | task 0 | draft acceptance = 0.64983 ( 193 accepted / 297 generated), mean acceptance length = 2.95, acceptance rate per position = (0.788, 0.636, 0.525) |
| 0.18.625.596 I statistics draft-mtp: #calls(b,g,a) = 1 99 99, #gen drafts = 99, #acc drafts = 78, #gen tokens = 297, #acc tokens = 193, #mean acc len = 2.95, #acc rate/pos = (0.788, 0.636, 0.525), dur(b,g,a) = 0.004, 1251.497, 0.179 ms |
| 0.18.625.658 I slot release: id 0 | task 0 | stop processing: n_tokens = 399, truncated = 0 |
| 0.18.625.675 I srv update_slots: all slots are idle |
| 0.18.649.009 I srv params_from_: Chat format: peg-native |
| 0.18.649.296 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.553 (> 0.100 thold), f_keep = 0.143 |
| 0.18.649.299 I srv get_availabl: updating prompt cache |
| 0.18.649.368 W srv prompt_save: - saving prompt with length 399, total state size = 164.448 MiB (draft: 1.566 MiB) |
| 0.18.855.059 I srv load: - looking for better prompt, base f_keep = 0.143, sim = 0.553 |
| 0.18.855.071 I srv update: - cache state: 1 prompts, 314.239 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.18.855.072 I srv update: - prompt 0x5a8a6eef03d0: 399 tokens, checkpoints: 1, 314.239 MiB |
| 0.18.855.073 I srv get_availabl: prompt cache update took 205.77 ms |
| 0.18.855.167 I slot launch_slot_: id 0 | task 103 | processing task, is_child = 0 |
| 0.18.855.181 W slot update_slots: id 0 | task 103 | erased invalidated context checkpoint (pos_min = 41, pos_max = 41, n_tokens = 42, n_swa = 0, pos_next = 0, size = 149.791 MiB) |
| 0.19.045.873 I slot create_check: id 0 | task 103 | created context checkpoint 1 of 32 (pos_min = 41, pos_max = 41, n_tokens = 42, size = 149.791 MiB) |
| 0.21.300.096 I slot print_timing: id 0 | task 103 | n_decoded = 101, tg = 47.70 t/s |
| 0.23.079.032 I slot print_timing: id 0 | task 103 | prompt eval time = 327.58 ms / 103 tokens ( 3.18 ms per token, 314.43 tokens per second) |
| 0.23.079.039 I slot print_timing: id 0 | task 103 | eval time = 3896.22 ms / 176 tokens ( 22.14 ms per token, 45.17 tokens per second) |
| 0.23.079.040 I slot print_timing: id 0 | task 103 | total time = 4223.80 ms / 279 tokens |
| 0.23.079.041 I slot print_timing: id 0 | task 103 | graphs reused = 152 |
| 0.23.079.045 I slot print_timing: id 0 | task 103 | draft acceptance = 0.70833 ( 119 accepted / 168 generated), mean acceptance length = 3.12, acceptance rate per position = (0.857, 0.696, 0.571) |
| 0.23.079.065 I statistics draft-mtp: #calls(b,g,a) = 2 155 155, #gen drafts = 155, #acc drafts = 126, #gen tokens = 465, #acc tokens = 312, #mean acc len = 3.01, #acc rate/pos = (0.813, 0.658, 0.542), dur(b,g,a) = 0.005, 1750.044, 0.278 ms |
| 0.23.079.104 I slot release: id 0 | task 103 | stop processing: n_tokens = 278, truncated = 0 |
| 0.23.079.121 I srv update_slots: all slots are idle |
| 0.23.112.555 I srv params_from_: Chat format: peg-native |
| 0.23.112.975 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.538 (> 0.100 thold), f_keep = 0.205 |
| 0.23.112.979 I srv get_availabl: updating prompt cache |
| 0.23.113.055 W srv prompt_save: - saving prompt with length 278, total state size = 159.954 MiB (draft: 1.091 MiB) |
| 0.23.314.160 I srv load: - looking for better prompt, base f_keep = 0.205, sim = 0.538 |
| 0.23.314.169 I srv update: - cache state: 2 prompts, 623.984 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.23.314.170 I srv update: - prompt 0x5a8a6eef03d0: 399 tokens, checkpoints: 1, 314.239 MiB |
| 0.23.314.171 I srv update: - prompt 0x5a8a6ea256f0: 278 tokens, checkpoints: 1, 309.745 MiB |
| 0.23.314.172 I srv get_availabl: prompt cache update took 201.19 ms |
| 0.23.314.265 I slot launch_slot_: id 0 | task 163 | processing task, is_child = 0 |
| 0.23.314.277 W slot update_slots: id 0 | task 163 | erased invalidated context checkpoint (pos_min = 41, pos_max = 41, n_tokens = 42, n_swa = 0, pos_next = 0, size = 149.791 MiB) |
| 0.23.669.362 I slot create_check: id 0 | task 163 | created context checkpoint 1 of 32 (pos_min = 41, pos_max = 41, n_tokens = 42, size = 149.791 MiB) |
| 0.25.850.289 I slot print_timing: id 0 | task 163 | n_decoded = 100, tg = 48.93 t/s |
| 0.28.878.751 I slot print_timing: id 0 | task 163 | n_decoded = 201, tg = 39.63 t/s |
| 0.31.913.903 I slot print_timing: id 0 | task 163 | n_decoded = 329, tg = 40.58 t/s |
| 0.34.921.045 I slot print_timing: id 0 | task 163 | n_decoded = 438, tg = 39.41 t/s |
| 0.37.926.982 I slot print_timing: id 0 | task 163 | n_decoded = 564, tg = 39.94 t/s |
| 0.40.973.266 I slot print_timing: id 0 | task 163 | n_decoded = 680, tg = 39.61 t/s |
| 0.43.976.603 I slot print_timing: id 0 | task 163 | n_decoded = 795, tg = 39.41 t/s |
| 0.46.389.746 I slot print_timing: id 0 | task 163 | prompt eval time = 492.10 ms / 106 tokens ( 4.64 ms per token, 215.41 tokens per second) |
| 0.46.389.756 I slot print_timing: id 0 | task 163 | eval time = 22583.33 ms / 900 tokens ( 25.09 ms per token, 39.85 tokens per second) |
| 0.46.389.756 I slot print_timing: id 0 | task 163 | total time = 23075.43 ms / 1006 tokens |
| 0.46.389.758 I slot print_timing: id 0 | task 163 | graphs reused = 469 |
| 0.46.389.763 I slot print_timing: id 0 | task 163 | draft acceptance = 0.59855 ( 577 accepted / 964 generated), mean acceptance length = 2.79, acceptance rate per position = (0.801, 0.584, 0.407) |
| 0.46.389.781 I statistics draft-mtp: #calls(b,g,a) = 3 477 477, #gen drafts = 477, #acc drafts = 384, #gen tokens = 1429, #acc tokens = 889, #mean acc len = 2.86, #acc rate/pos = (0.805, 0.608, 0.451), dur(b,g,a) = 0.007, 4962.768, 0.883 ms |
| 0.46.389.819 I slot release: id 0 | task 163 | stop processing: n_tokens = 1005, truncated = 0 |
| 0.46.389.844 I srv update_slots: all slots are idle |
| 0.46.419.054 I srv params_from_: Chat format: peg-native |
| 0.46.419.492 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.366 (> 0.100 thold), f_keep = 0.045 |
| 0.46.419.495 I srv get_availabl: updating prompt cache |
| 0.46.419.600 W srv prompt_save: - saving prompt with length 1005, total state size = 186.960 MiB (draft: 3.945 MiB) |
| 0.46.801.517 I srv load: - looking for better prompt, base f_keep = 0.045, sim = 0.366 |
| 0.46.801.529 I srv update: - cache state: 3 prompts, 960.735 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.46.801.530 I srv update: - prompt 0x5a8a6eef03d0: 399 tokens, checkpoints: 1, 314.239 MiB |
| 0.46.801.531 I srv update: - prompt 0x5a8a6ea256f0: 278 tokens, checkpoints: 1, 309.745 MiB |
| 0.46.801.532 I srv update: - prompt 0x5a8a68af9780: 1005 tokens, checkpoints: 1, 336.751 MiB |
| 0.46.801.533 I srv get_availabl: prompt cache update took 382.04 ms |
| 0.46.801.632 I slot launch_slot_: id 0 | task 489 | processing task, is_child = 0 |
| 0.46.801.646 W slot update_slots: id 0 | task 489 | erased invalidated context checkpoint (pos_min = 41, pos_max = 41, n_tokens = 42, n_swa = 0, pos_next = 0, size = 149.791 MiB) |
| 0.46.991.885 I slot create_check: id 0 | task 489 | created context checkpoint 1 of 32 (pos_min = 41, pos_max = 41, n_tokens = 42, size = 149.791 MiB) |
| 0.49.215.832 I slot print_timing: id 0 | task 489 | n_decoded = 103, tg = 49.82 t/s |
| 0.52.268.375 I slot print_timing: id 0 | task 489 | n_decoded = 209, tg = 40.82 t/s |
| 0.55.316.142 I slot print_timing: id 0 | task 489 | n_decoded = 315, tg = 38.57 t/s |
| 0.58.322.142 I slot print_timing: id 0 | task 489 | n_decoded = 440, tg = 39.38 t/s |
| 1.01.368.777 I slot print_timing: id 0 | task 489 | n_decoded = 556, tg = 39.10 t/s |
| 1.04.414.691 I slot print_timing: id 0 | task 489 | n_decoded = 682, tg = 39.50 t/s |
| 1.05.067.798 I slot print_timing: id 0 | task 489 | prompt eval time = 346.56 ms / 123 tokens ( 2.82 ms per token, 354.92 tokens per second) |
| 1.05.067.809 I slot print_timing: id 0 | task 489 | eval time = 17919.55 ms / 709 tokens ( 25.27 ms per token, 39.57 tokens per second) |
| 1.05.067.810 I slot print_timing: id 0 | task 489 | total time = 18266.11 ms / 832 tokens |
| 1.05.067.811 I slot print_timing: id 0 | task 489 | graphs reused = 714 |
| 1.05.067.815 I slot print_timing: id 0 | task 489 | draft acceptance = 0.61580 ( 460 accepted / 747 generated), mean acceptance length = 2.85, acceptance rate per position = (0.771, 0.590, 0.486) |
| 1.05.067.836 I statistics draft-mtp: #calls(b,g,a) = 4 726 726, #gen drafts = 726, #acc drafts = 576, #gen tokens = 2176, #acc tokens = 1349, #mean acc len = 2.86, #acc rate/pos = (0.793, 0.602, 0.463), dur(b,g,a) = 0.008, 7350.839, 1.263 ms |
| 1.05.067.877 I slot release: id 0 | task 489 | stop processing: n_tokens = 832, truncated = 0 |
| 1.05.067.899 I srv update_slots: all slots are idle |
| 1.05.091.811 I srv params_from_: Chat format: peg-native |
| 1.05.092.105 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.429 (> 0.100 thold), f_keep = 0.054 |
| 1.05.092.109 I srv get_availabl: updating prompt cache |
| 1.05.092.203 W srv prompt_save: - saving prompt with length 832, total state size = 180.533 MiB (draft: 3.266 MiB) |
| 1.05.308.328 I srv load: - looking for better prompt, base f_keep = 0.054, sim = 0.429 |
| 1.05.308.342 I srv update: - cache state: 4 prompts, 1291.059 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.05.308.343 I srv update: - prompt 0x5a8a6eef03d0: 399 tokens, checkpoints: 1, 314.239 MiB |
| 1.05.308.344 I srv update: - prompt 0x5a8a6ea256f0: 278 tokens, checkpoints: 1, 309.745 MiB |
| 1.05.308.345 I srv update: - prompt 0x5a8a68af9780: 1005 tokens, checkpoints: 1, 336.751 MiB |
| 1.05.308.346 I srv update: - prompt 0x5a8a6f1bcea0: 832 tokens, checkpoints: 1, 330.324 MiB |
| 1.05.308.347 I srv get_availabl: prompt cache update took 216.24 ms |
| 1.05.308.447 I slot launch_slot_: id 0 | task 742 | processing task, is_child = 0 |
| 1.05.308.461 W slot update_slots: id 0 | task 742 | erased invalidated context checkpoint (pos_min = 41, pos_max = 41, n_tokens = 42, n_swa = 0, pos_next = 0, size = 149.791 MiB) |
| 1.05.665.405 I slot create_check: id 0 | task 742 | created context checkpoint 1 of 32 (pos_min = 41, pos_max = 41, n_tokens = 42, size = 149.791 MiB) |
| 1.07.915.560 I slot print_timing: id 0 | task 742 | n_decoded = 102, tg = 48.36 t/s |
| 1.10.973.028 I slot print_timing: id 0 | task 742 | n_decoded = 202, tg = 39.10 t/s |
| 1.14.025.165 I slot print_timing: id 0 | task 742 | n_decoded = 324, tg = 39.42 t/s |
| 1.17.058.313 I slot print_timing: id 0 | task 742 | n_decoded = 448, tg = 39.81 t/s |
| 1.20.106.760 I slot print_timing: id 0 | task 742 | n_decoded = 559, tg = 39.09 t/s |
| 1.23.110.171 I slot print_timing: id 0 | task 742 | n_decoded = 665, tg = 38.43 t/s |
| 1.26.154.204 I slot print_timing: id 0 | task 742 | n_decoded = 783, tg = 38.48 t/s |
| 1.29.166.176 I slot print_timing: id 0 | task 742 | n_decoded = 888, tg = 38.01 t/s |
| 1.29.678.311 I slot print_timing: id 0 | task 742 | prompt eval time = 497.70 ms / 105 tokens ( 4.74 ms per token, 210.97 tokens per second) |
| 1.29.678.317 I slot print_timing: id 0 | task 742 | eval time = 23872.13 ms / 900 tokens ( 26.52 ms per token, 37.70 tokens per second) |
| 1.29.678.319 I slot print_timing: id 0 | task 742 | total time = 24369.83 ms / 1005 tokens |
| 1.29.678.320 I slot print_timing: id 0 | task 742 | graphs reused = 1045 |
| 1.29.678.325 I slot print_timing: id 0 | task 742 | draft acceptance = 0.56020 ( 563 accepted / 1005 generated), mean acceptance length = 2.68, acceptance rate per position = (0.743, 0.555, 0.382) |
| 1.29.678.346 I statistics draft-mtp: #calls(b,g,a) = 5 1061 1061, #gen drafts = 1061, #acc drafts = 825, #gen tokens = 3181, #acc tokens = 1912, #mean acc len = 2.80, #acc rate/pos = (0.778, 0.587, 0.437), dur(b,g,a) = 0.009, 11140.206, 1.885 ms |
| 1.29.678.388 I slot release: id 0 | task 742 | stop processing: n_tokens = 1004, truncated = 0 |
| 1.29.678.421 I srv update_slots: all slots are idle |
| 1.29.705.587 I srv params_from_: Chat format: peg-native |
| 1.29.706.023 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.385 (> 0.100 thold), f_keep = 0.045 |
| 1.29.706.026 I srv get_availabl: updating prompt cache |
| 1.29.706.162 W srv prompt_save: - saving prompt with length 1004, total state size = 186.923 MiB (draft: 3.941 MiB) |
| 1.29.925.380 I srv load: - looking for better prompt, base f_keep = 0.045, sim = 0.385 |
| 1.29.925.392 I srv update: - cache state: 5 prompts, 1627.773 MiB (limits: 8192.000 MiB, 16384 tokens, 17704 est) |
| 1.29.925.393 I srv update: - prompt 0x5a8a6eef03d0: 399 tokens, checkpoints: 1, 314.239 MiB |
| 1.29.925.394 I srv update: - prompt 0x5a8a6ea256f0: 278 tokens, checkpoints: 1, 309.745 MiB |
| 1.29.925.394 I srv update: - prompt 0x5a8a68af9780: 1005 tokens, checkpoints: 1, 336.751 MiB |
| 1.29.925.395 I srv update: - prompt 0x5a8a6f1bcea0: 832 tokens, checkpoints: 1, 330.324 MiB |
| 1.29.925.395 I srv update: - prompt 0x5a8a72115320: 1004 tokens, checkpoints: 1, 336.714 MiB |
| 1.29.925.396 I srv get_availabl: prompt cache update took 219.37 ms |
| 1.29.925.493 I slot launch_slot_: id 0 | task 1082 | processing task, is_child = 0 |
| 1.29.925.507 W slot update_slots: id 0 | task 1082 | erased invalidated context checkpoint (pos_min = 41, pos_max = 41, n_tokens = 42, n_swa = 0, pos_next = 0, size = 149.791 MiB) |
| 1.30.115.376 I slot create_check: id 0 | task 1082 | created context checkpoint 1 of 32 (pos_min = 41, pos_max = 41, n_tokens = 42, size = 149.791 MiB) |
| 1.32.346.204 I slot print_timing: id 0 | task 1082 | n_decoded = 103, tg = 49.58 t/s |
| 1.35.376.913 I slot print_timing: id 0 | task 1082 | n_decoded = 219, tg = 42.87 t/s |
| 1.38.386.870 I slot print_timing: id 0 | task 1082 | n_decoded = 347, tg = 42.74 t/s |
| 1.41.414.979 I slot print_timing: id 0 | task 1082 | n_decoded = 465, tg = 41.72 t/s |
| 1.44.468.608 I slot print_timing: id 0 | task 1082 | n_decoded = 559, tg = 39.37 t/s |
| 1.45.726.009 I slot print_timing: id 0 | task 1082 | prompt eval time = 343.10 ms / 117 tokens ( 2.93 ms per token, 341.00 tokens per second) |
| 1.45.726.018 I slot print_timing: id 0 | task 1082 | eval time = 15457.35 ms / 615 tokens ( 25.13 ms per token, 39.79 tokens per second) |
| 1.45.726.019 I slot print_timing: id 0 | task 1082 | total time = 15800.45 ms / 732 tokens |
| 1.45.726.021 I slot print_timing: id 0 | task 1082 | graphs reused = 1255 |
| 1.45.726.026 I slot print_timing: id 0 | task 1082 | draft acceptance = 0.63067 ( 403 accepted / 639 generated), mean acceptance length = 2.89, acceptance rate per position = (0.803, 0.620, 0.469) |
| 1.45.726.046 I statistics draft-mtp: #calls(b,g,a) = 6 1274 1274, #gen drafts = 1274, #acc drafts = 996, #gen tokens = 3820, #acc tokens = 2315, #mean acc len = 2.82, #acc rate/pos = (0.782, 0.593, 0.443), dur(b,g,a) = 0.010, 13453.443, 2.277 ms |
| 1.45.726.083 I slot release: id 0 | task 1082 | stop processing: n_tokens = 733, truncated = 0 |
| 1.45.726.104 I srv update_slots: all slots are idle |
| 1.45.759.152 I srv params_from_: Chat format: peg-native |
| 1.45.759.566 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.597 (> 0.100 thold), f_keep = 0.105 |
| 1.45.759.569 I srv get_availabl: updating prompt cache |
| 1.45.759.664 W srv prompt_save: - saving prompt with length 733, total state size = 176.856 MiB (draft: 2.877 MiB) |
| 1.45.969.501 I srv load: - looking for better prompt, base f_keep = 0.105, sim = 0.597 |
| 1.45.969.512 I srv update: - cache state: 6 prompts, 1954.420 MiB (limits: 8192.000 MiB, 16384 tokens, 17818 est) |
| 1.45.969.513 I srv update: - prompt 0x5a8a6eef03d0: 399 tokens, checkpoints: 1, 314.239 MiB |
| 1.45.969.514 I srv update: - prompt 0x5a8a6ea256f0: 278 tokens, checkpoints: 1, 309.745 MiB |
| 1.45.969.515 I srv update: - prompt 0x5a8a68af9780: 1005 tokens, checkpoints: 1, 336.751 MiB |
| 1.45.969.516 I srv update: - prompt 0x5a8a6f1bcea0: 832 tokens, checkpoints: 1, 330.324 MiB |
| 1.45.969.516 I srv update: - prompt 0x5a8a72115320: 1004 tokens, checkpoints: 1, 336.714 MiB |
| 1.45.969.517 I srv update: - prompt 0x5a8a6eeef220: 733 tokens, checkpoints: 1, 326.647 MiB |
| 1.45.969.518 I srv get_availabl: prompt cache update took 209.95 ms |
| 1.45.969.619 I slot launch_slot_: id 0 | task 1299 | processing task, is_child = 0 |
| 1.45.969.633 W slot update_slots: id 0 | task 1299 | erased invalidated context checkpoint (pos_min = 41, pos_max = 41, n_tokens = 42, n_swa = 0, pos_next = 0, size = 149.791 MiB) |
| 1.46.159.480 I slot create_check: id 0 | task 1299 | created context checkpoint 1 of 32 (pos_min = 41, pos_max = 41, n_tokens = 42, size = 149.791 MiB) |
| 1.48.287.028 I slot print_timing: id 0 | task 1299 | n_decoded = 102, tg = 51.87 t/s |
| 1.51.318.711 I slot print_timing: id 0 | task 1299 | n_decoded = 232, tg = 46.42 t/s |
| 1.54.342.640 I slot print_timing: id 0 | task 1299 | n_decoded = 327, tg = 40.76 t/s |
| 1.57.382.272 I slot print_timing: id 0 | task 1299 | n_decoded = 445, tg = 40.23 t/s |
| 2.00.391.427 I slot print_timing: id 0 | task 1299 | n_decoded = 535, tg = 38.02 t/s |
| 2.00.739.057 I srv operator(): operator(): cleaning up before exit... |
|
|
