| 0.00.545.882 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg) |
| 0.00.545.887 I device_info: |
| 0.00.680.880 I - CUDA0 : NVIDIA GeForce RTX 3080 (20052 MiB, 19814 MiB free) |
| 0.00.811.843 I - CUDA1 : NVIDIA GeForce RTX 3080 (20052 MiB, 19814 MiB free) |
| 0.00.939.459 I - CUDA2 : NVIDIA GeForce RTX 3080 (20052 MiB, 19814 MiB free) |
| 0.00.939.480 I - CPU : AMD EPYC 7F32 8-Core Processor (193092 MiB, 193092 MiB free) |
| 0.00.939.624 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CUDA : ARCHS = 860 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | |
| 0.00.939.776 I srv init: using 15 threads for HTTP server |
| 0.00.940.297 I srv start: binding port with default address family |
| 0.00.941.647 I srv llama_server: loading model |
| 0.00.941.659 I srv load_model: loading model '/mnt/models/PropellerA-models/Qwen3.6-27B-Q6_K.gguf' |
| 0.00.941.751 I common_init_result: fitting params to device memory ... |
| 0.00.941.754 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on) |
| 0.20.672.308 W llama_context: n_ctx_seq (16384) < n_ctx_train (262144) -- the full capacity of the model will not be utilized |
| 0.20.773.174 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable) |
| 0.20.900.548 I srv load_model: initializing slots, n_slots = 1 |
| 0.20.946.037 W srv load_model: speculative decoding will use checkpoints |
| 0.20.946.056 W common_speculative_init: no implementations specified for speculative decoding |
| 0.20.946.058 I slot load_model: id 0 | task -1 | new slot, n_ctx = 16384 |
| 0.20.946.171 I srv load_model: prompt cache is enabled, size limit: 8192 MiB |
| 0.20.946.173 I srv load_model: use `--cache-ram 0` to disable the prompt cache |
| 0.20.946.174 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391 |
| 0.20.946.174 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256 |
| 0.20.946.209 I srv init: idle slots will be saved to prompt cache upon starting a new task |
| 0.20.968.545 I init: chat template, example_format: '<|im_start|>system |
| You are a helpful assistant<|im_end|> |
| <|im_start|>user |
| Hello<|im_end|> |
| <|im_start|>assistant |
| Hi there<|im_end|> |
| <|im_start|>user |
| How are you?<|im_end|> |
| <|im_start|>assistant |
| <think> |
| ' |
| 0.20.985.260 I srv init: init: chat template, thinking = 1 |
| 0.20.985.322 I srv llama_server: model loaded |
| 0.20.985.327 I srv llama_server: server is listening on http://127.0.0.1:8291 |
| 0.20.985.336 I srv update_slots: all slots are idle |
| 0.22.388.544 I srv params_from_: Chat format: peg-native |
| 0.22.388.847 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = -1 |
| 0.22.388.852 I srv get_availabl: updating prompt cache |
| 0.22.388.866 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000 |
| 0.22.388.877 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 16384 tokens, 8589934592 est) |
| 0.22.388.882 I srv get_availabl: prompt cache update took 0.03 ms |
| 0.22.389.064 I slot launch_slot_: id 0 | task 0 | processing task, is_child = 0 |
| 0.22.662.610 I slot create_check: id 0 | task 0 | created context checkpoint 1 of 32 (pos_min = 62, pos_max = 62, n_tokens = 63, size = 149.626 MiB) |
| 0.22.949.494 I slot print_timing: id 0 | task 0 | prompt eval time = 333.23 ms / 67 tokens ( 4.97 ms per token, 201.06 tokens per second) |
| 0.22.949.500 I slot print_timing: id 0 | task 0 | eval time = 227.16 ms / 7 tokens ( 32.45 ms per token, 30.82 tokens per second) |
| 0.22.949.501 I slot print_timing: id 0 | task 0 | total time = 560.39 ms / 74 tokens |
| 0.22.949.508 I slot print_timing: id 0 | task 0 | graphs reused = 6 |
| 0.22.949.565 I slot release: id 0 | task 0 | stop processing: n_tokens = 73, truncated = 0 |
| 0.22.949.571 I srv update_slots: all slots are idle |
| 0.22.981.985 I srv params_from_: Chat format: peg-native |
| 0.22.982.237 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.238 (> 0.100 thold), f_keep = 0.205 |
| 0.22.982.241 I srv get_availabl: updating prompt cache |
| 0.22.982.276 W srv prompt_save: - saving prompt with length 73, total state size = 152.052 MiB (draft: 0.000 MiB) |
| 0.23.423.463 I srv load: - looking for better prompt, base f_keep = 0.205, sim = 0.238 |
| 0.23.423.475 I srv update: - cache state: 1 prompts, 301.678 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.23.423.476 I srv update: - prompt 0x58989dd500f0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.23.423.478 I srv get_availabl: prompt cache update took 441.24 ms |
| 0.23.423.584 I slot launch_slot_: id 0 | task 9 | processing task, is_child = 0 |
| 0.23.423.597 W slot update_slots: id 0 | task 9 | erased invalidated context checkpoint (pos_min = 62, pos_max = 62, n_tokens = 63, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.23.546.439 I slot create_check: id 0 | task 9 | created context checkpoint 1 of 32 (pos_min = 58, pos_max = 58, n_tokens = 59, size = 149.626 MiB) |
| 0.23.894.188 I slot print_timing: id 0 | task 9 | prompt eval time = 174.69 ms / 63 tokens ( 2.77 ms per token, 360.63 tokens per second) |
| 0.23.894.197 I slot print_timing: id 0 | task 9 | eval time = 295.89 ms / 9 tokens ( 32.88 ms per token, 30.42 tokens per second) |
| 0.23.894.197 I slot print_timing: id 0 | task 9 | total time = 470.58 ms / 72 tokens |
| 0.23.894.198 I slot print_timing: id 0 | task 9 | graphs reused = 13 |
| 0.23.894.233 I slot release: id 0 | task 9 | stop processing: n_tokens = 71, truncated = 0 |
| 0.23.894.245 I srv update_slots: all slots are idle |
| 0.23.929.446 I srv params_from_: Chat format: peg-native |
| 0.23.929.686 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.227 (> 0.100 thold), f_keep = 0.211 |
| 0.23.929.689 I srv get_availabl: updating prompt cache |
| 0.23.929.715 W srv prompt_save: - saving prompt with length 71, total state size = 151.985 MiB (draft: 0.000 MiB) |
| 0.24.124.734 I srv load: - looking for better prompt, base f_keep = 0.211, sim = 0.227 |
| 0.24.124.748 I srv update: - cache state: 2 prompts, 603.289 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.24.124.749 I srv update: - prompt 0x58989dd500f0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.24.124.750 I srv update: - prompt 0x58989381fee0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.24.124.751 I srv get_availabl: prompt cache update took 195.06 ms |
| 0.24.124.854 I slot launch_slot_: id 0 | task 20 | processing task, is_child = 0 |
| 0.24.124.866 W slot update_slots: id 0 | task 20 | erased invalidated context checkpoint (pos_min = 58, pos_max = 58, n_tokens = 59, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.24.436.238 I slot create_check: id 0 | task 20 | created context checkpoint 1 of 32 (pos_min = 61, pos_max = 61, n_tokens = 62, size = 149.626 MiB) |
| 0.24.819.816 I slot print_timing: id 0 | task 20 | prompt eval time = 363.24 ms / 66 tokens ( 5.50 ms per token, 181.70 tokens per second) |
| 0.24.819.823 I slot print_timing: id 0 | task 20 | eval time = 331.71 ms / 10 tokens ( 33.17 ms per token, 30.15 tokens per second) |
| 0.24.819.823 I slot print_timing: id 0 | task 20 | total time = 694.94 ms / 76 tokens |
| 0.24.819.824 I slot print_timing: id 0 | task 20 | graphs reused = 21 |
| 0.24.819.851 I slot release: id 0 | task 20 | stop processing: n_tokens = 75, truncated = 0 |
| 0.24.819.862 I srv update_slots: all slots are idle |
| 0.24.845.371 I srv params_from_: Chat format: peg-native |
| 0.24.845.623 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17687162632 |
| 0.24.845.626 I srv get_availabl: updating prompt cache |
| 0.24.845.658 W srv prompt_save: - saving prompt with length 75, total state size = 152.118 MiB (draft: 0.000 MiB) |
| 0.25.038.870 I srv load: - looking for better prompt, base f_keep = 0.040, sim = 0.036 |
| 0.25.038.882 I srv update: - cache state: 3 prompts, 905.034 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.25.038.883 I srv update: - prompt 0x58989dd500f0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.25.038.884 I srv update: - prompt 0x58989381fee0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.25.038.884 I srv update: - prompt 0x5898934dfd20: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.25.038.886 I srv get_availabl: prompt cache update took 193.26 ms |
| 0.25.038.993 I slot launch_slot_: id 0 | task 32 | processing task, is_child = 0 |
| 0.25.039.006 W slot update_slots: id 0 | task 32 | erased invalidated context checkpoint (pos_min = 61, pos_max = 61, n_tokens = 62, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.25.392.803 I slot create_check: id 0 | task 32 | created context checkpoint 1 of 32 (pos_min = 78, pos_max = 78, n_tokens = 79, size = 149.626 MiB) |
| 0.26.065.839 I slot print_timing: id 0 | task 32 | prompt eval time = 405.64 ms / 83 tokens ( 4.89 ms per token, 204.62 tokens per second) |
| 0.26.065.845 I slot print_timing: id 0 | task 32 | eval time = 621.19 ms / 18 tokens ( 34.51 ms per token, 28.98 tokens per second) |
| 0.26.065.845 I slot print_timing: id 0 | task 32 | total time = 1026.83 ms / 101 tokens |
| 0.26.065.846 I slot print_timing: id 0 | task 32 | graphs reused = 37 |
| 0.26.065.872 I slot release: id 0 | task 32 | stop processing: n_tokens = 100, truncated = 0 |
| 0.26.065.884 I srv update_slots: all slots are idle |
| 0.26.099.135 I srv params_from_: Chat format: peg-native |
| 0.26.099.386 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17688408653 |
| 0.26.099.388 I srv get_availabl: updating prompt cache |
| 0.26.099.415 W srv prompt_save: - saving prompt with length 100, total state size = 152.949 MiB (draft: 0.000 MiB) |
| 0.26.462.584 I srv load: - looking for better prompt, base f_keep = 0.030, sim = 0.046 |
| 0.26.462.598 I srv update: - cache state: 4 prompts, 1207.608 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.26.462.599 I srv update: - prompt 0x58989dd500f0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.26.462.601 I srv update: - prompt 0x58989381fee0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.26.462.601 I srv update: - prompt 0x5898934dfd20: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.26.462.602 I srv update: - prompt 0x58989f8ed210: 100 tokens, checkpoints: 1, 302.575 MiB |
| 0.26.462.603 I srv get_availabl: prompt cache update took 363.21 ms |
| 0.26.462.703 I slot launch_slot_: id 0 | task 52 | processing task, is_child = 0 |
| 0.26.462.716 W slot update_slots: id 0 | task 52 | erased invalidated context checkpoint (pos_min = 78, pos_max = 78, n_tokens = 79, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.26.584.419 I slot create_check: id 0 | task 52 | created context checkpoint 1 of 32 (pos_min = 60, pos_max = 60, n_tokens = 61, size = 149.626 MiB) |
| 0.27.040.401 I slot print_timing: id 0 | task 52 | prompt eval time = 173.60 ms / 65 tokens ( 2.67 ms per token, 374.42 tokens per second) |
| 0.27.040.407 I slot print_timing: id 0 | task 52 | eval time = 404.08 ms / 12 tokens ( 33.67 ms per token, 29.70 tokens per second) |
| 0.27.040.408 I slot print_timing: id 0 | task 52 | total time = 577.68 ms / 77 tokens |
| 0.27.040.409 I slot print_timing: id 0 | task 52 | graphs reused = 47 |
| 0.27.040.433 I slot release: id 0 | task 52 | stop processing: n_tokens = 76, truncated = 0 |
| 0.27.040.443 I srv update_slots: all slots are idle |
| 0.27.073.367 I srv params_from_: Chat format: peg-native |
| 0.27.073.598 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17689383214 |
| 0.27.073.600 I srv get_availabl: updating prompt cache |
| 0.27.073.627 W srv prompt_save: - saving prompt with length 76, total state size = 152.151 MiB (draft: 0.000 MiB) |
| 0.27.443.163 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.039 |
| 0.27.443.169 I srv load: - found better prompt with f_keep = 0.340, sim = 0.442 |
| 0.27.490.764 I srv update: - cache state: 4 prompts, 1206.811 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.27.490.771 I srv update: - prompt 0x58989dd500f0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.27.490.772 I srv update: - prompt 0x58989381fee0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.27.490.773 I srv update: - prompt 0x5898934dfd20: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.27.490.773 I srv update: - prompt 0x589893eb5090: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.27.490.775 I srv get_availabl: prompt cache update took 417.17 ms |
| 0.27.490.886 I slot launch_slot_: id 0 | task 66 | processing task, is_child = 0 |
| 0.27.490.900 W slot update_slots: id 0 | task 66 | erased invalidated context checkpoint (pos_min = 78, pos_max = 78, n_tokens = 79, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.27.623.135 I slot create_check: id 0 | task 66 | created context checkpoint 1 of 32 (pos_min = 72, pos_max = 72, n_tokens = 73, size = 149.626 MiB) |
| 0.28.657.184 I slot print_timing: id 0 | task 66 | prompt eval time = 184.14 ms / 77 tokens ( 2.39 ms per token, 418.15 tokens per second) |
| 0.28.657.191 I slot print_timing: id 0 | task 66 | eval time = 982.13 ms / 23 tokens ( 42.70 ms per token, 23.42 tokens per second) |
| 0.28.657.192 I slot print_timing: id 0 | task 66 | total time = 1166.28 ms / 100 tokens |
| 0.28.657.192 I slot print_timing: id 0 | task 66 | graphs reused = 68 |
| 0.28.657.229 I slot release: id 0 | task 66 | stop processing: n_tokens = 99, truncated = 0 |
| 0.28.657.243 I srv update_slots: all slots are idle |
| 0.28.691.972 I srv params_from_: Chat format: peg-native |
| 0.28.692.212 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.393 (> 0.100 thold), f_keep = 0.354 |
| 0.28.692.215 I srv get_availabl: updating prompt cache |
| 0.28.692.251 W srv prompt_save: - saving prompt with length 99, total state size = 152.916 MiB (draft: 0.000 MiB) |
| 0.28.888.715 I srv load: - looking for better prompt, base f_keep = 0.354, sim = 0.393 |
| 0.28.888.725 I srv update: - cache state: 5 prompts, 1509.353 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.28.888.726 I srv update: - prompt 0x58989dd500f0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.28.888.727 I srv update: - prompt 0x58989381fee0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.28.888.727 I srv update: - prompt 0x5898934dfd20: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.28.888.728 I srv update: - prompt 0x589893eb5090: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.28.888.729 I srv update: - prompt 0x58989dd50b10: 99 tokens, checkpoints: 1, 302.542 MiB |
| 0.28.888.730 I srv get_availabl: prompt cache update took 196.51 ms |
| 0.28.888.834 I slot launch_slot_: id 0 | task 91 | processing task, is_child = 0 |
| 0.28.888.853 W slot update_slots: id 0 | task 91 | erased invalidated context checkpoint (pos_min = 72, pos_max = 72, n_tokens = 73, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.29.077.404 I slot create_check: id 0 | task 91 | created context checkpoint 1 of 32 (pos_min = 84, pos_max = 84, n_tokens = 85, size = 149.626 MiB) |
| 0.29.959.077 I slot print_timing: id 0 | task 91 | prompt eval time = 241.63 ms / 89 tokens ( 2.71 ms per token, 368.33 tokens per second) |
| 0.29.959.085 I slot print_timing: id 0 | task 91 | eval time = 828.57 ms / 19 tokens ( 43.61 ms per token, 22.93 tokens per second) |
| 0.29.959.086 I slot print_timing: id 0 | task 91 | total time = 1070.20 ms / 108 tokens |
| 0.29.959.088 I slot print_timing: id 0 | task 91 | graphs reused = 85 |
| 0.29.959.133 I slot release: id 0 | task 91 | stop processing: n_tokens = 107, truncated = 0 |
| 0.29.959.153 I srv update_slots: all slots are idle |
| 0.29.994.899 I srv params_from_: Chat format: peg-native |
| 0.29.995.147 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17692301914 |
| 0.29.995.149 I srv get_availabl: updating prompt cache |
| 0.29.995.188 W srv prompt_save: - saving prompt with length 107, total state size = 153.181 MiB (draft: 0.000 MiB) |
| 0.30.356.813 I srv load: - looking for better prompt, base f_keep = 0.028, sim = 0.050 |
| 0.30.356.825 I srv update: - cache state: 6 prompts, 1812.160 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.30.356.826 I srv update: - prompt 0x58989dd500f0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.30.356.827 I srv update: - prompt 0x58989381fee0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.30.356.828 I srv update: - prompt 0x5898934dfd20: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.30.356.828 I srv update: - prompt 0x589893eb5090: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.30.356.829 I srv update: - prompt 0x58989dd50b10: 99 tokens, checkpoints: 1, 302.542 MiB |
| 0.30.356.830 I srv update: - prompt 0x5898930b5ed0: 107 tokens, checkpoints: 1, 302.807 MiB |
| 0.30.356.831 I srv get_availabl: prompt cache update took 361.68 ms |
| 0.30.356.931 I slot launch_slot_: id 0 | task 112 | processing task, is_child = 0 |
| 0.30.356.943 W slot update_slots: id 0 | task 112 | erased invalidated context checkpoint (pos_min = 84, pos_max = 84, n_tokens = 85, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.30.476.779 I slot create_check: id 0 | task 112 | created context checkpoint 1 of 32 (pos_min = 55, pos_max = 55, n_tokens = 56, size = 149.626 MiB) |
| 0.30.824.239 I slot print_timing: id 0 | task 112 | prompt eval time = 171.76 ms / 60 tokens ( 2.86 ms per token, 349.32 tokens per second) |
| 0.30.824.248 I slot print_timing: id 0 | task 112 | eval time = 295.53 ms / 9 tokens ( 32.84 ms per token, 30.45 tokens per second) |
| 0.30.824.248 I slot print_timing: id 0 | task 112 | total time = 467.29 ms / 69 tokens |
| 0.30.824.249 I slot print_timing: id 0 | task 112 | graphs reused = 92 |
| 0.30.824.274 I slot release: id 0 | task 112 | stop processing: n_tokens = 68, truncated = 0 |
| 0.30.824.283 I srv update_slots: all slots are idle |
| 0.30.855.007 I srv params_from_: Chat format: peg-native |
| 0.30.855.178 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.183 (> 0.100 thold), f_keep = 0.221 |
| 0.30.855.182 I srv get_availabl: updating prompt cache |
| 0.30.855.234 W srv prompt_save: - saving prompt with length 68, total state size = 151.886 MiB (draft: 0.000 MiB) |
| 0.31.050.606 I srv load: - looking for better prompt, base f_keep = 0.221, sim = 0.183 |
| 0.31.050.619 I srv update: - cache state: 7 prompts, 2113.672 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.31.050.620 I srv update: - prompt 0x58989dd500f0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.31.050.621 I srv update: - prompt 0x58989381fee0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.31.050.622 I srv update: - prompt 0x5898934dfd20: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.31.050.623 I srv update: - prompt 0x589893eb5090: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.31.050.623 I srv update: - prompt 0x58989dd50b10: 99 tokens, checkpoints: 1, 302.542 MiB |
| 0.31.050.624 I srv update: - prompt 0x5898930b5ed0: 107 tokens, checkpoints: 1, 302.807 MiB |
| 0.31.050.625 I srv update: - prompt 0x589893e686f0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.31.050.626 I srv get_availabl: prompt cache update took 195.44 ms |
| 0.31.050.728 I slot launch_slot_: id 0 | task 123 | processing task, is_child = 0 |
| 0.31.050.739 W slot update_slots: id 0 | task 123 | erased invalidated context checkpoint (pos_min = 55, pos_max = 55, n_tokens = 56, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.31.347.372 I slot create_check: id 0 | task 123 | created context checkpoint 1 of 32 (pos_min = 77, pos_max = 77, n_tokens = 78, size = 149.626 MiB) |
| 0.31.948.095 I slot print_timing: id 0 | task 123 | prompt eval time = 348.49 ms / 82 tokens ( 4.25 ms per token, 235.30 tokens per second) |
| 0.31.948.102 I slot print_timing: id 0 | task 123 | eval time = 548.86 ms / 16 tokens ( 34.30 ms per token, 29.15 tokens per second) |
| 0.31.948.103 I slot print_timing: id 0 | task 123 | total time = 897.35 ms / 98 tokens |
| 0.31.948.104 I slot print_timing: id 0 | task 123 | graphs reused = 106 |
| 0.31.948.140 I slot release: id 0 | task 123 | stop processing: n_tokens = 97, truncated = 0 |
| 0.31.948.152 I srv update_slots: all slots are idle |
| 0.31.980.466 I srv params_from_: Chat format: peg-native |
| 0.31.980.698 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17694290921 |
| 0.31.980.700 I srv get_availabl: updating prompt cache |
| 0.31.980.733 W srv prompt_save: - saving prompt with length 97, total state size = 152.849 MiB (draft: 0.000 MiB) |
| 0.32.372.298 I srv load: - looking for better prompt, base f_keep = 0.031, sim = 0.033 |
| 0.32.372.308 I srv load: - found better prompt with f_keep = 0.343, sim = 0.374 |
| 0.32.423.386 I srv update: - cache state: 7 prompts, 2113.605 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.32.423.393 I srv update: - prompt 0x58989dd500f0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.32.423.394 I srv update: - prompt 0x58989381fee0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.32.423.395 I srv update: - prompt 0x5898934dfd20: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.32.423.396 I srv update: - prompt 0x589893eb5090: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.32.423.396 I srv update: - prompt 0x5898930b5ed0: 107 tokens, checkpoints: 1, 302.807 MiB |
| 0.32.423.397 I srv update: - prompt 0x589893e686f0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.32.423.398 I srv update: - prompt 0x58989dd503b0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.32.423.400 I srv get_availabl: prompt cache update took 442.70 ms |
| 0.32.423.518 I slot launch_slot_: id 0 | task 141 | processing task, is_child = 0 |
| 0.32.423.531 W slot update_slots: id 0 | task 141 | erased invalidated context checkpoint (pos_min = 72, pos_max = 72, n_tokens = 73, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.32.572.341 I slot create_check: id 0 | task 141 | created context checkpoint 1 of 32 (pos_min = 86, pos_max = 86, n_tokens = 87, size = 149.626 MiB) |
| 0.33.732.483 I slot print_timing: id 0 | task 141 | prompt eval time = 200.61 ms / 91 tokens ( 2.20 ms per token, 453.61 tokens per second) |
| 0.33.732.489 I slot print_timing: id 0 | task 141 | eval time = 1108.33 ms / 27 tokens ( 41.05 ms per token, 24.36 tokens per second) |
| 0.33.732.490 I slot print_timing: id 0 | task 141 | total time = 1308.95 ms / 118 tokens |
| 0.33.732.491 I slot print_timing: id 0 | task 141 | graphs reused = 131 |
| 0.33.732.527 I slot release: id 0 | task 141 | stop processing: n_tokens = 117, truncated = 0 |
| 0.33.732.540 I srv update_slots: all slots are idle |
| 0.33.765.519 I srv params_from_: Chat format: peg-native |
| 0.33.765.747 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17696075308 |
| 0.33.765.749 I srv get_availabl: updating prompt cache |
| 0.33.765.784 W srv prompt_save: - saving prompt with length 117, total state size = 153.514 MiB (draft: 0.000 MiB) |
| 0.33.961.566 I srv load: - looking for better prompt, base f_keep = 0.026, sim = 0.024 |
| 0.33.961.580 I srv update: - cache state: 8 prompts, 2416.745 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.33.961.582 I srv update: - prompt 0x58989dd500f0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.33.961.583 I srv update: - prompt 0x58989381fee0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.33.961.584 I srv update: - prompt 0x5898934dfd20: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.33.961.585 I srv update: - prompt 0x589893eb5090: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.33.961.585 I srv update: - prompt 0x5898930b5ed0: 107 tokens, checkpoints: 1, 302.807 MiB |
| 0.33.961.586 I srv update: - prompt 0x589893e686f0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.33.961.587 I srv update: - prompt 0x58989dd503b0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.33.961.588 I srv update: - prompt 0x589893e6cba0: 117 tokens, checkpoints: 1, 303.140 MiB |
| 0.33.961.589 I srv get_availabl: prompt cache update took 195.84 ms |
| 0.33.961.705 I slot launch_slot_: id 0 | task 170 | processing task, is_child = 0 |
| 0.33.961.718 W slot update_slots: id 0 | task 170 | erased invalidated context checkpoint (pos_min = 86, pos_max = 86, n_tokens = 87, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.34.338.978 I slot create_check: id 0 | task 170 | created context checkpoint 1 of 32 (pos_min = 119, pos_max = 119, n_tokens = 120, size = 149.626 MiB) |
| 0.38.652.169 I slot print_timing: id 0 | task 170 | n_decoded = 100, tg = 23.47 t/s |
| 0.41.668.604 I slot print_timing: id 0 | task 170 | n_decoded = 169, tg = 23.22 t/s |
| 0.41.813.894 I slot print_timing: id 0 | task 170 | prompt eval time = 429.09 ms / 124 tokens ( 3.46 ms per token, 288.99 tokens per second) |
| 0.41.813.900 I slot print_timing: id 0 | task 170 | eval time = 7423.08 ms / 173 tokens ( 42.91 ms per token, 23.31 tokens per second) |
| 0.41.813.901 I slot print_timing: id 0 | task 170 | total time = 7852.17 ms / 297 tokens |
| 0.41.813.902 I slot print_timing: id 0 | task 170 | graphs reused = 301 |
| 0.41.813.939 I slot release: id 0 | task 170 | stop processing: n_tokens = 296, truncated = 0 |
| 0.41.813.953 I srv update_slots: all slots are idle |
| 0.41.851.246 I srv params_from_: Chat format: peg-native |
| 0.41.851.419 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.129 (> 0.100 thold), f_keep = 0.041 |
| 0.41.851.423 I srv get_availabl: updating prompt cache |
| 0.41.851.461 W srv prompt_save: - saving prompt with length 296, total state size = 159.460 MiB (draft: 0.000 MiB) |
| 0.42.050.001 I srv load: - looking for better prompt, base f_keep = 0.041, sim = 0.129 |
| 0.42.050.016 I srv update: - cache state: 9 prompts, 2725.831 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.42.050.018 I srv update: - prompt 0x58989dd500f0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.42.050.019 I srv update: - prompt 0x58989381fee0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.42.050.020 I srv update: - prompt 0x5898934dfd20: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.42.050.021 I srv update: - prompt 0x589893eb5090: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.42.050.022 I srv update: - prompt 0x5898930b5ed0: 107 tokens, checkpoints: 1, 302.807 MiB |
| 0.42.050.023 I srv update: - prompt 0x589893e686f0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.42.050.023 I srv update: - prompt 0x58989dd503b0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.42.050.024 I srv update: - prompt 0x589893e6cba0: 117 tokens, checkpoints: 1, 303.140 MiB |
| 0.42.050.025 I srv update: - prompt 0x58989386eb00: 296 tokens, checkpoints: 1, 309.086 MiB |
| 0.42.050.029 I srv get_availabl: prompt cache update took 198.60 ms |
| 0.42.050.151 I slot launch_slot_: id 0 | task 345 | processing task, is_child = 0 |
| 0.42.050.165 W slot update_slots: id 0 | task 345 | erased invalidated context checkpoint (pos_min = 119, pos_max = 119, n_tokens = 120, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.42.434.680 I slot create_check: id 0 | task 345 | created context checkpoint 1 of 32 (pos_min = 88, pos_max = 88, n_tokens = 89, size = 149.626 MiB) |
| 0.46.692.096 I slot print_timing: id 0 | task 345 | prompt eval time = 436.49 ms / 93 tokens ( 4.69 ms per token, 213.07 tokens per second) |
| 0.46.692.102 I slot print_timing: id 0 | task 345 | eval time = 4205.44 ms / 99 tokens ( 42.48 ms per token, 23.54 tokens per second) |
| 0.46.692.103 I slot print_timing: id 0 | task 345 | total time = 4641.93 ms / 192 tokens |
| 0.46.692.104 I slot print_timing: id 0 | task 345 | graphs reused = 398 |
| 0.46.692.139 I slot release: id 0 | task 345 | stop processing: n_tokens = 191, truncated = 0 |
| 0.46.692.153 I srv update_slots: all slots are idle |
| 0.46.746.891 I srv params_from_: Chat format: peg-native |
| 0.46.747.191 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.138 (> 0.100 thold), f_keep = 0.063 |
| 0.46.747.195 I srv get_availabl: updating prompt cache |
| 0.46.747.245 W srv prompt_save: - saving prompt with length 191, total state size = 155.972 MiB (draft: 0.000 MiB) |
| 0.46.945.649 I srv load: - looking for better prompt, base f_keep = 0.063, sim = 0.138 |
| 0.46.945.661 I srv update: - cache state: 10 prompts, 3031.430 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.46.945.662 I srv update: - prompt 0x58989dd500f0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.46.945.663 I srv update: - prompt 0x58989381fee0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.46.945.664 I srv update: - prompt 0x5898934dfd20: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.46.945.665 I srv update: - prompt 0x589893eb5090: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.46.945.665 I srv update: - prompt 0x5898930b5ed0: 107 tokens, checkpoints: 1, 302.807 MiB |
| 0.46.945.666 I srv update: - prompt 0x589893e686f0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.46.945.667 I srv update: - prompt 0x58989dd503b0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.46.945.667 I srv update: - prompt 0x589893e6cba0: 117 tokens, checkpoints: 1, 303.140 MiB |
| 0.46.945.668 I srv update: - prompt 0x58989386eb00: 296 tokens, checkpoints: 1, 309.086 MiB |
| 0.46.945.668 I srv update: - prompt 0x58987fd51970: 191 tokens, checkpoints: 1, 305.598 MiB |
| 0.46.945.670 I srv get_availabl: prompt cache update took 198.47 ms |
| 0.46.945.778 I slot launch_slot_: id 0 | task 446 | processing task, is_child = 0 |
| 0.46.945.791 W slot update_slots: id 0 | task 446 | erased invalidated context checkpoint (pos_min = 88, pos_max = 88, n_tokens = 89, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.47.090.717 I slot create_check: id 0 | task 446 | created context checkpoint 1 of 32 (pos_min = 82, pos_max = 82, n_tokens = 83, size = 149.626 MiB) |
| 0.51.556.141 I slot print_timing: id 0 | task 446 | n_decoded = 100, tg = 23.55 t/s |
| 0.54.589.737 I slot print_timing: id 0 | task 446 | n_decoded = 170, tg = 23.35 t/s |
| 0.57.052.347 I slot print_timing: id 0 | task 446 | prompt eval time = 364.88 ms / 87 tokens ( 4.19 ms per token, 238.43 tokens per second) |
| 0.57.052.353 I slot print_timing: id 0 | task 446 | eval time = 9741.66 ms / 229 tokens ( 42.54 ms per token, 23.51 tokens per second) |
| 0.57.052.354 I slot print_timing: id 0 | task 446 | total time = 10106.55 ms / 316 tokens |
| 0.57.052.355 I slot print_timing: id 0 | task 446 | graphs reused = 624 |
| 0.57.052.385 I slot release: id 0 | task 446 | stop processing: n_tokens = 315, truncated = 0 |
| 0.57.052.398 I srv update_slots: all slots are idle |
| 0.57.104.163 I srv params_from_: Chat format: peg-native |
| 0.57.104.403 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.038 |
| 0.57.104.405 I srv get_availabl: updating prompt cache |
| 0.57.104.442 W srv prompt_save: - saving prompt with length 315, total state size = 160.091 MiB (draft: 0.000 MiB) |
| 0.57.472.122 I srv load: - looking for better prompt, base f_keep = 0.038, sim = 0.107 |
| 0.57.472.139 I srv update: - cache state: 11 prompts, 3341.147 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.57.472.140 I srv update: - prompt 0x58989dd500f0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.57.472.140 I srv update: - prompt 0x58989381fee0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.57.472.141 I srv update: - prompt 0x5898934dfd20: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.57.472.142 I srv update: - prompt 0x589893eb5090: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.57.472.143 I srv update: - prompt 0x5898930b5ed0: 107 tokens, checkpoints: 1, 302.807 MiB |
| 0.57.472.143 I srv update: - prompt 0x589893e686f0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.57.472.144 I srv update: - prompt 0x58989dd503b0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.57.472.144 I srv update: - prompt 0x589893e6cba0: 117 tokens, checkpoints: 1, 303.140 MiB |
| 0.57.472.145 I srv update: - prompt 0x58989386eb00: 296 tokens, checkpoints: 1, 309.086 MiB |
| 0.57.472.145 I srv update: - prompt 0x58987fd51970: 191 tokens, checkpoints: 1, 305.598 MiB |
| 0.57.472.146 I srv update: - prompt 0x5898940d1060: 315 tokens, checkpoints: 1, 309.718 MiB |
| 0.57.472.148 I srv get_availabl: prompt cache update took 367.74 ms |
| 0.57.472.261 I slot launch_slot_: id 0 | task 677 | processing task, is_child = 0 |
| 0.57.472.272 W slot update_slots: id 0 | task 677 | erased invalidated context checkpoint (pos_min = 82, pos_max = 82, n_tokens = 83, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.57.679.934 I slot create_check: id 0 | task 677 | created context checkpoint 1 of 32 (pos_min = 107, pos_max = 107, n_tokens = 108, size = 149.626 MiB) |
| 1.01.082.433 I slot print_timing: id 0 | task 677 | prompt eval time = 259.69 ms / 112 tokens ( 2.32 ms per token, 431.28 tokens per second) |
| 1.01.082.439 I slot print_timing: id 0 | task 677 | eval time = 3350.46 ms / 80 tokens ( 41.88 ms per token, 23.88 tokens per second) |
| 1.01.082.440 I slot print_timing: id 0 | task 677 | total time = 3610.15 ms / 192 tokens |
| 1.01.082.441 I slot print_timing: id 0 | task 677 | graphs reused = 702 |
| 1.01.082.472 I slot release: id 0 | task 677 | stop processing: n_tokens = 191, truncated = 0 |
| 1.01.082.485 I srv update_slots: all slots are idle |
| 1.01.317.836 I srv params_from_: Chat format: peg-native |
| 1.01.318.030 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.136 (> 0.100 thold), f_keep = 0.063 |
| 1.01.318.035 I srv get_availabl: updating prompt cache |
| 1.01.318.083 W srv prompt_save: - saving prompt with length 191, total state size = 155.972 MiB (draft: 0.000 MiB) |
| 1.01.511.291 I srv load: - looking for better prompt, base f_keep = 0.063, sim = 0.136 |
| 1.01.511.306 I srv update: - cache state: 12 prompts, 3646.745 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.01.511.307 I srv update: - prompt 0x58989dd500f0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.01.511.308 I srv update: - prompt 0x58989381fee0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.01.511.308 I srv update: - prompt 0x5898934dfd20: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.01.511.309 I srv update: - prompt 0x589893eb5090: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.01.511.310 I srv update: - prompt 0x5898930b5ed0: 107 tokens, checkpoints: 1, 302.807 MiB |
| 1.01.511.312 I srv update: - prompt 0x589893e686f0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.01.511.313 I srv update: - prompt 0x58989dd503b0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.01.511.314 I srv update: - prompt 0x589893e6cba0: 117 tokens, checkpoints: 1, 303.140 MiB |
| 1.01.511.314 I srv update: - prompt 0x58989386eb00: 296 tokens, checkpoints: 1, 309.086 MiB |
| 1.01.511.315 I srv update: - prompt 0x58987fd51970: 191 tokens, checkpoints: 1, 305.598 MiB |
| 1.01.511.316 I srv update: - prompt 0x5898940d1060: 315 tokens, checkpoints: 1, 309.718 MiB |
| 1.01.511.316 I srv update: - prompt 0x58989f8f0400: 191 tokens, checkpoints: 1, 305.598 MiB |
| 1.01.511.317 I srv get_availabl: prompt cache update took 193.28 ms |
| 1.01.511.424 I slot launch_slot_: id 0 | task 759 | processing task, is_child = 0 |
| 1.01.511.435 W slot update_slots: id 0 | task 759 | erased invalidated context checkpoint (pos_min = 107, pos_max = 107, n_tokens = 108, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.01.656.516 I slot create_check: id 0 | task 759 | created context checkpoint 1 of 32 (pos_min = 83, pos_max = 83, n_tokens = 84, size = 149.626 MiB) |
| 1.05.914.266 I slot print_timing: id 0 | task 759 | n_decoded = 100, tg = 23.78 t/s |
| 1.06.095.151 I slot print_timing: id 0 | task 759 | prompt eval time = 197.28 ms / 88 tokens ( 2.24 ms per token, 446.08 tokens per second) |
| 1.06.095.157 I slot print_timing: id 0 | task 759 | eval time = 4386.44 ms / 105 tokens ( 41.78 ms per token, 23.94 tokens per second) |
| 1.06.095.157 I slot print_timing: id 0 | task 759 | total time = 4583.71 ms / 193 tokens |
| 1.06.095.158 I slot print_timing: id 0 | task 759 | graphs reused = 805 |
| 1.06.095.186 I slot release: id 0 | task 759 | stop processing: n_tokens = 192, truncated = 0 |
| 1.06.095.197 I srv update_slots: all slots are idle |
| 1.06.308.611 I srv params_from_: Chat format: peg-native |
| 1.06.308.799 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.114 (> 0.100 thold), f_keep = 0.062 |
| 1.06.308.803 I srv get_availabl: updating prompt cache |
| 1.06.308.875 W srv prompt_save: - saving prompt with length 192, total state size = 156.005 MiB (draft: 0.000 MiB) |
| 1.06.503.434 I srv load: - looking for better prompt, base f_keep = 0.062, sim = 0.114 |
| 1.06.503.445 I srv update: - cache state: 13 prompts, 3952.377 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.06.503.446 I srv update: - prompt 0x58989dd500f0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.06.503.447 I srv update: - prompt 0x58989381fee0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.06.503.448 I srv update: - prompt 0x5898934dfd20: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.06.503.453 I srv update: - prompt 0x589893eb5090: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.06.503.454 I srv update: - prompt 0x5898930b5ed0: 107 tokens, checkpoints: 1, 302.807 MiB |
| 1.06.503.455 I srv update: - prompt 0x589893e686f0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.06.503.455 I srv update: - prompt 0x58989dd503b0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.06.503.456 I srv update: - prompt 0x589893e6cba0: 117 tokens, checkpoints: 1, 303.140 MiB |
| 1.06.503.456 I srv update: - prompt 0x58989386eb00: 296 tokens, checkpoints: 1, 309.086 MiB |
| 1.06.503.457 I srv update: - prompt 0x58987fd51970: 191 tokens, checkpoints: 1, 305.598 MiB |
| 1.06.503.457 I srv update: - prompt 0x5898940d1060: 315 tokens, checkpoints: 1, 309.718 MiB |
| 1.06.503.458 I srv update: - prompt 0x58989f8f0400: 191 tokens, checkpoints: 1, 305.598 MiB |
| 1.06.503.458 I srv update: - prompt 0x589893cc3230: 192 tokens, checkpoints: 1, 305.631 MiB |
| 1.06.503.460 I srv get_availabl: prompt cache update took 194.66 ms |
| 1.06.503.567 I slot launch_slot_: id 0 | task 866 | processing task, is_child = 0 |
| 1.06.503.577 W slot update_slots: id 0 | task 866 | erased invalidated context checkpoint (pos_min = 83, pos_max = 83, n_tokens = 84, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.06.661.388 I slot create_check: id 0 | task 866 | created context checkpoint 1 of 32 (pos_min = 100, pos_max = 100, n_tokens = 101, size = 149.626 MiB) |
| 1.10.978.381 I slot print_timing: id 0 | task 866 | n_decoded = 100, tg = 23.45 t/s |
| 1.13.996.793 I slot print_timing: id 0 | task 866 | n_decoded = 170, tg = 23.34 t/s |
| 1.15.397.713 I slot print_timing: id 0 | task 866 | prompt eval time = 209.84 ms / 105 tokens ( 2.00 ms per token, 500.38 tokens per second) |
| 1.15.397.719 I slot print_timing: id 0 | task 866 | eval time = 8684.29 ms / 200 tokens ( 43.42 ms per token, 23.03 tokens per second) |
| 1.15.397.720 I slot print_timing: id 0 | task 866 | total time = 8894.13 ms / 305 tokens |
| 1.15.397.720 I slot print_timing: id 0 | task 866 | graphs reused = 1002 |
| 1.15.397.750 I slot release: id 0 | task 866 | stop processing: n_tokens = 304, truncated = 0 |
| 1.15.397.762 I srv update_slots: all slots are idle |
| 1.15.447.845 I srv params_from_: Chat format: peg-native |
| 1.15.448.096 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.133 (> 0.100 thold), f_keep = 0.039 |
| 1.15.448.102 I srv get_availabl: updating prompt cache |
| 1.15.448.172 W srv prompt_save: - saving prompt with length 304, total state size = 159.726 MiB (draft: 0.000 MiB) |
| 1.15.645.500 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.133 |
| 1.15.645.513 I srv update: - cache state: 14 prompts, 4261.729 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.15.645.514 I srv update: - prompt 0x58989dd500f0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.15.645.514 I srv update: - prompt 0x58989381fee0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.15.645.515 I srv update: - prompt 0x5898934dfd20: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.15.645.516 I srv update: - prompt 0x589893eb5090: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.15.645.516 I srv update: - prompt 0x5898930b5ed0: 107 tokens, checkpoints: 1, 302.807 MiB |
| 1.15.645.517 I srv update: - prompt 0x589893e686f0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.15.645.518 I srv update: - prompt 0x58989dd503b0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.15.645.518 I srv update: - prompt 0x589893e6cba0: 117 tokens, checkpoints: 1, 303.140 MiB |
| 1.15.645.519 I srv update: - prompt 0x58989386eb00: 296 tokens, checkpoints: 1, 309.086 MiB |
| 1.15.645.519 I srv update: - prompt 0x58987fd51970: 191 tokens, checkpoints: 1, 305.598 MiB |
| 1.15.645.520 I srv update: - prompt 0x5898940d1060: 315 tokens, checkpoints: 1, 309.718 MiB |
| 1.15.645.520 I srv update: - prompt 0x58989f8f0400: 191 tokens, checkpoints: 1, 305.598 MiB |
| 1.15.645.521 I srv update: - prompt 0x589893cc3230: 192 tokens, checkpoints: 1, 305.631 MiB |
| 1.15.645.521 I srv update: - prompt 0x58987fd51800: 304 tokens, checkpoints: 1, 309.352 MiB |
| 1.15.645.524 I srv get_availabl: prompt cache update took 197.42 ms |
| 1.15.645.630 I slot launch_slot_: id 0 | task 1068 | processing task, is_child = 0 |
| 1.15.645.642 W slot update_slots: id 0 | task 1068 | erased invalidated context checkpoint (pos_min = 100, pos_max = 100, n_tokens = 101, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.15.791.781 I slot create_check: id 0 | task 1068 | created context checkpoint 1 of 32 (pos_min = 85, pos_max = 85, n_tokens = 86, size = 149.626 MiB) |
| 1.20.046.632 I slot print_timing: id 0 | task 1068 | n_decoded = 100, tg = 23.79 t/s |
| 1.22.294.570 I slot print_timing: id 0 | task 1068 | prompt eval time = 198.24 ms / 90 tokens ( 2.20 ms per token, 454.00 tokens per second) |
| 1.22.294.578 I slot print_timing: id 0 | task 1068 | eval time = 6450.69 ms / 151 tokens ( 42.72 ms per token, 23.41 tokens per second) |
| 1.22.294.579 I slot print_timing: id 0 | task 1068 | total time = 6648.92 ms / 241 tokens |
| 1.22.294.580 I slot print_timing: id 0 | task 1068 | graphs reused = 1151 |
| 1.22.294.611 I slot release: id 0 | task 1068 | stop processing: n_tokens = 240, truncated = 0 |
| 1.22.294.624 I srv update_slots: all slots are idle |
| 1.22.349.215 I srv params_from_: Chat format: peg-native |
| 1.22.349.462 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17744637392 |
| 1.22.349.464 I srv get_availabl: updating prompt cache |
| 1.22.349.497 W srv prompt_save: - saving prompt with length 240, total state size = 157.600 MiB (draft: 0.000 MiB) |
| 1.22.543.266 I srv load: - looking for better prompt, base f_keep = 0.033, sim = 0.060 |
| 1.22.543.283 I srv update: - cache state: 15 prompts, 4568.955 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.22.543.284 I srv update: - prompt 0x58989dd500f0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.22.543.285 I srv update: - prompt 0x58989381fee0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.22.543.286 I srv update: - prompt 0x5898934dfd20: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.22.543.286 I srv update: - prompt 0x589893eb5090: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.22.543.287 I srv update: - prompt 0x5898930b5ed0: 107 tokens, checkpoints: 1, 302.807 MiB |
| 1.22.543.288 I srv update: - prompt 0x589893e686f0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.22.543.289 I srv update: - prompt 0x58989dd503b0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.22.543.290 I srv update: - prompt 0x589893e6cba0: 117 tokens, checkpoints: 1, 303.140 MiB |
| 1.22.543.291 I srv update: - prompt 0x58989386eb00: 296 tokens, checkpoints: 1, 309.086 MiB |
| 1.22.543.299 I srv update: - prompt 0x58987fd51970: 191 tokens, checkpoints: 1, 305.598 MiB |
| 1.22.543.299 I srv update: - prompt 0x5898940d1060: 315 tokens, checkpoints: 1, 309.718 MiB |
| 1.22.543.300 I srv update: - prompt 0x58989f8f0400: 191 tokens, checkpoints: 1, 305.598 MiB |
| 1.22.543.300 I srv update: - prompt 0x589893cc3230: 192 tokens, checkpoints: 1, 305.631 MiB |
| 1.22.543.301 I srv update: - prompt 0x58987fd51800: 304 tokens, checkpoints: 1, 309.352 MiB |
| 1.22.543.301 I srv update: - prompt 0x5898932efb90: 240 tokens, checkpoints: 1, 307.226 MiB |
| 1.22.543.303 I srv get_availabl: prompt cache update took 193.84 ms |
| 1.22.543.408 I slot launch_slot_: id 0 | task 1221 | processing task, is_child = 0 |
| 1.22.543.418 W slot update_slots: id 0 | task 1221 | erased invalidated context checkpoint (pos_min = 85, pos_max = 85, n_tokens = 86, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.22.756.758 I slot create_check: id 0 | task 1221 | created context checkpoint 1 of 32 (pos_min = 128, pos_max = 128, n_tokens = 129, size = 149.626 MiB) |
| 1.27.014.315 I slot print_timing: id 0 | task 1221 | n_decoded = 100, tg = 23.78 t/s |
| 1.29.515.794 I slot print_timing: id 0 | task 1221 | prompt eval time = 265.44 ms / 133 tokens ( 2.00 ms per token, 501.06 tokens per second) |
| 1.29.515.801 I slot print_timing: id 0 | task 1221 | eval time = 6706.93 ms / 156 tokens ( 42.99 ms per token, 23.26 tokens per second) |
| 1.29.515.801 I slot print_timing: id 0 | task 1221 | total time = 6972.37 ms / 289 tokens |
| 1.29.515.802 I slot print_timing: id 0 | task 1221 | graphs reused = 1304 |
| 1.29.515.836 I slot release: id 0 | task 1221 | stop processing: n_tokens = 288, truncated = 0 |
| 1.29.515.848 I srv update_slots: all slots are idle |
| 1.29.564.720 I srv params_from_: Chat format: peg-native |
| 1.29.564.929 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17751858617 |
| 1.29.564.930 I srv get_availabl: updating prompt cache |
| 1.29.564.965 W srv prompt_save: - saving prompt with length 288, total state size = 159.194 MiB (draft: 0.000 MiB) |
| 1.29.761.304 I srv load: - looking for better prompt, base f_keep = 0.028, sim = 0.075 |
| 1.29.761.319 I srv update: - cache state: 16 prompts, 4877.775 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.29.761.320 I srv update: - prompt 0x58989dd500f0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.29.761.321 I srv update: - prompt 0x58989381fee0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.29.761.321 I srv update: - prompt 0x5898934dfd20: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.29.761.322 I srv update: - prompt 0x589893eb5090: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.29.761.323 I srv update: - prompt 0x5898930b5ed0: 107 tokens, checkpoints: 1, 302.807 MiB |
| 1.29.761.323 I srv update: - prompt 0x589893e686f0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.29.761.324 I srv update: - prompt 0x58989dd503b0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.29.761.324 I srv update: - prompt 0x589893e6cba0: 117 tokens, checkpoints: 1, 303.140 MiB |
| 1.29.761.325 I srv update: - prompt 0x58989386eb00: 296 tokens, checkpoints: 1, 309.086 MiB |
| 1.29.761.325 I srv update: - prompt 0x58987fd51970: 191 tokens, checkpoints: 1, 305.598 MiB |
| 1.29.761.326 I srv update: - prompt 0x5898940d1060: 315 tokens, checkpoints: 1, 309.718 MiB |
| 1.29.761.326 I srv update: - prompt 0x58989f8f0400: 191 tokens, checkpoints: 1, 305.598 MiB |
| 1.29.761.327 I srv update: - prompt 0x589893cc3230: 192 tokens, checkpoints: 1, 305.631 MiB |
| 1.29.761.328 I srv update: - prompt 0x58987fd51800: 304 tokens, checkpoints: 1, 309.352 MiB |
| 1.29.761.329 I srv update: - prompt 0x5898932efb90: 240 tokens, checkpoints: 1, 307.226 MiB |
| 1.29.761.335 I srv update: - prompt 0x589893d97650: 288 tokens, checkpoints: 1, 308.821 MiB |
| 1.29.761.337 I srv get_availabl: prompt cache update took 196.41 ms |
| 1.29.761.450 I slot launch_slot_: id 0 | task 1379 | processing task, is_child = 0 |
| 1.29.761.462 W slot update_slots: id 0 | task 1379 | erased invalidated context checkpoint (pos_min = 128, pos_max = 128, n_tokens = 129, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.29.919.143 I slot create_check: id 0 | task 1379 | created context checkpoint 1 of 32 (pos_min = 101, pos_max = 101, n_tokens = 102, size = 149.626 MiB) |
| 1.32.430.238 I slot print_timing: id 0 | task 1379 | prompt eval time = 209.90 ms / 106 tokens ( 1.98 ms per token, 505.00 tokens per second) |
| 1.32.430.244 I slot print_timing: id 0 | task 1379 | eval time = 2458.87 ms / 56 tokens ( 43.91 ms per token, 22.77 tokens per second) |
| 1.32.430.245 I slot print_timing: id 0 | task 1379 | total time = 2668.77 ms / 162 tokens |
| 1.32.430.246 I slot print_timing: id 0 | task 1379 | graphs reused = 1358 |
| 1.32.430.283 I slot release: id 0 | task 1379 | stop processing: n_tokens = 161, truncated = 0 |
| 1.32.430.294 I srv update_slots: all slots are idle |
| 1.32.481.742 I srv params_from_: Chat format: peg-native |
| 1.32.481.997 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17754773064 |
| 1.32.482.001 I srv get_availabl: updating prompt cache |
| 1.32.482.066 W srv prompt_save: - saving prompt with length 161, total state size = 154.975 MiB (draft: 0.000 MiB) |
| 1.32.666.766 I srv load: - looking for better prompt, base f_keep = 0.075, sim = 0.094 |
| 1.32.666.783 I srv update: - cache state: 17 prompts, 5182.377 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.32.666.784 I srv update: - prompt 0x58989dd500f0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.32.666.785 I srv update: - prompt 0x58989381fee0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.32.666.786 I srv update: - prompt 0x5898934dfd20: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.32.666.786 I srv update: - prompt 0x589893eb5090: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.32.666.787 I srv update: - prompt 0x5898930b5ed0: 107 tokens, checkpoints: 1, 302.807 MiB |
| 1.32.666.788 I srv update: - prompt 0x589893e686f0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.32.666.788 I srv update: - prompt 0x58989dd503b0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.32.666.789 I srv update: - prompt 0x589893e6cba0: 117 tokens, checkpoints: 1, 303.140 MiB |
| 1.32.666.789 I srv update: - prompt 0x58989386eb00: 296 tokens, checkpoints: 1, 309.086 MiB |
| 1.32.666.790 I srv update: - prompt 0x58987fd51970: 191 tokens, checkpoints: 1, 305.598 MiB |
| 1.32.666.790 I srv update: - prompt 0x5898940d1060: 315 tokens, checkpoints: 1, 309.718 MiB |
| 1.32.666.791 I srv update: - prompt 0x58989f8f0400: 191 tokens, checkpoints: 1, 305.598 MiB |
| 1.32.666.791 I srv update: - prompt 0x589893cc3230: 192 tokens, checkpoints: 1, 305.631 MiB |
| 1.32.666.792 I srv update: - prompt 0x58987fd51800: 304 tokens, checkpoints: 1, 309.352 MiB |
| 1.32.666.792 I srv update: - prompt 0x5898932efb90: 240 tokens, checkpoints: 1, 307.226 MiB |
| 1.32.666.793 I srv update: - prompt 0x589893d97650: 288 tokens, checkpoints: 1, 308.821 MiB |
| 1.32.666.793 I srv update: - prompt 0x5898940eaaa0: 161 tokens, checkpoints: 1, 304.601 MiB |
| 1.32.666.795 I srv get_availabl: prompt cache update took 184.79 ms |
| 1.32.667.268 I slot launch_slot_: id 0 | task 1437 | processing task, is_child = 0 |
| 1.32.667.282 W slot update_slots: id 0 | task 1437 | erased invalidated context checkpoint (pos_min = 101, pos_max = 101, n_tokens = 102, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.32.839.139 I slot create_check: id 0 | task 1437 | created context checkpoint 1 of 32 (pos_min = 122, pos_max = 122, n_tokens = 123, size = 149.626 MiB) |
| 1.37.334.610 I slot print_timing: id 0 | task 1437 | n_decoded = 100, tg = 22.51 t/s |
| 1.40.361.473 I slot print_timing: id 0 | task 1437 | n_decoded = 170, tg = 22.76 t/s |
| 1.41.316.254 I slot print_timing: id 0 | task 1437 | prompt eval time = 224.04 ms / 127 tokens ( 1.76 ms per token, 566.85 tokens per second) |
| 1.41.316.263 I slot print_timing: id 0 | task 1437 | eval time = 8424.92 ms / 192 tokens ( 43.88 ms per token, 22.79 tokens per second) |
| 1.41.316.264 I slot print_timing: id 0 | task 1437 | total time = 8648.97 ms / 319 tokens |
| 1.41.316.265 I slot print_timing: id 0 | task 1437 | graphs reused = 1547 |
| 1.41.316.308 I slot release: id 0 | task 1437 | stop processing: n_tokens = 318, truncated = 0 |
| 1.41.316.322 I srv update_slots: all slots are idle |
| 1.41.399.245 I srv operator(): operator(): cleaning up before exit... |
|
|
