| 0.00.496.635 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg) |
| 0.00.496.640 I device_info: |
| 0.00.620.007 I - CUDA0 : NVIDIA GeForce RTX 3080 (20052 MiB, 19814 MiB free) |
| 0.00.746.041 I - CUDA1 : NVIDIA GeForce RTX 3080 (20052 MiB, 19814 MiB free) |
| 0.00.867.161 I - CUDA2 : NVIDIA GeForce RTX 3080 (20052 MiB, 19814 MiB free) |
| 0.00.867.181 I - CPU : AMD EPYC 7F32 8-Core Processor (193092 MiB, 193092 MiB free) |
| 0.00.867.308 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CUDA : ARCHS = 860 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | |
| 0.00.867.454 I srv init: using 15 threads for HTTP server |
| 0.00.867.934 I srv start: binding port with default address family |
| 0.00.869.267 I srv llama_server: loading model |
| 0.00.869.275 I srv load_model: loading model '/mnt/models/tmp-qwen3.8-27b-BF16.gguf' |
| 0.00.869.350 I common_init_result: fitting params to device memory ... |
| 0.00.869.353 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on) |
| 0.48.039.297 W llama_context: n_ctx_seq (16384) < n_ctx_train (262144) -- the full capacity of the model will not be utilized |
| 0.48.140.521 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable) |
| 0.48.642.621 I srv load_model: initializing slots, n_slots = 1 |
| 0.48.731.893 W srv load_model: speculative decoding will use checkpoints |
| 0.48.731.915 W common_speculative_init: no implementations specified for speculative decoding |
| 0.48.731.917 I slot load_model: id 0 | task -1 | new slot, n_ctx = 16384 |
| 0.48.732.009 I srv load_model: prompt cache is enabled, size limit: 8192 MiB |
| 0.48.732.012 I srv load_model: use `--cache-ram 0` to disable the prompt cache |
| 0.48.732.012 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391 |
| 0.48.732.012 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256 |
| 0.48.732.040 I srv init: idle slots will be saved to prompt cache upon starting a new task |
| 0.48.752.428 I init: chat template, example_format: '<|im_start|>system |
| Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer. |
|
|
| You are a helpful assistant<|im_end|> |
| <|im_start|>user |
| Hello<|im_end|> |
| <|im_start|>assistant |
| <think> |
|
|
| </think> |
|
|
| Hi there<|im_end|> |
| <|im_start|>user |
| How are you?<|im_end|> |
| <|im_start|>assistant |
| <think> |
| ' |
| 0.48.768.038 I srv init: init: chat template, thinking = 1 |
| 0.48.768.094 I srv llama_server: model loaded |
| 0.48.768.100 I srv llama_server: server is listening on http://127.0.0.1:8291 |
| 0.48.768.107 I srv update_slots: all slots are idle |
| 0.50.641.381 I srv params_from_: Chat format: peg-native |
| 0.50.641.666 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = -1 |
| 0.50.641.670 I srv get_availabl: updating prompt cache |
| 0.50.641.678 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000 |
| 0.50.641.684 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 16384 tokens, 8589934592 est) |
| 0.50.641.685 I srv get_availabl: prompt cache update took 0.01 ms |
| 0.50.641.761 I slot launch_slot_: id 0 | task 0 | processing task, is_child = 0 |
| 0.50.876.474 I slot create_check: id 0 | task 0 | created context checkpoint 1 of 32 (pos_min = 62, pos_max = 62, n_tokens = 63, size = 149.626 MiB) |
| 0.51.621.064 I slot print_timing: id 0 | task 0 | prompt eval time = 331.56 ms / 67 tokens ( 4.95 ms per token, 202.07 tokens per second) |
| 0.51.621.070 I slot print_timing: id 0 | task 0 | eval time = 647.71 ms / 7 tokens ( 92.53 ms per token, 10.81 tokens per second) |
| 0.51.621.072 I slot print_timing: id 0 | task 0 | total time = 979.27 ms / 74 tokens |
| 0.51.621.078 I slot print_timing: id 0 | task 0 | graphs reused = 6 |
| 0.51.621.132 I slot release: id 0 | task 0 | stop processing: n_tokens = 73, truncated = 0 |
| 0.51.621.138 I srv update_slots: all slots are idle |
| 0.51.654.824 I srv params_from_: Chat format: peg-native |
| 0.51.655.087 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.238 (> 0.100 thold), f_keep = 0.205 |
| 0.51.655.092 I srv get_availabl: updating prompt cache |
| 0.51.655.128 W srv prompt_save: - saving prompt with length 73, total state size = 152.052 MiB (draft: 0.000 MiB) |
| 0.51.856.803 I srv load: - looking for better prompt, base f_keep = 0.205, sim = 0.238 |
| 0.51.856.815 I srv update: - cache state: 1 prompts, 301.678 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.51.856.816 I srv update: - prompt 0x58ab4d19bb60: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.51.856.818 I srv get_availabl: prompt cache update took 201.72 ms |
| 0.51.856.927 I slot launch_slot_: id 0 | task 9 | processing task, is_child = 0 |
| 0.51.856.940 W slot update_slots: id 0 | task 9 | erased invalidated context checkpoint (pos_min = 62, pos_max = 62, n_tokens = 63, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.52.021.625 I slot create_check: id 0 | task 9 | created context checkpoint 1 of 32 (pos_min = 58, pos_max = 58, n_tokens = 59, size = 149.626 MiB) |
| 0.52.855.664 I slot print_timing: id 0 | task 9 | prompt eval time = 254.17 ms / 63 tokens ( 4.03 ms per token, 247.87 tokens per second) |
| 0.52.855.670 I slot print_timing: id 0 | task 9 | eval time = 744.55 ms / 9 tokens ( 82.73 ms per token, 12.09 tokens per second) |
| 0.52.855.671 I slot print_timing: id 0 | task 9 | total time = 998.72 ms / 72 tokens |
| 0.52.855.672 I slot print_timing: id 0 | task 9 | graphs reused = 13 |
| 0.52.855.699 I slot release: id 0 | task 9 | stop processing: n_tokens = 71, truncated = 0 |
| 0.52.855.712 I srv update_slots: all slots are idle |
| 0.52.889.306 I srv params_from_: Chat format: peg-native |
| 0.52.889.642 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.227 (> 0.100 thold), f_keep = 0.211 |
| 0.52.889.646 I srv get_availabl: updating prompt cache |
| 0.52.889.674 W srv prompt_save: - saving prompt with length 71, total state size = 151.985 MiB (draft: 0.000 MiB) |
| 0.53.085.418 I srv load: - looking for better prompt, base f_keep = 0.211, sim = 0.227 |
| 0.53.085.427 I srv update: - cache state: 2 prompts, 603.289 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.53.085.428 I srv update: - prompt 0x58ab4d19bb60: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.53.085.429 I srv update: - prompt 0x58ab4c82f840: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.53.085.430 I srv get_availabl: prompt cache update took 195.78 ms |
| 0.53.085.536 I slot launch_slot_: id 0 | task 20 | processing task, is_child = 0 |
| 0.53.085.550 W slot update_slots: id 0 | task 20 | erased invalidated context checkpoint (pos_min = 58, pos_max = 58, n_tokens = 59, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.53.426.363 I slot create_check: id 0 | task 20 | created context checkpoint 1 of 32 (pos_min = 61, pos_max = 61, n_tokens = 62, size = 149.626 MiB) |
| 0.54.309.748 I slot print_timing: id 0 | task 20 | prompt eval time = 430.20 ms / 66 tokens ( 6.52 ms per token, 153.42 tokens per second) |
| 0.54.309.758 I slot print_timing: id 0 | task 20 | eval time = 793.99 ms / 10 tokens ( 79.40 ms per token, 12.59 tokens per second) |
| 0.54.309.759 I slot print_timing: id 0 | task 20 | total time = 1224.19 ms / 76 tokens |
| 0.54.309.760 I slot print_timing: id 0 | task 20 | graphs reused = 21 |
| 0.54.309.801 I slot release: id 0 | task 20 | stop processing: n_tokens = 75, truncated = 0 |
| 0.54.309.820 I srv update_slots: all slots are idle |
| 0.54.342.960 I srv params_from_: Chat format: peg-native |
| 0.54.343.201 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17325636412 |
| 0.54.343.205 I srv get_availabl: updating prompt cache |
| 0.54.343.256 W srv prompt_save: - saving prompt with length 75, total state size = 152.118 MiB (draft: 0.000 MiB) |
| 0.54.539.359 I srv load: - looking for better prompt, base f_keep = 0.040, sim = 0.036 |
| 0.54.539.371 I srv update: - cache state: 3 prompts, 905.034 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.54.539.372 I srv update: - prompt 0x58ab4d19bb60: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.54.539.373 I srv update: - prompt 0x58ab4c82f840: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.54.539.374 I srv update: - prompt 0x58ab4c6c3c60: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.54.539.375 I srv get_availabl: prompt cache update took 196.17 ms |
| 0.54.539.485 I slot launch_slot_: id 0 | task 32 | processing task, is_child = 0 |
| 0.54.539.498 W slot update_slots: id 0 | task 32 | erased invalidated context checkpoint (pos_min = 61, pos_max = 61, n_tokens = 62, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.54.765.259 I slot create_check: id 0 | task 32 | created context checkpoint 1 of 32 (pos_min = 78, pos_max = 78, n_tokens = 79, size = 149.626 MiB) |
| 0.56.503.781 I slot print_timing: id 0 | task 32 | prompt eval time = 315.17 ms / 83 tokens ( 3.80 ms per token, 263.35 tokens per second) |
| 0.56.503.789 I slot print_timing: id 0 | task 32 | eval time = 1649.10 ms / 18 tokens ( 91.62 ms per token, 10.92 tokens per second) |
| 0.56.503.790 I slot print_timing: id 0 | task 32 | total time = 1964.27 ms / 101 tokens |
| 0.56.503.791 I slot print_timing: id 0 | task 32 | graphs reused = 37 |
| 0.56.503.832 I slot release: id 0 | task 32 | stop processing: n_tokens = 100, truncated = 0 |
| 0.56.503.850 I srv update_slots: all slots are idle |
| 0.56.526.997 I srv params_from_: Chat format: peg-native |
| 0.56.527.173 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17327830441 |
| 0.56.527.177 I srv get_availabl: updating prompt cache |
| 0.56.527.208 W srv prompt_save: - saving prompt with length 100, total state size = 152.949 MiB (draft: 0.000 MiB) |
| 0.56.723.127 I srv load: - looking for better prompt, base f_keep = 0.030, sim = 0.046 |
| 0.56.723.142 I srv update: - cache state: 4 prompts, 1207.608 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.56.723.143 I srv update: - prompt 0x58ab4d19bb60: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.56.723.144 I srv update: - prompt 0x58ab4c82f840: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.56.723.145 I srv update: - prompt 0x58ab4c6c3c60: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.56.723.145 I srv update: - prompt 0x58ab4d5af330: 100 tokens, checkpoints: 1, 302.575 MiB |
| 0.56.723.147 I srv get_availabl: prompt cache update took 195.97 ms |
| 0.56.723.250 I slot launch_slot_: id 0 | task 52 | processing task, is_child = 0 |
| 0.56.723.263 W slot update_slots: id 0 | task 52 | erased invalidated context checkpoint (pos_min = 78, pos_max = 78, n_tokens = 79, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.56.888.771 I slot create_check: id 0 | task 52 | created context checkpoint 1 of 32 (pos_min = 60, pos_max = 60, n_tokens = 61, size = 149.626 MiB) |
| 0.57.928.796 I slot print_timing: id 0 | task 52 | prompt eval time = 254.79 ms / 65 tokens ( 3.92 ms per token, 255.11 tokens per second) |
| 0.57.928.802 I slot print_timing: id 0 | task 52 | eval time = 950.73 ms / 12 tokens ( 79.23 ms per token, 12.62 tokens per second) |
| 0.57.928.803 I slot print_timing: id 0 | task 52 | total time = 1205.52 ms / 77 tokens |
| 0.57.928.803 I slot print_timing: id 0 | task 52 | graphs reused = 47 |
| 0.57.928.834 I slot release: id 0 | task 52 | stop processing: n_tokens = 76, truncated = 0 |
| 0.57.928.847 I srv update_slots: all slots are idle |
| 0.57.962.142 I srv params_from_: Chat format: peg-native |
| 0.57.962.368 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17329255442 |
| 0.57.962.371 I srv get_availabl: updating prompt cache |
| 0.57.962.412 W srv prompt_save: - saving prompt with length 76, total state size = 152.151 MiB (draft: 0.000 MiB) |
| 0.58.156.637 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.039 |
| 0.58.156.645 I srv load: - found better prompt with f_keep = 0.340, sim = 0.442 |
| 0.58.390.236 I srv update: - cache state: 4 prompts, 1206.811 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.58.390.242 I srv update: - prompt 0x58ab4d19bb60: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.58.390.243 I srv update: - prompt 0x58ab4c82f840: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.58.390.243 I srv update: - prompt 0x58ab4c6c3c60: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.58.390.244 I srv update: - prompt 0x58ab4d229020: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.58.390.245 I srv get_availabl: prompt cache update took 427.87 ms |
| 0.58.390.356 I slot launch_slot_: id 0 | task 66 | processing task, is_child = 0 |
| 0.58.390.370 W slot update_slots: id 0 | task 66 | erased invalidated context checkpoint (pos_min = 78, pos_max = 78, n_tokens = 79, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.58.632.977 I slot create_check: id 0 | task 66 | created context checkpoint 1 of 32 (pos_min = 72, pos_max = 72, n_tokens = 73, size = 149.626 MiB) |
| 1.00.776.991 I slot print_timing: id 0 | task 66 | prompt eval time = 333.34 ms / 77 tokens ( 4.33 ms per token, 231.00 tokens per second) |
| 1.00.776.997 I slot print_timing: id 0 | task 66 | eval time = 2053.28 ms / 24 tokens ( 85.55 ms per token, 11.69 tokens per second) |
| 1.00.776.998 I slot print_timing: id 0 | task 66 | total time = 2386.61 ms / 101 tokens |
| 1.00.776.999 I slot print_timing: id 0 | task 66 | graphs reused = 69 |
| 1.00.777.027 I slot release: id 0 | task 66 | stop processing: n_tokens = 100, truncated = 0 |
| 1.00.777.038 I srv update_slots: all slots are idle |
| 1.00.810.657 I srv params_from_: Chat format: peg-native |
| 1.00.810.890 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.393 (> 0.100 thold), f_keep = 0.350 |
| 1.00.810.894 I srv get_availabl: updating prompt cache |
| 1.00.810.922 W srv prompt_save: - saving prompt with length 100, total state size = 152.949 MiB (draft: 0.000 MiB) |
| 1.01.006.489 I srv load: - looking for better prompt, base f_keep = 0.350, sim = 0.393 |
| 1.01.006.503 I srv update: - cache state: 5 prompts, 1509.386 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.01.006.504 I srv update: - prompt 0x58ab4d19bb60: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.01.006.505 I srv update: - prompt 0x58ab4c82f840: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.01.006.506 I srv update: - prompt 0x58ab4c6c3c60: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.01.006.506 I srv update: - prompt 0x58ab4d229020: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.01.006.507 I srv update: - prompt 0x58ab4caedb20: 100 tokens, checkpoints: 1, 302.575 MiB |
| 1.01.006.508 I srv get_availabl: prompt cache update took 195.61 ms |
| 1.01.006.614 I slot launch_slot_: id 0 | task 92 | processing task, is_child = 0 |
| 1.01.006.632 W slot update_slots: id 0 | task 92 | erased invalidated context checkpoint (pos_min = 72, pos_max = 72, n_tokens = 73, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.01.404.002 I slot create_check: id 0 | task 92 | created context checkpoint 1 of 32 (pos_min = 84, pos_max = 84, n_tokens = 85, size = 149.626 MiB) |
| 1.05.847.240 I slot print_timing: id 0 | task 92 | prompt eval time = 486.65 ms / 89 tokens ( 5.47 ms per token, 182.88 tokens per second) |
| 1.05.847.249 I slot print_timing: id 0 | task 92 | eval time = 4353.95 ms / 50 tokens ( 87.08 ms per token, 11.48 tokens per second) |
| 1.05.847.250 I slot print_timing: id 0 | task 92 | total time = 4840.60 ms / 139 tokens |
| 1.05.847.251 I slot print_timing: id 0 | task 92 | graphs reused = 117 |
| 1.05.847.287 I slot release: id 0 | task 92 | stop processing: n_tokens = 138, truncated = 0 |
| 1.05.847.299 I srv update_slots: all slots are idle |
| 1.05.884.056 I srv params_from_: Chat format: peg-native |
| 1.05.884.301 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17337173895 |
| 1.05.884.304 I srv get_availabl: updating prompt cache |
| 1.05.884.342 W srv prompt_save: - saving prompt with length 138, total state size = 154.211 MiB (draft: 0.000 MiB) |
| 1.06.078.853 I srv load: - looking for better prompt, base f_keep = 0.022, sim = 0.050 |
| 1.06.078.868 I srv update: - cache state: 6 prompts, 1813.223 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.06.078.869 I srv update: - prompt 0x58ab4d19bb60: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.06.078.870 I srv update: - prompt 0x58ab4c82f840: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.06.078.870 I srv update: - prompt 0x58ab4c6c3c60: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.06.078.871 I srv update: - prompt 0x58ab4d229020: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.06.078.872 I srv update: - prompt 0x58ab4caedb20: 100 tokens, checkpoints: 1, 302.575 MiB |
| 1.06.078.872 I srv update: - prompt 0x58ab4cd4b120: 138 tokens, checkpoints: 1, 303.837 MiB |
| 1.06.078.874 I srv get_availabl: prompt cache update took 194.57 ms |
| 1.06.078.984 I slot launch_slot_: id 0 | task 144 | processing task, is_child = 0 |
| 1.06.078.996 W slot update_slots: id 0 | task 144 | erased invalidated context checkpoint (pos_min = 84, pos_max = 84, n_tokens = 85, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.06.424.474 I slot create_check: id 0 | task 144 | created context checkpoint 1 of 32 (pos_min = 55, pos_max = 55, n_tokens = 56, size = 149.626 MiB) |
| 1.07.311.921 I slot print_timing: id 0 | task 144 | prompt eval time = 436.10 ms / 60 tokens ( 7.27 ms per token, 137.58 tokens per second) |
| 1.07.311.929 I slot print_timing: id 0 | task 144 | eval time = 796.82 ms / 9 tokens ( 88.54 ms per token, 11.29 tokens per second) |
| 1.07.311.930 I slot print_timing: id 0 | task 144 | total time = 1232.92 ms / 69 tokens |
| 1.07.311.931 I slot print_timing: id 0 | task 144 | graphs reused = 124 |
| 1.07.311.970 I slot release: id 0 | task 144 | stop processing: n_tokens = 68, truncated = 0 |
| 1.07.311.984 I srv update_slots: all slots are idle |
| 1.07.335.887 I srv params_from_: Chat format: peg-native |
| 1.07.336.063 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.183 (> 0.100 thold), f_keep = 0.221 |
| 1.07.336.066 I srv get_availabl: updating prompt cache |
| 1.07.336.095 W srv prompt_save: - saving prompt with length 68, total state size = 151.886 MiB (draft: 0.000 MiB) |
| 1.07.526.487 I srv load: - looking for better prompt, base f_keep = 0.221, sim = 0.183 |
| 1.07.526.499 I srv update: - cache state: 7 prompts, 2114.735 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.07.526.500 I srv update: - prompt 0x58ab4d19bb60: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.07.526.501 I srv update: - prompt 0x58ab4c82f840: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.07.526.502 I srv update: - prompt 0x58ab4c6c3c60: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.07.526.502 I srv update: - prompt 0x58ab4d229020: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.07.526.503 I srv update: - prompt 0x58ab4caedb20: 100 tokens, checkpoints: 1, 302.575 MiB |
| 1.07.526.503 I srv update: - prompt 0x58ab4cd4b120: 138 tokens, checkpoints: 1, 303.837 MiB |
| 1.07.526.504 I srv update: - prompt 0x58ab4ca750f0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.07.526.506 I srv get_availabl: prompt cache update took 190.44 ms |
| 1.07.526.606 I slot launch_slot_: id 0 | task 155 | processing task, is_child = 0 |
| 1.07.526.619 W slot update_slots: id 0 | task 155 | erased invalidated context checkpoint (pos_min = 55, pos_max = 55, n_tokens = 56, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.07.748.189 I slot create_check: id 0 | task 155 | created context checkpoint 1 of 32 (pos_min = 77, pos_max = 77, n_tokens = 78, size = 149.626 MiB) |
| 1.09.082.400 I slot print_timing: id 0 | task 155 | prompt eval time = 310.87 ms / 82 tokens ( 3.79 ms per token, 263.78 tokens per second) |
| 1.09.082.406 I slot print_timing: id 0 | task 155 | eval time = 1244.90 ms / 16 tokens ( 77.81 ms per token, 12.85 tokens per second) |
| 1.09.082.407 I slot print_timing: id 0 | task 155 | total time = 1555.77 ms / 98 tokens |
| 1.09.082.408 I slot print_timing: id 0 | task 155 | graphs reused = 138 |
| 1.09.082.435 I slot release: id 0 | task 155 | stop processing: n_tokens = 97, truncated = 0 |
| 1.09.082.448 I srv update_slots: all slots are idle |
| 1.09.106.201 I srv params_from_: Chat format: peg-native |
| 1.09.106.374 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17340409043 |
| 1.09.106.377 I srv get_availabl: updating prompt cache |
| 1.09.106.416 W srv prompt_save: - saving prompt with length 97, total state size = 152.849 MiB (draft: 0.000 MiB) |
| 1.09.475.492 I srv load: - looking for better prompt, base f_keep = 0.031, sim = 0.033 |
| 1.09.475.501 I srv load: - found better prompt with f_keep = 0.340, sim = 0.374 |
| 1.09.522.295 I srv update: - cache state: 7 prompts, 2114.635 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.09.522.302 I srv update: - prompt 0x58ab4d19bb60: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.09.522.302 I srv update: - prompt 0x58ab4c82f840: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.09.522.303 I srv update: - prompt 0x58ab4c6c3c60: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.09.522.304 I srv update: - prompt 0x58ab4d229020: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.09.522.304 I srv update: - prompt 0x58ab4cd4b120: 138 tokens, checkpoints: 1, 303.837 MiB |
| 1.09.522.305 I srv update: - prompt 0x58ab4ca750f0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.09.522.305 I srv update: - prompt 0x58ab522b6b50: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.09.522.307 I srv get_availabl: prompt cache update took 415.93 ms |
| 1.09.522.422 I slot launch_slot_: id 0 | task 173 | processing task, is_child = 0 |
| 1.09.522.434 W slot update_slots: id 0 | task 173 | erased invalidated context checkpoint (pos_min = 72, pos_max = 72, n_tokens = 73, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.09.744.292 I slot create_check: id 0 | task 173 | created context checkpoint 1 of 32 (pos_min = 86, pos_max = 86, n_tokens = 87, size = 149.626 MiB) |
| 1.11.885.100 I slot print_timing: id 0 | task 173 | prompt eval time = 311.18 ms / 91 tokens ( 3.42 ms per token, 292.43 tokens per second) |
| 1.11.885.106 I slot print_timing: id 0 | task 173 | eval time = 2051.48 ms / 23 tokens ( 89.19 ms per token, 11.21 tokens per second) |
| 1.11.885.107 I slot print_timing: id 0 | task 173 | total time = 2362.66 ms / 114 tokens |
| 1.11.885.107 I slot print_timing: id 0 | task 173 | graphs reused = 159 |
| 1.11.885.134 I slot release: id 0 | task 173 | stop processing: n_tokens = 113, truncated = 0 |
| 1.11.885.147 I srv update_slots: all slots are idle |
| 1.11.919.948 I srv params_from_: Chat format: peg-native |
| 1.11.920.179 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17343211742 |
| 1.11.920.183 I srv get_availabl: updating prompt cache |
| 1.11.920.217 W srv prompt_save: - saving prompt with length 113, total state size = 153.381 MiB (draft: 0.000 MiB) |
| 1.12.113.721 I srv load: - looking for better prompt, base f_keep = 0.027, sim = 0.024 |
| 1.12.113.737 I srv update: - cache state: 8 prompts, 2417.642 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.12.113.738 I srv update: - prompt 0x58ab4d19bb60: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.12.113.738 I srv update: - prompt 0x58ab4c82f840: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.12.113.739 I srv update: - prompt 0x58ab4c6c3c60: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.12.113.740 I srv update: - prompt 0x58ab4d229020: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.12.113.740 I srv update: - prompt 0x58ab4cd4b120: 138 tokens, checkpoints: 1, 303.837 MiB |
| 1.12.113.741 I srv update: - prompt 0x58ab4ca750f0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.12.113.742 I srv update: - prompt 0x58ab522b6b50: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.12.113.742 I srv update: - prompt 0x58ab4ce20a00: 113 tokens, checkpoints: 1, 303.007 MiB |
| 1.12.113.744 I srv get_availabl: prompt cache update took 193.56 ms |
| 1.12.113.845 I slot launch_slot_: id 0 | task 198 | processing task, is_child = 0 |
| 1.12.113.855 W slot update_slots: id 0 | task 198 | erased invalidated context checkpoint (pos_min = 86, pos_max = 86, n_tokens = 87, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.12.452.639 I slot create_check: id 0 | task 198 | created context checkpoint 1 of 32 (pos_min = 119, pos_max = 119, n_tokens = 120, size = 149.626 MiB) |
| 1.21.311.145 I slot print_timing: id 0 | task 198 | n_decoded = 100, tg = 11.40 t/s |
| 1.22.886.342 I slot print_timing: id 0 | task 198 | prompt eval time = 427.95 ms / 124 tokens ( 3.45 ms per token, 289.75 tokens per second) |
| 1.22.886.347 I slot print_timing: id 0 | task 198 | eval time = 10344.53 ms / 119 tokens ( 86.93 ms per token, 11.50 tokens per second) |
| 1.22.886.348 I slot print_timing: id 0 | task 198 | total time = 10772.48 ms / 243 tokens |
| 1.22.886.349 I slot print_timing: id 0 | task 198 | graphs reused = 276 |
| 1.22.886.404 I slot release: id 0 | task 198 | stop processing: n_tokens = 242, truncated = 0 |
| 1.22.886.421 I srv update_slots: all slots are idle |
| 1.22.928.915 I srv params_from_: Chat format: peg-native |
| 1.22.929.087 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.129 (> 0.100 thold), f_keep = 0.050 |
| 1.22.929.090 I srv get_availabl: updating prompt cache |
| 1.22.929.135 W srv prompt_save: - saving prompt with length 242, total state size = 157.666 MiB (draft: 0.000 MiB) |
| 1.23.316.473 I srv load: - looking for better prompt, base f_keep = 0.050, sim = 0.129 |
| 1.23.316.488 I srv update: - cache state: 9 prompts, 2724.934 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.23.316.489 I srv update: - prompt 0x58ab4d19bb60: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.23.316.490 I srv update: - prompt 0x58ab4c82f840: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.23.316.491 I srv update: - prompt 0x58ab4c6c3c60: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.23.316.492 I srv update: - prompt 0x58ab4d229020: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.23.316.493 I srv update: - prompt 0x58ab4cd4b120: 138 tokens, checkpoints: 1, 303.837 MiB |
| 1.23.316.493 I srv update: - prompt 0x58ab4ca750f0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.23.316.494 I srv update: - prompt 0x58ab522b6b50: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.23.316.494 I srv update: - prompt 0x58ab4ce20a00: 113 tokens, checkpoints: 1, 303.007 MiB |
| 1.23.316.495 I srv update: - prompt 0x58ab4c5ebe10: 242 tokens, checkpoints: 1, 307.292 MiB |
| 1.23.316.497 I srv get_availabl: prompt cache update took 387.41 ms |
| 1.23.316.629 I slot launch_slot_: id 0 | task 319 | processing task, is_child = 0 |
| 1.23.316.647 W slot update_slots: id 0 | task 319 | erased invalidated context checkpoint (pos_min = 119, pos_max = 119, n_tokens = 120, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.23.539.236 I slot create_check: id 0 | task 319 | created context checkpoint 1 of 32 (pos_min = 88, pos_max = 88, n_tokens = 89, size = 149.626 MiB) |
| 1.31.922.804 I slot print_timing: id 0 | task 319 | prompt eval time = 311.96 ms / 93 tokens ( 3.35 ms per token, 298.11 tokens per second) |
| 1.31.922.811 I slot print_timing: id 0 | task 319 | eval time = 8294.18 ms / 94 tokens ( 88.24 ms per token, 11.33 tokens per second) |
| 1.31.922.812 I slot print_timing: id 0 | task 319 | total time = 8606.15 ms / 187 tokens |
| 1.31.922.813 I slot print_timing: id 0 | task 319 | graphs reused = 368 |
| 1.31.922.861 I slot release: id 0 | task 319 | stop processing: n_tokens = 186, truncated = 0 |
| 1.31.922.877 I srv update_slots: all slots are idle |
| 1.31.973.489 I srv params_from_: Chat format: peg-native |
| 1.31.973.720 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.138 (> 0.100 thold), f_keep = 0.065 |
| 1.31.973.723 I srv get_availabl: updating prompt cache |
| 1.31.973.761 W srv prompt_save: - saving prompt with length 186, total state size = 155.806 MiB (draft: 0.000 MiB) |
| 1.32.339.081 I srv load: - looking for better prompt, base f_keep = 0.065, sim = 0.138 |
| 1.32.339.095 I srv update: - cache state: 10 prompts, 3030.366 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.32.339.096 I srv update: - prompt 0x58ab4d19bb60: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.32.339.097 I srv update: - prompt 0x58ab4c82f840: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.32.339.098 I srv update: - prompt 0x58ab4c6c3c60: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.32.339.099 I srv update: - prompt 0x58ab4d229020: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.32.339.100 I srv update: - prompt 0x58ab4cd4b120: 138 tokens, checkpoints: 1, 303.837 MiB |
| 1.32.339.101 I srv update: - prompt 0x58ab4ca750f0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.32.339.106 I srv update: - prompt 0x58ab522b6b50: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.32.339.106 I srv update: - prompt 0x58ab4ce20a00: 113 tokens, checkpoints: 1, 303.007 MiB |
| 1.32.339.107 I srv update: - prompt 0x58ab4c5ebe10: 242 tokens, checkpoints: 1, 307.292 MiB |
| 1.32.339.108 I srv update: - prompt 0x58ab573ba670: 186 tokens, checkpoints: 1, 305.432 MiB |
| 1.32.339.110 I srv get_availabl: prompt cache update took 365.39 ms |
| 1.32.339.227 I slot launch_slot_: id 0 | task 415 | processing task, is_child = 0 |
| 1.32.339.243 W slot update_slots: id 0 | task 415 | erased invalidated context checkpoint (pos_min = 88, pos_max = 88, n_tokens = 89, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.32.557.983 I slot create_check: id 0 | task 415 | created context checkpoint 1 of 32 (pos_min = 82, pos_max = 82, n_tokens = 83, size = 149.626 MiB) |
| 1.41.425.908 I slot print_timing: id 0 | task 415 | n_decoded = 100, tg = 11.39 t/s |
| 1.44.350.816 I slot print_timing: id 0 | task 415 | prompt eval time = 308.11 ms / 87 tokens ( 3.54 ms per token, 282.36 tokens per second) |
| 1.44.350.823 I slot print_timing: id 0 | task 415 | eval time = 11703.45 ms / 134 tokens ( 87.34 ms per token, 11.45 tokens per second) |
| 1.44.350.824 I slot print_timing: id 0 | task 415 | total time = 12011.57 ms / 221 tokens |
| 1.44.350.825 I slot print_timing: id 0 | task 415 | graphs reused = 500 |
| 1.44.350.858 I slot release: id 0 | task 415 | stop processing: n_tokens = 220, truncated = 0 |
| 1.44.350.869 I srv update_slots: all slots are idle |
| 1.44.390.113 I srv params_from_: Chat format: peg-native |
| 1.44.390.274 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.055 |
| 1.44.390.277 I srv get_availabl: updating prompt cache |
| 1.44.390.308 W srv prompt_save: - saving prompt with length 220, total state size = 156.935 MiB (draft: 0.000 MiB) |
| 1.44.588.994 I srv load: - looking for better prompt, base f_keep = 0.055, sim = 0.107 |
| 1.44.589.010 I srv update: - cache state: 11 prompts, 3336.928 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.44.589.011 I srv update: - prompt 0x58ab4d19bb60: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.44.589.012 I srv update: - prompt 0x58ab4c82f840: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.44.589.012 I srv update: - prompt 0x58ab4c6c3c60: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.44.589.013 I srv update: - prompt 0x58ab4d229020: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.44.589.014 I srv update: - prompt 0x58ab4cd4b120: 138 tokens, checkpoints: 1, 303.837 MiB |
| 1.44.589.014 I srv update: - prompt 0x58ab4ca750f0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.44.589.015 I srv update: - prompt 0x58ab522b6b50: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.44.589.015 I srv update: - prompt 0x58ab4ce20a00: 113 tokens, checkpoints: 1, 303.007 MiB |
| 1.44.589.016 I srv update: - prompt 0x58ab4c5ebe10: 242 tokens, checkpoints: 1, 307.292 MiB |
| 1.44.589.017 I srv update: - prompt 0x58ab573ba670: 186 tokens, checkpoints: 1, 305.432 MiB |
| 1.44.589.017 I srv update: - prompt 0x58ab4d5af330: 220 tokens, checkpoints: 1, 306.562 MiB |
| 1.44.589.019 I srv get_availabl: prompt cache update took 198.74 ms |
| 1.44.589.116 I slot launch_slot_: id 0 | task 551 | processing task, is_child = 0 |
| 1.44.589.126 W slot update_slots: id 0 | task 551 | erased invalidated context checkpoint (pos_min = 82, pos_max = 82, n_tokens = 83, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.44.813.292 I slot create_check: id 0 | task 551 | created context checkpoint 1 of 32 (pos_min = 107, pos_max = 107, n_tokens = 108, size = 149.626 MiB) |
| 1.53.617.059 I slot print_timing: id 0 | task 551 | n_decoded = 100, tg = 11.48 t/s |
| 1.53.849.786 I slot print_timing: id 0 | task 551 | prompt eval time = 313.35 ms / 112 tokens ( 2.80 ms per token, 357.43 tokens per second) |
| 1.53.849.793 I slot print_timing: id 0 | task 551 | eval time = 8947.29 ms / 103 tokens ( 86.87 ms per token, 11.51 tokens per second) |
| 1.53.849.794 I slot print_timing: id 0 | task 551 | total time = 9260.64 ms / 215 tokens |
| 1.53.849.795 I slot print_timing: id 0 | task 551 | graphs reused = 601 |
| 1.53.849.843 I slot release: id 0 | task 551 | stop processing: n_tokens = 214, truncated = 0 |
| 1.53.849.863 I srv update_slots: all slots are idle |
| 1.53.903.516 I srv params_from_: Chat format: peg-native |
| 1.53.903.811 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.136 (> 0.100 thold), f_keep = 0.056 |
| 1.53.903.816 I srv get_availabl: updating prompt cache |
| 1.53.903.877 W srv prompt_save: - saving prompt with length 214, total state size = 156.736 MiB (draft: 0.000 MiB) |
| 1.54.094.075 I srv load: - looking for better prompt, base f_keep = 0.056, sim = 0.136 |
| 1.54.094.091 I srv update: - cache state: 12 prompts, 3643.290 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.54.094.092 I srv update: - prompt 0x58ab4d19bb60: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.54.094.092 I srv update: - prompt 0x58ab4c82f840: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.54.094.093 I srv update: - prompt 0x58ab4c6c3c60: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.54.094.094 I srv update: - prompt 0x58ab4d229020: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.54.094.094 I srv update: - prompt 0x58ab4cd4b120: 138 tokens, checkpoints: 1, 303.837 MiB |
| 1.54.094.095 I srv update: - prompt 0x58ab4ca750f0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.54.094.095 I srv update: - prompt 0x58ab522b6b50: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.54.094.096 I srv update: - prompt 0x58ab4ce20a00: 113 tokens, checkpoints: 1, 303.007 MiB |
| 1.54.094.096 I srv update: - prompt 0x58ab4c5ebe10: 242 tokens, checkpoints: 1, 307.292 MiB |
| 1.54.094.097 I srv update: - prompt 0x58ab573ba670: 186 tokens, checkpoints: 1, 305.432 MiB |
| 1.54.094.097 I srv update: - prompt 0x58ab4d5af330: 220 tokens, checkpoints: 1, 306.562 MiB |
| 1.54.094.098 I srv update: - prompt 0x58ab4cd4d510: 214 tokens, checkpoints: 1, 306.362 MiB |
| 1.54.094.099 I srv get_availabl: prompt cache update took 190.28 ms |
| 1.54.094.198 I slot launch_slot_: id 0 | task 656 | processing task, is_child = 0 |
| 1.54.094.209 W slot update_slots: id 0 | task 656 | erased invalidated context checkpoint (pos_min = 107, pos_max = 107, n_tokens = 108, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.54.509.605 I slot create_check: id 0 | task 656 | created context checkpoint 1 of 32 (pos_min = 83, pos_max = 83, n_tokens = 84, size = 149.626 MiB) |
| 2.02.731.748 I slot print_timing: id 0 | task 656 | prompt eval time = 504.58 ms / 88 tokens ( 5.73 ms per token, 174.40 tokens per second) |
| 2.02.731.754 I slot print_timing: id 0 | task 656 | eval time = 8132.95 ms / 93 tokens ( 87.45 ms per token, 11.43 tokens per second) |
| 2.02.731.754 I slot print_timing: id 0 | task 656 | total time = 8637.53 ms / 181 tokens |
| 2.02.731.755 I slot print_timing: id 0 | task 656 | graphs reused = 692 |
| 2.02.731.784 I slot release: id 0 | task 656 | stop processing: n_tokens = 180, truncated = 0 |
| 2.02.731.796 I srv update_slots: all slots are idle |
| 2.02.782.921 I srv params_from_: Chat format: peg-native |
| 2.02.783.148 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.114 (> 0.100 thold), f_keep = 0.067 |
| 2.02.783.152 I srv get_availabl: updating prompt cache |
| 2.02.783.183 W srv prompt_save: - saving prompt with length 180, total state size = 155.607 MiB (draft: 0.000 MiB) |
| 2.02.978.403 I srv load: - looking for better prompt, base f_keep = 0.067, sim = 0.114 |
| 2.02.978.416 I srv update: - cache state: 13 prompts, 3948.523 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 2.02.978.417 I srv update: - prompt 0x58ab4d19bb60: 73 tokens, checkpoints: 1, 301.678 MiB |
| 2.02.978.418 I srv update: - prompt 0x58ab4c82f840: 71 tokens, checkpoints: 1, 301.611 MiB |
| 2.02.978.419 I srv update: - prompt 0x58ab4c6c3c60: 75 tokens, checkpoints: 1, 301.744 MiB |
| 2.02.978.419 I srv update: - prompt 0x58ab4d229020: 76 tokens, checkpoints: 1, 301.778 MiB |
| 2.02.978.420 I srv update: - prompt 0x58ab4cd4b120: 138 tokens, checkpoints: 1, 303.837 MiB |
| 2.02.978.421 I srv update: - prompt 0x58ab4ca750f0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 2.02.978.421 I srv update: - prompt 0x58ab522b6b50: 97 tokens, checkpoints: 1, 302.475 MiB |
| 2.02.978.422 I srv update: - prompt 0x58ab4ce20a00: 113 tokens, checkpoints: 1, 303.007 MiB |
| 2.02.978.423 I srv update: - prompt 0x58ab4c5ebe10: 242 tokens, checkpoints: 1, 307.292 MiB |
| 2.02.978.423 I srv update: - prompt 0x58ab573ba670: 186 tokens, checkpoints: 1, 305.432 MiB |
| 2.02.978.424 I srv update: - prompt 0x58ab4d5af330: 220 tokens, checkpoints: 1, 306.562 MiB |
| 2.02.978.424 I srv update: - prompt 0x58ab4cd4d510: 214 tokens, checkpoints: 1, 306.362 MiB |
| 2.02.978.425 I srv update: - prompt 0x58ab4c70c570: 180 tokens, checkpoints: 1, 305.233 MiB |
| 2.02.978.426 I srv get_availabl: prompt cache update took 195.27 ms |
| 2.02.978.526 I slot launch_slot_: id 0 | task 751 | processing task, is_child = 0 |
| 2.02.978.537 W slot update_slots: id 0 | task 751 | erased invalidated context checkpoint (pos_min = 83, pos_max = 83, n_tokens = 84, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 2.03.385.975 I slot create_check: id 0 | task 751 | created context checkpoint 1 of 32 (pos_min = 100, pos_max = 100, n_tokens = 101, size = 149.626 MiB) |
| 2.12.121.252 I slot print_timing: id 0 | task 751 | n_decoded = 100, tg = 11.57 t/s |
| 2.15.146.314 I slot print_timing: id 0 | task 751 | n_decoded = 135, tg = 11.57 t/s |
| 2.18.151.462 I slot print_timing: id 0 | task 751 | n_decoded = 170, tg = 11.58 t/s |
| 2.19.401.415 I slot print_timing: id 0 | task 751 | prompt eval time = 496.65 ms / 105 tokens ( 4.73 ms per token, 211.41 tokens per second) |
| 2.19.401.421 I slot print_timing: id 0 | task 751 | eval time = 15926.20 ms / 183 tokens ( 87.03 ms per token, 11.49 tokens per second) |
| 2.19.401.422 I slot print_timing: id 0 | task 751 | total time = 16422.86 ms / 288 tokens |
| 2.19.401.423 I slot print_timing: id 0 | task 751 | graphs reused = 872 |
| 2.19.401.466 I slot release: id 0 | task 751 | stop processing: n_tokens = 287, truncated = 0 |
| 2.19.401.483 I srv update_slots: all slots are idle |
| 2.19.454.302 I srv params_from_: Chat format: peg-native |
| 2.19.454.548 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.133 (> 0.100 thold), f_keep = 0.042 |
| 2.19.454.552 I srv get_availabl: updating prompt cache |
| 2.19.454.608 W srv prompt_save: - saving prompt with length 287, total state size = 159.161 MiB (draft: 0.000 MiB) |
| 2.19.650.268 I srv load: - looking for better prompt, base f_keep = 0.042, sim = 0.133 |
| 2.19.650.282 I srv update: - cache state: 14 prompts, 4257.310 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 2.19.650.283 I srv update: - prompt 0x58ab4d19bb60: 73 tokens, checkpoints: 1, 301.678 MiB |
| 2.19.650.284 I srv update: - prompt 0x58ab4c82f840: 71 tokens, checkpoints: 1, 301.611 MiB |
| 2.19.650.284 I srv update: - prompt 0x58ab4c6c3c60: 75 tokens, checkpoints: 1, 301.744 MiB |
| 2.19.650.285 I srv update: - prompt 0x58ab4d229020: 76 tokens, checkpoints: 1, 301.778 MiB |
| 2.19.650.286 I srv update: - prompt 0x58ab4cd4b120: 138 tokens, checkpoints: 1, 303.837 MiB |
| 2.19.650.286 I srv update: - prompt 0x58ab4ca750f0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 2.19.650.287 I srv update: - prompt 0x58ab522b6b50: 97 tokens, checkpoints: 1, 302.475 MiB |
| 2.19.650.288 I srv update: - prompt 0x58ab4ce20a00: 113 tokens, checkpoints: 1, 303.007 MiB |
| 2.19.650.288 I srv update: - prompt 0x58ab4c5ebe10: 242 tokens, checkpoints: 1, 307.292 MiB |
| 2.19.650.289 I srv update: - prompt 0x58ab573ba670: 186 tokens, checkpoints: 1, 305.432 MiB |
| 2.19.650.289 I srv update: - prompt 0x58ab4d5af330: 220 tokens, checkpoints: 1, 306.562 MiB |
| 2.19.650.290 I srv update: - prompt 0x58ab4cd4d510: 214 tokens, checkpoints: 1, 306.362 MiB |
| 2.19.650.290 I srv update: - prompt 0x58ab4c70c570: 180 tokens, checkpoints: 1, 305.233 MiB |
| 2.19.650.291 I srv update: - prompt 0x58ab4d1e7d50: 287 tokens, checkpoints: 1, 308.787 MiB |
| 2.19.650.293 I srv get_availabl: prompt cache update took 195.74 ms |
| 2.19.650.391 I slot launch_slot_: id 0 | task 936 | processing task, is_child = 0 |
| 2.19.650.406 W slot update_slots: id 0 | task 936 | erased invalidated context checkpoint (pos_min = 100, pos_max = 100, n_tokens = 101, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 2.19.869.018 I slot create_check: id 0 | task 936 | created context checkpoint 1 of 32 (pos_min = 85, pos_max = 85, n_tokens = 86, size = 149.626 MiB) |
| 2.28.533.215 I slot print_timing: id 0 | task 936 | n_decoded = 100, tg = 11.66 t/s |
| 2.29.822.488 I slot print_timing: id 0 | task 936 | prompt eval time = 308.42 ms / 90 tokens ( 3.43 ms per token, 291.81 tokens per second) |
| 2.29.822.495 I slot print_timing: id 0 | task 936 | eval time = 9863.66 ms / 114 tokens ( 86.52 ms per token, 11.56 tokens per second) |
| 2.29.822.496 I slot print_timing: id 0 | task 936 | total time = 10172.08 ms / 204 tokens |
| 2.29.822.497 I slot print_timing: id 0 | task 936 | graphs reused = 984 |
| 2.29.822.526 I slot release: id 0 | task 936 | stop processing: n_tokens = 203, truncated = 0 |
| 2.29.822.536 I srv update_slots: all slots are idle |
| 2.29.878.965 I srv params_from_: Chat format: peg-native |
| 2.29.879.199 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17421149134 |
| 2.29.879.203 I srv get_availabl: updating prompt cache |
| 2.29.879.239 W srv prompt_save: - saving prompt with length 203, total state size = 156.371 MiB (draft: 0.000 MiB) |
| 2.30.071.531 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.060 |
| 2.30.071.548 I srv update: - cache state: 15 prompts, 4563.307 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 2.30.071.549 I srv update: - prompt 0x58ab4d19bb60: 73 tokens, checkpoints: 1, 301.678 MiB |
| 2.30.071.550 I srv update: - prompt 0x58ab4c82f840: 71 tokens, checkpoints: 1, 301.611 MiB |
| 2.30.071.551 I srv update: - prompt 0x58ab4c6c3c60: 75 tokens, checkpoints: 1, 301.744 MiB |
| 2.30.071.551 I srv update: - prompt 0x58ab4d229020: 76 tokens, checkpoints: 1, 301.778 MiB |
| 2.30.071.552 I srv update: - prompt 0x58ab4cd4b120: 138 tokens, checkpoints: 1, 303.837 MiB |
| 2.30.071.553 I srv update: - prompt 0x58ab4ca750f0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 2.30.071.553 I srv update: - prompt 0x58ab522b6b50: 97 tokens, checkpoints: 1, 302.475 MiB |
| 2.30.071.554 I srv update: - prompt 0x58ab4ce20a00: 113 tokens, checkpoints: 1, 303.007 MiB |
| 2.30.071.555 I srv update: - prompt 0x58ab4c5ebe10: 242 tokens, checkpoints: 1, 307.292 MiB |
| 2.30.071.555 I srv update: - prompt 0x58ab573ba670: 186 tokens, checkpoints: 1, 305.432 MiB |
| 2.30.071.556 I srv update: - prompt 0x58ab4d5af330: 220 tokens, checkpoints: 1, 306.562 MiB |
| 2.30.071.556 I srv update: - prompt 0x58ab4cd4d510: 214 tokens, checkpoints: 1, 306.362 MiB |
| 2.30.071.557 I srv update: - prompt 0x58ab4c70c570: 180 tokens, checkpoints: 1, 305.233 MiB |
| 2.30.071.557 I srv update: - prompt 0x58ab4d1e7d50: 287 tokens, checkpoints: 1, 308.787 MiB |
| 2.30.071.558 I srv update: - prompt 0x58ab4ccc3880: 203 tokens, checkpoints: 1, 305.997 MiB |
| 2.30.071.559 I srv get_availabl: prompt cache update took 192.36 ms |
| 2.30.071.661 I slot launch_slot_: id 0 | task 1052 | processing task, is_child = 0 |
| 2.30.071.672 W slot update_slots: id 0 | task 1052 | erased invalidated context checkpoint (pos_min = 85, pos_max = 85, n_tokens = 86, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 2.30.357.336 I slot create_check: id 0 | task 1052 | created context checkpoint 1 of 32 (pos_min = 128, pos_max = 128, n_tokens = 129, size = 149.626 MiB) |
| 2.39.233.160 I slot print_timing: id 0 | task 1052 | n_decoded = 100, tg = 11.38 t/s |
| 2.42.407.859 I slot print_timing: id 0 | task 1052 | n_decoded = 136, tg = 11.37 t/s |
| 2.43.947.332 I slot print_timing: id 0 | task 1052 | prompt eval time = 373.88 ms / 133 tokens ( 2.81 ms per token, 355.73 tokens per second) |
| 2.43.947.338 I slot print_timing: id 0 | task 1052 | eval time = 13501.78 ms / 156 tokens ( 86.55 ms per token, 11.55 tokens per second) |
| 2.43.947.339 I slot print_timing: id 0 | task 1052 | total time = 13875.65 ms / 289 tokens |
| 2.43.947.340 I slot print_timing: id 0 | task 1052 | graphs reused = 1137 |
| 2.43.947.375 I slot release: id 0 | task 1052 | stop processing: n_tokens = 288, truncated = 0 |
| 2.43.947.387 I srv update_slots: all slots are idle |
| 2.43.999.040 I srv params_from_: Chat format: peg-native |
| 2.43.999.261 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17435273983 |
| 2.43.999.264 I srv get_availabl: updating prompt cache |
| 2.43.999.298 W srv prompt_save: - saving prompt with length 288, total state size = 159.194 MiB (draft: 0.000 MiB) |
| 2.44.429.484 I srv load: - looking for better prompt, base f_keep = 0.028, sim = 0.075 |
| 2.44.429.501 I srv update: - cache state: 16 prompts, 4872.127 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 2.44.429.502 I srv update: - prompt 0x58ab4d19bb60: 73 tokens, checkpoints: 1, 301.678 MiB |
| 2.44.429.503 I srv update: - prompt 0x58ab4c82f840: 71 tokens, checkpoints: 1, 301.611 MiB |
| 2.44.429.503 I srv update: - prompt 0x58ab4c6c3c60: 75 tokens, checkpoints: 1, 301.744 MiB |
| 2.44.429.504 I srv update: - prompt 0x58ab4d229020: 76 tokens, checkpoints: 1, 301.778 MiB |
| 2.44.429.505 I srv update: - prompt 0x58ab4cd4b120: 138 tokens, checkpoints: 1, 303.837 MiB |
| 2.44.429.505 I srv update: - prompt 0x58ab4ca750f0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 2.44.429.507 I srv update: - prompt 0x58ab522b6b50: 97 tokens, checkpoints: 1, 302.475 MiB |
| 2.44.429.508 I srv update: - prompt 0x58ab4ce20a00: 113 tokens, checkpoints: 1, 303.007 MiB |
| 2.44.429.509 I srv update: - prompt 0x58ab4c5ebe10: 242 tokens, checkpoints: 1, 307.292 MiB |
| 2.44.429.510 I srv update: - prompt 0x58ab573ba670: 186 tokens, checkpoints: 1, 305.432 MiB |
| 2.44.429.511 I srv update: - prompt 0x58ab4d5af330: 220 tokens, checkpoints: 1, 306.562 MiB |
| 2.44.429.511 I srv update: - prompt 0x58ab4cd4d510: 214 tokens, checkpoints: 1, 306.362 MiB |
| 2.44.429.512 I srv update: - prompt 0x58ab4c70c570: 180 tokens, checkpoints: 1, 305.233 MiB |
| 2.44.429.512 I srv update: - prompt 0x58ab4d1e7d50: 287 tokens, checkpoints: 1, 308.787 MiB |
| 2.44.429.513 I srv update: - prompt 0x58ab4ccc3880: 203 tokens, checkpoints: 1, 305.997 MiB |
| 2.44.429.513 I srv update: - prompt 0x58ab576adca0: 288 tokens, checkpoints: 1, 308.821 MiB |
| 2.44.429.515 I srv get_availabl: prompt cache update took 430.25 ms |
| 2.44.429.612 I slot launch_slot_: id 0 | task 1210 | processing task, is_child = 0 |
| 2.44.429.624 W slot update_slots: id 0 | task 1210 | erased invalidated context checkpoint (pos_min = 128, pos_max = 128, n_tokens = 129, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 2.44.649.076 I slot create_check: id 0 | task 1210 | created context checkpoint 1 of 32 (pos_min = 101, pos_max = 101, n_tokens = 102, size = 149.626 MiB) |
| 2.49.463.583 I slot print_timing: id 0 | task 1210 | prompt eval time = 308.55 ms / 106 tokens ( 2.91 ms per token, 343.54 tokens per second) |
| 2.49.463.589 I slot print_timing: id 0 | task 1210 | eval time = 4725.40 ms / 54 tokens ( 87.51 ms per token, 11.43 tokens per second) |
| 2.49.463.590 I slot print_timing: id 0 | task 1210 | total time = 5033.95 ms / 160 tokens |
| 2.49.463.591 I slot print_timing: id 0 | task 1210 | graphs reused = 1189 |
| 2.49.463.624 I slot release: id 0 | task 1210 | stop processing: n_tokens = 159, truncated = 0 |
| 2.49.463.635 I srv update_slots: all slots are idle |
| 2.49.515.722 I srv params_from_: Chat format: peg-native |
| 2.49.516.014 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17440790232 |
| 2.49.516.019 I srv get_availabl: updating prompt cache |
| 2.49.516.075 W srv prompt_save: - saving prompt with length 159, total state size = 154.909 MiB (draft: 0.000 MiB) |
| 2.49.728.505 I srv load: - looking for better prompt, base f_keep = 0.075, sim = 0.094 |
| 2.49.728.519 I srv update: - cache state: 17 prompts, 5176.662 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 2.49.728.520 I srv update: - prompt 0x58ab4d19bb60: 73 tokens, checkpoints: 1, 301.678 MiB |
| 2.49.728.521 I srv update: - prompt 0x58ab4c82f840: 71 tokens, checkpoints: 1, 301.611 MiB |
| 2.49.728.521 I srv update: - prompt 0x58ab4c6c3c60: 75 tokens, checkpoints: 1, 301.744 MiB |
| 2.49.728.522 I srv update: - prompt 0x58ab4d229020: 76 tokens, checkpoints: 1, 301.778 MiB |
| 2.49.728.523 I srv update: - prompt 0x58ab4cd4b120: 138 tokens, checkpoints: 1, 303.837 MiB |
| 2.49.728.523 I srv update: - prompt 0x58ab4ca750f0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 2.49.728.524 I srv update: - prompt 0x58ab522b6b50: 97 tokens, checkpoints: 1, 302.475 MiB |
| 2.49.728.525 I srv update: - prompt 0x58ab4ce20a00: 113 tokens, checkpoints: 1, 303.007 MiB |
| 2.49.728.525 I srv update: - prompt 0x58ab4c5ebe10: 242 tokens, checkpoints: 1, 307.292 MiB |
| 2.49.728.526 I srv update: - prompt 0x58ab573ba670: 186 tokens, checkpoints: 1, 305.432 MiB |
| 2.49.728.526 I srv update: - prompt 0x58ab4d5af330: 220 tokens, checkpoints: 1, 306.562 MiB |
| 2.49.728.527 I srv update: - prompt 0x58ab4cd4d510: 214 tokens, checkpoints: 1, 306.362 MiB |
| 2.49.728.527 I srv update: - prompt 0x58ab4c70c570: 180 tokens, checkpoints: 1, 305.233 MiB |
| 2.49.728.528 I srv update: - prompt 0x58ab4d1e7d50: 287 tokens, checkpoints: 1, 308.787 MiB |
| 2.49.728.528 I srv update: - prompt 0x58ab4ccc3880: 203 tokens, checkpoints: 1, 305.997 MiB |
| 2.49.728.529 I srv update: - prompt 0x58ab576adca0: 288 tokens, checkpoints: 1, 308.821 MiB |
| 2.49.728.529 I srv update: - prompt 0x58ab4d2bcf20: 159 tokens, checkpoints: 1, 304.535 MiB |
| 2.49.728.531 I srv get_availabl: prompt cache update took 212.51 ms |
| 2.49.728.630 I slot launch_slot_: id 0 | task 1266 | processing task, is_child = 0 |
| 2.49.728.641 W slot update_slots: id 0 | task 1266 | erased invalidated context checkpoint (pos_min = 101, pos_max = 101, n_tokens = 102, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 2.49.955.054 I slot create_check: id 0 | task 1266 | created context checkpoint 1 of 32 (pos_min = 122, pos_max = 122, n_tokens = 123, size = 149.626 MiB) |
| 2.58.657.229 I slot print_timing: id 0 | task 1266 | n_decoded = 100, tg = 11.61 t/s |
| 3.01.667.347 I slot print_timing: id 0 | task 1266 | n_decoded = 134, tg = 11.53 t/s |
| 3.04.732.904 I slot print_timing: id 0 | task 1266 | n_decoded = 170, tg = 11.57 t/s |
| 3.06.351.852 I slot print_timing: id 0 | task 1266 | prompt eval time = 315.49 ms / 127 tokens ( 2.48 ms per token, 402.55 tokens per second) |
| 3.06.351.861 I slot print_timing: id 0 | task 1266 | eval time = 16307.72 ms / 188 tokens ( 86.74 ms per token, 11.53 tokens per second) |
| 3.06.351.862 I slot print_timing: id 0 | task 1266 | total time = 16623.20 ms / 315 tokens |
| 3.06.351.863 I slot print_timing: id 0 | task 1266 | graphs reused = 1374 |
| 3.06.351.904 I slot release: id 0 | task 1266 | stop processing: n_tokens = 314, truncated = 0 |
| 3.06.351.917 I srv update_slots: all slots are idle |
| 3.06.422.680 I srv operator(): operator(): cleaning up before exit... |
|
|
