| 0.00.626.290 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg) |
| 0.00.626.293 I device_info: |
| 0.00.752.158 I - CUDA0 : NVIDIA GeForce RTX 3080 (20052 MiB, 19814 MiB free) |
| 0.00.872.516 I - CUDA1 : NVIDIA GeForce RTX 3080 (20052 MiB, 19814 MiB free) |
| 0.01.004.917 I - CUDA2 : NVIDIA GeForce RTX 3080 (20052 MiB, 19814 MiB free) |
| 0.01.004.939 I - CPU : AMD EPYC 7F32 8-Core Processor (193092 MiB, 193092 MiB free) |
| 0.01.005.066 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CUDA : ARCHS = 860 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | |
| 0.01.005.198 I srv init: using 15 threads for HTTP server |
| 0.01.005.724 I srv start: binding port with default address family |
| 0.01.007.034 I srv llama_server: loading model |
| 0.01.007.042 I srv load_model: loading model '/mnt/models/PropellerA-models/qwen3.8-27b-Q8_0.gguf' |
| 0.01.007.139 I common_init_result: fitting params to device memory ... |
| 0.01.007.142 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on) |
| 0.07.745.531 W llama_context: n_ctx_seq (16384) < n_ctx_train (262144) -- the full capacity of the model will not be utilized |
| 0.07.845.273 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable) |
| 0.07.961.951 I srv load_model: initializing slots, n_slots = 1 |
| 0.08.015.019 W srv load_model: speculative decoding will use checkpoints |
| 0.08.015.041 W common_speculative_init: no implementations specified for speculative decoding |
| 0.08.015.042 I slot load_model: id 0 | task -1 | new slot, n_ctx = 16384 |
| 0.08.015.142 I srv load_model: prompt cache is enabled, size limit: 8192 MiB |
| 0.08.015.145 I srv load_model: use `--cache-ram 0` to disable the prompt cache |
| 0.08.015.146 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391 |
| 0.08.015.146 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256 |
| 0.08.015.176 I srv init: idle slots will be saved to prompt cache upon starting a new task |
| 0.08.035.459 I init: chat template, example_format: '<|im_start|>system |
| Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer. |
|
|
| You are a helpful assistant<|im_end|> |
| <|im_start|>user |
| Hello<|im_end|> |
| <|im_start|>assistant |
| <think> |
|
|
| </think> |
|
|
| Hi there<|im_end|> |
| <|im_start|>user |
| How are you?<|im_end|> |
| <|im_start|>assistant |
| <think> |
| ' |
| 0.08.050.911 I srv init: init: chat template, thinking = 1 |
| 0.08.050.963 I srv llama_server: model loaded |
| 0.08.050.969 I srv llama_server: server is listening on http://127.0.0.1:8291 |
| 0.08.050.976 I srv update_slots: all slots are idle |
| 0.08.756.600 I srv params_from_: Chat format: peg-native |
| 0.08.757.001 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = -1 |
| 0.08.757.006 I srv get_availabl: updating prompt cache |
| 0.08.757.017 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000 |
| 0.08.757.026 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 16384 tokens, 8589934592 est) |
| 0.08.757.031 I srv get_availabl: prompt cache update took 0.02 ms |
| 0.08.757.170 I slot launch_slot_: id 0 | task 0 | processing task, is_child = 0 |
| 0.08.928.555 I slot create_check: id 0 | task 0 | created context checkpoint 1 of 32 (pos_min = 62, pos_max = 62, n_tokens = 63, size = 149.626 MiB) |
| 0.09.254.322 I slot print_timing: id 0 | task 0 | prompt eval time = 225.90 ms / 67 tokens ( 3.37 ms per token, 296.60 tokens per second) |
| 0.09.254.328 I slot print_timing: id 0 | task 0 | eval time = 271.22 ms / 7 tokens ( 38.75 ms per token, 25.81 tokens per second) |
| 0.09.254.329 I slot print_timing: id 0 | task 0 | total time = 497.12 ms / 74 tokens |
| 0.09.254.335 I slot print_timing: id 0 | task 0 | graphs reused = 6 |
| 0.09.254.392 I slot release: id 0 | task 0 | stop processing: n_tokens = 73, truncated = 0 |
| 0.09.254.398 I srv update_slots: all slots are idle |
| 0.09.286.769 I srv params_from_: Chat format: peg-native |
| 0.09.287.017 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.238 (> 0.100 thold), f_keep = 0.205 |
| 0.09.287.021 I srv get_availabl: updating prompt cache |
| 0.09.287.057 W srv prompt_save: - saving prompt with length 73, total state size = 152.052 MiB (draft: 0.000 MiB) |
| 0.09.654.414 I srv load: - looking for better prompt, base f_keep = 0.205, sim = 0.238 |
| 0.09.654.422 I srv update: - cache state: 1 prompts, 301.678 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.09.654.423 I srv update: - prompt 0x60f4303137d0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.09.654.425 I srv get_availabl: prompt cache update took 367.40 ms |
| 0.09.654.539 I slot launch_slot_: id 0 | task 9 | processing task, is_child = 0 |
| 0.09.654.551 W slot update_slots: id 0 | task 9 | erased invalidated context checkpoint (pos_min = 62, pos_max = 62, n_tokens = 63, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.09.777.839 I slot create_check: id 0 | task 9 | created context checkpoint 1 of 32 (pos_min = 58, pos_max = 58, n_tokens = 59, size = 149.626 MiB) |
| 0.10.188.270 I slot print_timing: id 0 | task 9 | prompt eval time = 174.97 ms / 63 tokens ( 2.78 ms per token, 360.07 tokens per second) |
| 0.10.188.276 I slot print_timing: id 0 | task 9 | eval time = 358.74 ms / 9 tokens ( 39.86 ms per token, 25.09 tokens per second) |
| 0.10.188.277 I slot print_timing: id 0 | task 9 | total time = 533.71 ms / 72 tokens |
| 0.10.188.278 I slot print_timing: id 0 | task 9 | graphs reused = 13 |
| 0.10.188.314 I slot release: id 0 | task 9 | stop processing: n_tokens = 71, truncated = 0 |
| 0.10.188.328 I srv update_slots: all slots are idle |
| 0.10.222.983 I srv params_from_: Chat format: peg-native |
| 0.10.223.217 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.227 (> 0.100 thold), f_keep = 0.211 |
| 0.10.223.219 I srv get_availabl: updating prompt cache |
| 0.10.223.265 W srv prompt_save: - saving prompt with length 71, total state size = 151.985 MiB (draft: 0.000 MiB) |
| 0.10.600.669 I srv load: - looking for better prompt, base f_keep = 0.211, sim = 0.227 |
| 0.10.600.687 I srv update: - cache state: 2 prompts, 603.289 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.10.600.688 I srv update: - prompt 0x60f4303137d0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.10.600.690 I srv update: - prompt 0x60f42fe5fe40: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.10.600.691 I srv get_availabl: prompt cache update took 377.47 ms |
| 0.10.600.807 I slot launch_slot_: id 0 | task 20 | processing task, is_child = 0 |
| 0.10.600.821 W slot update_slots: id 0 | task 20 | erased invalidated context checkpoint (pos_min = 58, pos_max = 58, n_tokens = 59, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.10.722.851 I slot create_check: id 0 | task 20 | created context checkpoint 1 of 32 (pos_min = 61, pos_max = 61, n_tokens = 62, size = 149.626 MiB) |
| 0.11.173.504 I slot print_timing: id 0 | task 20 | prompt eval time = 173.74 ms / 66 tokens ( 2.63 ms per token, 379.87 tokens per second) |
| 0.11.173.510 I slot print_timing: id 0 | task 20 | eval time = 398.93 ms / 10 tokens ( 39.89 ms per token, 25.07 tokens per second) |
| 0.11.173.511 I slot print_timing: id 0 | task 20 | total time = 572.68 ms / 76 tokens |
| 0.11.173.512 I slot print_timing: id 0 | task 20 | graphs reused = 21 |
| 0.11.173.550 I slot release: id 0 | task 20 | stop processing: n_tokens = 75, truncated = 0 |
| 0.11.173.564 I srv update_slots: all slots are idle |
| 0.11.207.825 I srv params_from_: Chat format: peg-native |
| 0.11.208.055 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17477262306 |
| 0.11.208.058 I srv get_availabl: updating prompt cache |
| 0.11.208.095 W srv prompt_save: - saving prompt with length 75, total state size = 152.118 MiB (draft: 0.000 MiB) |
| 0.11.619.959 I srv load: - looking for better prompt, base f_keep = 0.040, sim = 0.036 |
| 0.11.619.972 I srv update: - cache state: 3 prompts, 905.034 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.11.619.973 I srv update: - prompt 0x60f4303137d0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.11.619.974 I srv update: - prompt 0x60f42fe5fe40: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.11.619.974 I srv update: - prompt 0x60f42f622220: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.11.619.976 I srv get_availabl: prompt cache update took 411.92 ms |
| 0.11.620.086 I slot launch_slot_: id 0 | task 32 | processing task, is_child = 0 |
| 0.11.620.098 W slot update_slots: id 0 | task 32 | erased invalidated context checkpoint (pos_min = 61, pos_max = 61, n_tokens = 62, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.11.753.412 I slot create_check: id 0 | task 32 | created context checkpoint 1 of 32 (pos_min = 78, pos_max = 78, n_tokens = 79, size = 149.626 MiB) |
| 0.12.708.902 I slot print_timing: id 0 | task 32 | prompt eval time = 184.97 ms / 83 tokens ( 2.23 ms per token, 448.72 tokens per second) |
| 0.12.708.907 I slot print_timing: id 0 | task 32 | eval time = 903.83 ms / 18 tokens ( 50.21 ms per token, 19.92 tokens per second) |
| 0.12.708.908 I slot print_timing: id 0 | task 32 | total time = 1088.80 ms / 101 tokens |
| 0.12.708.909 I slot print_timing: id 0 | task 32 | graphs reused = 37 |
| 0.12.708.936 I slot release: id 0 | task 32 | stop processing: n_tokens = 100, truncated = 0 |
| 0.12.708.947 I srv update_slots: all slots are idle |
| 0.12.732.764 I srv params_from_: Chat format: peg-native |
| 0.12.732.936 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17478797692 |
| 0.12.732.939 I srv get_availabl: updating prompt cache |
| 0.12.732.968 W srv prompt_save: - saving prompt with length 100, total state size = 152.949 MiB (draft: 0.000 MiB) |
| 0.12.933.116 I srv load: - looking for better prompt, base f_keep = 0.030, sim = 0.046 |
| 0.12.933.129 I srv update: - cache state: 4 prompts, 1207.608 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.12.933.130 I srv update: - prompt 0x60f4303137d0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.12.933.131 I srv update: - prompt 0x60f42fe5fe40: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.12.933.132 I srv update: - prompt 0x60f42f622220: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.12.933.132 I srv update: - prompt 0x60f4328f04b0: 100 tokens, checkpoints: 1, 302.575 MiB |
| 0.12.933.134 I srv get_availabl: prompt cache update took 200.19 ms |
| 0.12.933.238 I slot launch_slot_: id 0 | task 52 | processing task, is_child = 0 |
| 0.12.933.256 W slot update_slots: id 0 | task 52 | erased invalidated context checkpoint (pos_min = 78, pos_max = 78, n_tokens = 79, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.13.055.625 I slot create_check: id 0 | task 52 | created context checkpoint 1 of 32 (pos_min = 60, pos_max = 60, n_tokens = 61, size = 149.626 MiB) |
| 0.13.719.591 I slot print_timing: id 0 | task 52 | prompt eval time = 174.02 ms / 65 tokens ( 2.68 ms per token, 373.52 tokens per second) |
| 0.13.719.597 I slot print_timing: id 0 | task 52 | eval time = 612.31 ms / 12 tokens ( 51.03 ms per token, 19.60 tokens per second) |
| 0.13.719.598 I slot print_timing: id 0 | task 52 | total time = 786.33 ms / 77 tokens |
| 0.13.719.599 I slot print_timing: id 0 | task 52 | graphs reused = 47 |
| 0.13.719.625 I slot release: id 0 | task 52 | stop processing: n_tokens = 76, truncated = 0 |
| 0.13.719.638 I srv update_slots: all slots are idle |
| 0.13.749.101 I srv params_from_: Chat format: peg-native |
| 0.13.749.344 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17479808381 |
| 0.13.749.347 I srv get_availabl: updating prompt cache |
| 0.13.749.375 W srv prompt_save: - saving prompt with length 76, total state size = 152.151 MiB (draft: 0.000 MiB) |
| 0.13.945.474 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.039 |
| 0.13.945.483 I srv load: - found better prompt with f_keep = 0.340, sim = 0.442 |
| 0.13.992.879 I srv update: - cache state: 4 prompts, 1206.811 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.13.992.885 I srv update: - prompt 0x60f4303137d0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.13.992.886 I srv update: - prompt 0x60f42fe5fe40: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.13.992.886 I srv update: - prompt 0x60f42f622220: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.13.992.887 I srv update: - prompt 0x60f43046fe10: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.13.992.889 I srv get_availabl: prompt cache update took 243.54 ms |
| 0.13.993.001 I slot launch_slot_: id 0 | task 66 | processing task, is_child = 0 |
| 0.13.993.017 W slot update_slots: id 0 | task 66 | erased invalidated context checkpoint (pos_min = 78, pos_max = 78, n_tokens = 79, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.14.119.802 I slot create_check: id 0 | task 66 | created context checkpoint 1 of 32 (pos_min = 72, pos_max = 72, n_tokens = 73, size = 149.626 MiB) |
| 0.15.377.079 I slot print_timing: id 0 | task 66 | prompt eval time = 178.49 ms / 77 tokens ( 2.32 ms per token, 431.40 tokens per second) |
| 0.15.377.085 I slot print_timing: id 0 | task 66 | eval time = 1205.57 ms / 25 tokens ( 48.22 ms per token, 20.74 tokens per second) |
| 0.15.377.086 I slot print_timing: id 0 | task 66 | total time = 1384.06 ms / 102 tokens |
| 0.15.377.087 I slot print_timing: id 0 | task 66 | graphs reused = 70 |
| 0.15.377.114 I slot release: id 0 | task 66 | stop processing: n_tokens = 101, truncated = 0 |
| 0.15.377.127 I srv update_slots: all slots are idle |
| 0.15.593.716 I srv params_from_: Chat format: peg-native |
| 0.15.593.898 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.393 (> 0.100 thold), f_keep = 0.347 |
| 0.15.593.902 I srv get_availabl: updating prompt cache |
| 0.15.593.943 W srv prompt_save: - saving prompt with length 101, total state size = 152.982 MiB (draft: 0.000 MiB) |
| 0.15.792.215 I srv load: - looking for better prompt, base f_keep = 0.347, sim = 0.393 |
| 0.15.792.227 I srv update: - cache state: 5 prompts, 1509.419 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.15.792.228 I srv update: - prompt 0x60f4303137d0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.15.792.228 I srv update: - prompt 0x60f42fe5fe40: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.15.792.229 I srv update: - prompt 0x60f42f622220: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.15.792.230 I srv update: - prompt 0x60f43046fe10: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.15.792.230 I srv update: - prompt 0x60f42f80f2b0: 101 tokens, checkpoints: 1, 302.608 MiB |
| 0.15.792.232 I srv get_availabl: prompt cache update took 198.33 ms |
| 0.15.792.345 I slot launch_slot_: id 0 | task 93 | processing task, is_child = 0 |
| 0.15.792.358 W slot update_slots: id 0 | task 93 | erased invalidated context checkpoint (pos_min = 72, pos_max = 72, n_tokens = 73, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.15.926.194 I slot create_check: id 0 | task 93 | created context checkpoint 1 of 32 (pos_min = 84, pos_max = 84, n_tokens = 85, size = 149.626 MiB) |
| 0.18.590.921 I slot print_timing: id 0 | task 93 | prompt eval time = 185.52 ms / 89 tokens ( 2.08 ms per token, 479.73 tokens per second) |
| 0.18.590.930 I slot print_timing: id 0 | task 93 | eval time = 2613.03 ms / 50 tokens ( 52.26 ms per token, 19.13 tokens per second) |
| 0.18.590.931 I slot print_timing: id 0 | task 93 | total time = 2798.55 ms / 139 tokens |
| 0.18.590.932 I slot print_timing: id 0 | task 93 | graphs reused = 118 |
| 0.18.590.969 I slot release: id 0 | task 93 | stop processing: n_tokens = 138, truncated = 0 |
| 0.18.590.980 I srv update_slots: all slots are idle |
| 0.18.630.434 I srv params_from_: Chat format: peg-native |
| 0.18.630.695 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17484679725 |
| 0.18.630.699 I srv get_availabl: updating prompt cache |
| 0.18.630.745 W srv prompt_save: - saving prompt with length 138, total state size = 154.211 MiB (draft: 0.000 MiB) |
| 0.18.830.723 I srv load: - looking for better prompt, base f_keep = 0.022, sim = 0.050 |
| 0.18.830.737 I srv update: - cache state: 6 prompts, 1813.256 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.18.830.738 I srv update: - prompt 0x60f4303137d0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.18.830.739 I srv update: - prompt 0x60f42fe5fe40: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.18.830.739 I srv update: - prompt 0x60f42f622220: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.18.830.740 I srv update: - prompt 0x60f43046fe10: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.18.830.741 I srv update: - prompt 0x60f42f80f2b0: 101 tokens, checkpoints: 1, 302.608 MiB |
| 0.18.830.741 I srv update: - prompt 0x60f42f89f6b0: 138 tokens, checkpoints: 1, 303.837 MiB |
| 0.18.830.743 I srv get_availabl: prompt cache update took 200.04 ms |
| 0.18.830.850 I slot launch_slot_: id 0 | task 145 | processing task, is_child = 0 |
| 0.18.830.864 W slot update_slots: id 0 | task 145 | erased invalidated context checkpoint (pos_min = 84, pos_max = 84, n_tokens = 85, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.18.951.663 I slot create_check: id 0 | task 145 | created context checkpoint 1 of 32 (pos_min = 55, pos_max = 55, n_tokens = 56, size = 149.626 MiB) |
| 0.19.358.717 I slot print_timing: id 0 | task 145 | prompt eval time = 172.42 ms / 60 tokens ( 2.87 ms per token, 347.99 tokens per second) |
| 0.19.358.723 I slot print_timing: id 0 | task 145 | eval time = 355.42 ms / 9 tokens ( 39.49 ms per token, 25.32 tokens per second) |
| 0.19.358.723 I slot print_timing: id 0 | task 145 | total time = 527.85 ms / 69 tokens |
| 0.19.358.724 I slot print_timing: id 0 | task 145 | graphs reused = 125 |
| 0.19.358.750 I slot release: id 0 | task 145 | stop processing: n_tokens = 68, truncated = 0 |
| 0.19.358.762 I srv update_slots: all slots are idle |
| 0.19.557.172 I srv params_from_: Chat format: peg-native |
| 0.19.557.349 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.183 (> 0.100 thold), f_keep = 0.221 |
| 0.19.557.353 I srv get_availabl: updating prompt cache |
| 0.19.557.397 W srv prompt_save: - saving prompt with length 68, total state size = 151.886 MiB (draft: 0.000 MiB) |
| 0.19.754.329 I srv load: - looking for better prompt, base f_keep = 0.221, sim = 0.183 |
| 0.19.754.344 I srv update: - cache state: 7 prompts, 2114.768 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.19.754.345 I srv update: - prompt 0x60f4303137d0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.19.754.346 I srv update: - prompt 0x60f42fe5fe40: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.19.754.346 I srv update: - prompt 0x60f42f622220: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.19.754.347 I srv update: - prompt 0x60f43046fe10: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.19.754.348 I srv update: - prompt 0x60f42f80f2b0: 101 tokens, checkpoints: 1, 302.608 MiB |
| 0.19.754.348 I srv update: - prompt 0x60f42f89f6b0: 138 tokens, checkpoints: 1, 303.837 MiB |
| 0.19.754.349 I srv update: - prompt 0x60f4328f6e30: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.19.754.351 I srv get_availabl: prompt cache update took 197.00 ms |
| 0.19.754.457 I slot launch_slot_: id 0 | task 156 | processing task, is_child = 0 |
| 0.19.754.469 W slot update_slots: id 0 | task 156 | erased invalidated context checkpoint (pos_min = 55, pos_max = 55, n_tokens = 56, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.19.882.548 I slot create_check: id 0 | task 156 | created context checkpoint 1 of 32 (pos_min = 77, pos_max = 77, n_tokens = 78, size = 149.626 MiB) |
| 0.20.752.578 I slot print_timing: id 0 | task 156 | prompt eval time = 179.77 ms / 82 tokens ( 2.19 ms per token, 456.14 tokens per second) |
| 0.20.752.583 I slot print_timing: id 0 | task 156 | eval time = 818.33 ms / 16 tokens ( 51.15 ms per token, 19.55 tokens per second) |
| 0.20.752.584 I slot print_timing: id 0 | task 156 | total time = 998.10 ms / 98 tokens |
| 0.20.752.585 I slot print_timing: id 0 | task 156 | graphs reused = 139 |
| 0.20.752.613 I slot release: id 0 | task 156 | stop processing: n_tokens = 97, truncated = 0 |
| 0.20.752.628 I srv update_slots: all slots are idle |
| 0.20.786.968 I srv params_from_: Chat format: peg-native |
| 0.20.787.253 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17486841369 |
| 0.20.787.254 I srv get_availabl: updating prompt cache |
| 0.20.787.282 W srv prompt_save: - saving prompt with length 97, total state size = 152.849 MiB (draft: 0.000 MiB) |
| 0.20.984.220 I srv load: - looking for better prompt, base f_keep = 0.031, sim = 0.033 |
| 0.20.984.233 I srv load: - found better prompt with f_keep = 0.337, sim = 0.374 |
| 0.21.031.737 I srv update: - cache state: 7 prompts, 2114.635 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.21.031.744 I srv update: - prompt 0x60f4303137d0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.21.031.745 I srv update: - prompt 0x60f42fe5fe40: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.21.031.745 I srv update: - prompt 0x60f42f622220: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.21.031.746 I srv update: - prompt 0x60f43046fe10: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.21.031.747 I srv update: - prompt 0x60f42f89f6b0: 138 tokens, checkpoints: 1, 303.837 MiB |
| 0.21.031.747 I srv update: - prompt 0x60f4328f6e30: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.21.031.748 I srv update: - prompt 0x60f43027cdc0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.21.031.750 I srv get_availabl: prompt cache update took 244.50 ms |
| 0.21.031.866 I slot launch_slot_: id 0 | task 174 | processing task, is_child = 0 |
| 0.21.031.880 W slot update_slots: id 0 | task 174 | erased invalidated context checkpoint (pos_min = 72, pos_max = 72, n_tokens = 73, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.21.165.152 I slot create_check: id 0 | task 174 | created context checkpoint 1 of 32 (pos_min = 86, pos_max = 86, n_tokens = 87, size = 149.626 MiB) |
| 0.22.373.178 I slot print_timing: id 0 | task 174 | prompt eval time = 184.92 ms / 91 tokens ( 2.03 ms per token, 492.12 tokens per second) |
| 0.22.373.184 I slot print_timing: id 0 | task 174 | eval time = 1156.38 ms / 24 tokens ( 48.18 ms per token, 20.75 tokens per second) |
| 0.22.373.185 I slot print_timing: id 0 | task 174 | total time = 1341.29 ms / 115 tokens |
| 0.22.373.186 I slot print_timing: id 0 | task 174 | graphs reused = 161 |
| 0.22.373.216 I slot release: id 0 | task 174 | stop processing: n_tokens = 114, truncated = 0 |
| 0.22.373.227 I srv update_slots: all slots are idle |
| 0.22.566.144 I srv params_from_: Chat format: peg-native |
| 0.22.566.331 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17488461972 |
| 0.22.566.335 I srv get_availabl: updating prompt cache |
| 0.22.566.381 W srv prompt_save: - saving prompt with length 114, total state size = 153.414 MiB (draft: 0.000 MiB) |
| 0.22.764.750 I srv load: - looking for better prompt, base f_keep = 0.026, sim = 0.024 |
| 0.22.764.763 I srv update: - cache state: 8 prompts, 2417.675 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.22.764.764 I srv update: - prompt 0x60f4303137d0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.22.764.765 I srv update: - prompt 0x60f42fe5fe40: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.22.764.766 I srv update: - prompt 0x60f42f622220: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.22.764.766 I srv update: - prompt 0x60f43046fe10: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.22.764.767 I srv update: - prompt 0x60f42f89f6b0: 138 tokens, checkpoints: 1, 303.837 MiB |
| 0.22.764.768 I srv update: - prompt 0x60f4328f6e30: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.22.764.768 I srv update: - prompt 0x60f43027cdc0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.22.764.769 I srv update: - prompt 0x60f42fe60130: 114 tokens, checkpoints: 1, 303.040 MiB |
| 0.22.764.771 I srv get_availabl: prompt cache update took 198.44 ms |
| 0.22.764.879 I slot launch_slot_: id 0 | task 200 | processing task, is_child = 0 |
| 0.22.764.893 W slot update_slots: id 0 | task 200 | erased invalidated context checkpoint (pos_min = 86, pos_max = 86, n_tokens = 87, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.22.922.444 I slot create_check: id 0 | task 200 | created context checkpoint 1 of 32 (pos_min = 119, pos_max = 119, n_tokens = 120, size = 149.626 MiB) |
| 0.28.172.672 I slot print_timing: id 0 | task 200 | n_decoded = 100, tg = 19.24 t/s |
| 0.29.138.320 I slot print_timing: id 0 | task 200 | prompt eval time = 209.18 ms / 124 tokens ( 1.69 ms per token, 592.80 tokens per second) |
| 0.29.138.326 I slot print_timing: id 0 | task 200 | eval time = 6164.24 ms / 119 tokens ( 51.80 ms per token, 19.30 tokens per second) |
| 0.29.138.327 I slot print_timing: id 0 | task 200 | total time = 6373.42 ms / 243 tokens |
| 0.29.138.328 I slot print_timing: id 0 | task 200 | graphs reused = 278 |
| 0.29.138.360 I slot release: id 0 | task 200 | stop processing: n_tokens = 242, truncated = 0 |
| 0.29.138.373 I srv update_slots: all slots are idle |
| 0.29.179.724 I srv params_from_: Chat format: peg-native |
| 0.29.179.889 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.129 (> 0.100 thold), f_keep = 0.050 |
| 0.29.179.893 I srv get_availabl: updating prompt cache |
| 0.29.179.930 W srv prompt_save: - saving prompt with length 242, total state size = 157.666 MiB (draft: 0.000 MiB) |
| 0.29.376.471 I srv load: - looking for better prompt, base f_keep = 0.050, sim = 0.129 |
| 0.29.376.482 I srv update: - cache state: 9 prompts, 2724.968 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.29.376.483 I srv update: - prompt 0x60f4303137d0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.29.376.484 I srv update: - prompt 0x60f42fe5fe40: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.29.376.486 I srv update: - prompt 0x60f42f622220: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.29.376.486 I srv update: - prompt 0x60f43046fe10: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.29.376.487 I srv update: - prompt 0x60f42f89f6b0: 138 tokens, checkpoints: 1, 303.837 MiB |
| 0.29.376.487 I srv update: - prompt 0x60f4328f6e30: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.29.376.488 I srv update: - prompt 0x60f43027cdc0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.29.376.489 I srv update: - prompt 0x60f42fe60130: 114 tokens, checkpoints: 1, 303.040 MiB |
| 0.29.376.489 I srv update: - prompt 0x60f42f89f760: 242 tokens, checkpoints: 1, 307.292 MiB |
| 0.29.376.491 I srv get_availabl: prompt cache update took 196.60 ms |
| 0.29.376.590 I slot launch_slot_: id 0 | task 321 | processing task, is_child = 0 |
| 0.29.376.602 W slot update_slots: id 0 | task 321 | erased invalidated context checkpoint (pos_min = 119, pos_max = 119, n_tokens = 120, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.29.668.112 I slot create_check: id 0 | task 321 | created context checkpoint 1 of 32 (pos_min = 88, pos_max = 88, n_tokens = 89, size = 149.626 MiB) |
| 0.34.358.031 I slot print_timing: id 0 | task 321 | prompt eval time = 343.05 ms / 93 tokens ( 3.69 ms per token, 271.10 tokens per second) |
| 0.34.358.036 I slot print_timing: id 0 | task 321 | eval time = 4638.37 ms / 94 tokens ( 49.34 ms per token, 20.27 tokens per second) |
| 0.34.358.037 I slot print_timing: id 0 | task 321 | total time = 4981.42 ms / 187 tokens |
| 0.34.358.037 I slot print_timing: id 0 | task 321 | graphs reused = 370 |
| 0.34.358.070 I slot release: id 0 | task 321 | stop processing: n_tokens = 186, truncated = 0 |
| 0.34.358.081 I srv update_slots: all slots are idle |
| 0.34.410.355 I srv params_from_: Chat format: peg-native |
| 0.34.410.651 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.138 (> 0.100 thold), f_keep = 0.065 |
| 0.34.410.653 I srv get_availabl: updating prompt cache |
| 0.34.410.721 W srv prompt_save: - saving prompt with length 186, total state size = 155.806 MiB (draft: 0.000 MiB) |
| 0.34.793.569 I srv load: - looking for better prompt, base f_keep = 0.065, sim = 0.138 |
| 0.34.793.580 I srv update: - cache state: 10 prompts, 3030.400 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.34.793.581 I srv update: - prompt 0x60f4303137d0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.34.793.582 I srv update: - prompt 0x60f42fe5fe40: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.34.793.583 I srv update: - prompt 0x60f42f622220: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.34.793.584 I srv update: - prompt 0x60f43046fe10: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.34.793.584 I srv update: - prompt 0x60f42f89f6b0: 138 tokens, checkpoints: 1, 303.837 MiB |
| 0.34.793.585 I srv update: - prompt 0x60f4328f6e30: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.34.793.586 I srv update: - prompt 0x60f43027cdc0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.34.793.588 I srv update: - prompt 0x60f42fe60130: 114 tokens, checkpoints: 1, 303.040 MiB |
| 0.34.793.589 I srv update: - prompt 0x60f42f89f760: 242 tokens, checkpoints: 1, 307.292 MiB |
| 0.34.793.590 I srv update: - prompt 0x60f432a46d90: 186 tokens, checkpoints: 1, 305.432 MiB |
| 0.34.793.591 I srv get_availabl: prompt cache update took 382.94 ms |
| 0.34.793.694 I slot launch_slot_: id 0 | task 417 | processing task, is_child = 0 |
| 0.34.793.706 W slot update_slots: id 0 | task 417 | erased invalidated context checkpoint (pos_min = 88, pos_max = 88, n_tokens = 89, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.34.928.365 I slot create_check: id 0 | task 417 | created context checkpoint 1 of 32 (pos_min = 82, pos_max = 82, n_tokens = 83, size = 149.626 MiB) |
| 0.40.102.579 I slot print_timing: id 0 | task 417 | n_decoded = 100, tg = 19.52 t/s |
| 0.42.079.765 I slot print_timing: id 0 | task 417 | prompt eval time = 186.20 ms / 87 tokens ( 2.14 ms per token, 467.24 tokens per second) |
| 0.42.079.771 I slot print_timing: id 0 | task 417 | eval time = 7099.85 ms / 139 tokens ( 51.08 ms per token, 19.58 tokens per second) |
| 0.42.079.772 I slot print_timing: id 0 | task 417 | total time = 7286.05 ms / 226 tokens |
| 0.42.079.772 I slot print_timing: id 0 | task 417 | graphs reused = 507 |
| 0.42.079.802 I slot release: id 0 | task 417 | stop processing: n_tokens = 225, truncated = 0 |
| 0.42.079.812 I srv update_slots: all slots are idle |
| 0.42.124.357 I srv params_from_: Chat format: peg-native |
| 0.42.124.591 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.053 |
| 0.42.124.595 I srv get_availabl: updating prompt cache |
| 0.42.124.644 W srv prompt_save: - saving prompt with length 225, total state size = 157.102 MiB (draft: 0.000 MiB) |
| 0.42.321.818 I srv load: - looking for better prompt, base f_keep = 0.053, sim = 0.107 |
| 0.42.321.833 I srv update: - cache state: 11 prompts, 3337.127 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.42.321.834 I srv update: - prompt 0x60f4303137d0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.42.321.835 I srv update: - prompt 0x60f42fe5fe40: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.42.321.836 I srv update: - prompt 0x60f42f622220: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.42.321.837 I srv update: - prompt 0x60f43046fe10: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.42.321.837 I srv update: - prompt 0x60f42f89f6b0: 138 tokens, checkpoints: 1, 303.837 MiB |
| 0.42.321.838 I srv update: - prompt 0x60f4328f6e30: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.42.321.838 I srv update: - prompt 0x60f43027cdc0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.42.321.839 I srv update: - prompt 0x60f42fe60130: 114 tokens, checkpoints: 1, 303.040 MiB |
| 0.42.321.839 I srv update: - prompt 0x60f42f89f760: 242 tokens, checkpoints: 1, 307.292 MiB |
| 0.42.321.840 I srv update: - prompt 0x60f432a46d90: 186 tokens, checkpoints: 1, 305.432 MiB |
| 0.42.321.840 I srv update: - prompt 0x60f4328f04b0: 225 tokens, checkpoints: 1, 306.728 MiB |
| 0.42.321.842 I srv get_availabl: prompt cache update took 197.25 ms |
| 0.42.321.939 I slot launch_slot_: id 0 | task 558 | processing task, is_child = 0 |
| 0.42.321.949 W slot update_slots: id 0 | task 558 | erased invalidated context checkpoint (pos_min = 82, pos_max = 82, n_tokens = 83, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.42.697.580 I slot create_check: id 0 | task 558 | created context checkpoint 1 of 32 (pos_min = 107, pos_max = 107, n_tokens = 108, size = 149.626 MiB) |
| 0.47.887.349 I slot print_timing: id 0 | task 558 | n_decoded = 100, tg = 19.47 t/s |
| 0.48.018.188 I slot print_timing: id 0 | task 558 | prompt eval time = 428.37 ms / 112 tokens ( 3.82 ms per token, 261.45 tokens per second) |
| 0.48.018.194 I slot print_timing: id 0 | task 558 | eval time = 5267.86 ms / 103 tokens ( 51.14 ms per token, 19.55 tokens per second) |
| 0.48.018.195 I slot print_timing: id 0 | task 558 | total time = 5696.24 ms / 215 tokens |
| 0.48.018.196 I slot print_timing: id 0 | task 558 | graphs reused = 608 |
| 0.48.018.226 I slot release: id 0 | task 558 | stop processing: n_tokens = 214, truncated = 0 |
| 0.48.018.237 I srv update_slots: all slots are idle |
| 0.48.070.536 I srv params_from_: Chat format: peg-native |
| 0.48.070.769 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.136 (> 0.100 thold), f_keep = 0.056 |
| 0.48.070.772 I srv get_availabl: updating prompt cache |
| 0.48.070.803 W srv prompt_save: - saving prompt with length 214, total state size = 156.736 MiB (draft: 0.000 MiB) |
| 0.48.260.728 I srv load: - looking for better prompt, base f_keep = 0.056, sim = 0.136 |
| 0.48.260.742 I srv update: - cache state: 12 prompts, 3643.489 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.48.260.743 I srv update: - prompt 0x60f4303137d0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.48.260.744 I srv update: - prompt 0x60f42fe5fe40: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.48.260.744 I srv update: - prompt 0x60f42f622220: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.48.260.745 I srv update: - prompt 0x60f43046fe10: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.48.260.746 I srv update: - prompt 0x60f42f89f6b0: 138 tokens, checkpoints: 1, 303.837 MiB |
| 0.48.260.746 I srv update: - prompt 0x60f4328f6e30: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.48.260.747 I srv update: - prompt 0x60f43027cdc0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.48.260.748 I srv update: - prompt 0x60f42fe60130: 114 tokens, checkpoints: 1, 303.040 MiB |
| 0.48.260.748 I srv update: - prompt 0x60f42f89f760: 242 tokens, checkpoints: 1, 307.292 MiB |
| 0.48.260.749 I srv update: - prompt 0x60f432a46d90: 186 tokens, checkpoints: 1, 305.432 MiB |
| 0.48.260.749 I srv update: - prompt 0x60f4328f04b0: 225 tokens, checkpoints: 1, 306.728 MiB |
| 0.48.260.750 I srv update: - prompt 0x60f42feac760: 214 tokens, checkpoints: 1, 306.362 MiB |
| 0.48.260.751 I srv get_availabl: prompt cache update took 189.98 ms |
| 0.48.260.851 I slot launch_slot_: id 0 | task 663 | processing task, is_child = 0 |
| 0.48.260.861 W slot update_slots: id 0 | task 663 | erased invalidated context checkpoint (pos_min = 107, pos_max = 107, n_tokens = 108, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.48.640.733 I slot create_check: id 0 | task 663 | created context checkpoint 1 of 32 (pos_min = 83, pos_max = 83, n_tokens = 84, size = 149.626 MiB) |
| 0.53.322.659 I slot print_timing: id 0 | task 663 | prompt eval time = 431.38 ms / 88 tokens ( 4.90 ms per token, 204.00 tokens per second) |
| 0.53.322.665 I slot print_timing: id 0 | task 663 | eval time = 4630.41 ms / 93 tokens ( 49.79 ms per token, 20.08 tokens per second) |
| 0.53.322.665 I slot print_timing: id 0 | task 663 | total time = 5061.79 ms / 181 tokens |
| 0.53.322.666 I slot print_timing: id 0 | task 663 | graphs reused = 699 |
| 0.53.322.694 I slot release: id 0 | task 663 | stop processing: n_tokens = 180, truncated = 0 |
| 0.53.322.704 I srv update_slots: all slots are idle |
| 0.53.373.600 I srv params_from_: Chat format: peg-native |
| 0.53.373.829 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.114 (> 0.100 thold), f_keep = 0.067 |
| 0.53.373.832 I srv get_availabl: updating prompt cache |
| 0.53.373.863 W srv prompt_save: - saving prompt with length 180, total state size = 155.607 MiB (draft: 0.000 MiB) |
| 0.53.831.338 I srv load: - looking for better prompt, base f_keep = 0.067, sim = 0.114 |
| 0.53.831.352 I srv update: - cache state: 13 prompts, 3948.722 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.53.831.353 I srv update: - prompt 0x60f4303137d0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.53.831.354 I srv update: - prompt 0x60f42fe5fe40: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.53.831.355 I srv update: - prompt 0x60f42f622220: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.53.831.357 I srv update: - prompt 0x60f43046fe10: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.53.831.358 I srv update: - prompt 0x60f42f89f6b0: 138 tokens, checkpoints: 1, 303.837 MiB |
| 0.53.831.359 I srv update: - prompt 0x60f4328f6e30: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.53.831.360 I srv update: - prompt 0x60f43027cdc0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.53.831.360 I srv update: - prompt 0x60f42fe60130: 114 tokens, checkpoints: 1, 303.040 MiB |
| 0.53.831.361 I srv update: - prompt 0x60f42f89f760: 242 tokens, checkpoints: 1, 307.292 MiB |
| 0.53.831.361 I srv update: - prompt 0x60f432a46d90: 186 tokens, checkpoints: 1, 305.432 MiB |
| 0.53.831.362 I srv update: - prompt 0x60f4328f04b0: 225 tokens, checkpoints: 1, 306.728 MiB |
| 0.53.831.362 I srv update: - prompt 0x60f42feac760: 214 tokens, checkpoints: 1, 306.362 MiB |
| 0.53.831.363 I srv update: - prompt 0x60f4328f6a70: 180 tokens, checkpoints: 1, 305.233 MiB |
| 0.53.831.364 I srv get_availabl: prompt cache update took 457.53 ms |
| 0.53.831.462 I slot launch_slot_: id 0 | task 758 | processing task, is_child = 0 |
| 0.53.831.474 W slot update_slots: id 0 | task 758 | erased invalidated context checkpoint (pos_min = 83, pos_max = 83, n_tokens = 84, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.53.975.455 I slot create_check: id 0 | task 758 | created context checkpoint 1 of 32 (pos_min = 100, pos_max = 100, n_tokens = 101, size = 149.626 MiB) |
| 0.59.268.981 I slot print_timing: id 0 | task 758 | n_decoded = 100, tg = 19.08 t/s |
| 1.02.296.653 I slot print_timing: id 0 | task 758 | n_decoded = 157, tg = 18.98 t/s |
| 1.03.807.553 I slot print_timing: id 0 | task 758 | prompt eval time = 195.47 ms / 105 tokens ( 1.86 ms per token, 537.16 tokens per second) |
| 1.03.807.558 I slot print_timing: id 0 | task 758 | eval time = 9780.60 ms / 184 tokens ( 53.16 ms per token, 18.81 tokens per second) |
| 1.03.807.560 I slot print_timing: id 0 | task 758 | total time = 9976.07 ms / 289 tokens |
| 1.03.807.560 I slot print_timing: id 0 | task 758 | graphs reused = 880 |
| 1.03.807.589 I slot release: id 0 | task 758 | stop processing: n_tokens = 288, truncated = 0 |
| 1.03.807.601 I srv update_slots: all slots are idle |
| 1.03.859.502 I srv params_from_: Chat format: peg-native |
| 1.03.859.747 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.133 (> 0.100 thold), f_keep = 0.042 |
| 1.03.859.750 I srv get_availabl: updating prompt cache |
| 1.03.859.784 W srv prompt_save: - saving prompt with length 288, total state size = 159.194 MiB (draft: 0.000 MiB) |
| 1.04.059.632 I srv load: - looking for better prompt, base f_keep = 0.042, sim = 0.133 |
| 1.04.059.648 I srv update: - cache state: 14 prompts, 4257.543 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.04.059.649 I srv update: - prompt 0x60f4303137d0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.04.059.649 I srv update: - prompt 0x60f42fe5fe40: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.04.059.650 I srv update: - prompt 0x60f42f622220: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.04.059.651 I srv update: - prompt 0x60f43046fe10: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.04.059.651 I srv update: - prompt 0x60f42f89f6b0: 138 tokens, checkpoints: 1, 303.837 MiB |
| 1.04.059.652 I srv update: - prompt 0x60f4328f6e30: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.04.059.652 I srv update: - prompt 0x60f43027cdc0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.04.059.653 I srv update: - prompt 0x60f42fe60130: 114 tokens, checkpoints: 1, 303.040 MiB |
| 1.04.059.653 I srv update: - prompt 0x60f42f89f760: 242 tokens, checkpoints: 1, 307.292 MiB |
| 1.04.059.655 I srv update: - prompt 0x60f432a46d90: 186 tokens, checkpoints: 1, 305.432 MiB |
| 1.04.059.655 I srv update: - prompt 0x60f4328f04b0: 225 tokens, checkpoints: 1, 306.728 MiB |
| 1.04.059.661 I srv update: - prompt 0x60f42feac760: 214 tokens, checkpoints: 1, 306.362 MiB |
| 1.04.059.662 I srv update: - prompt 0x60f4328f6a70: 180 tokens, checkpoints: 1, 305.233 MiB |
| 1.04.059.663 I srv update: - prompt 0x60f4303523b0: 288 tokens, checkpoints: 1, 308.821 MiB |
| 1.04.059.664 I srv get_availabl: prompt cache update took 199.91 ms |
| 1.04.059.763 I slot launch_slot_: id 0 | task 944 | processing task, is_child = 0 |
| 1.04.059.774 W slot update_slots: id 0 | task 944 | erased invalidated context checkpoint (pos_min = 100, pos_max = 100, n_tokens = 101, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.04.193.108 I slot create_check: id 0 | task 944 | created context checkpoint 1 of 32 (pos_min = 85, pos_max = 85, n_tokens = 86, size = 149.626 MiB) |
| 1.09.278.564 I slot print_timing: id 0 | task 944 | n_decoded = 100, tg = 19.87 t/s |
| 1.10.112.707 I slot print_timing: id 0 | task 944 | prompt eval time = 184.87 ms / 90 tokens ( 2.05 ms per token, 486.83 tokens per second) |
| 1.10.112.712 I slot print_timing: id 0 | task 944 | eval time = 5868.06 ms / 114 tokens ( 51.47 ms per token, 19.43 tokens per second) |
| 1.10.112.713 I slot print_timing: id 0 | task 944 | total time = 6052.93 ms / 204 tokens |
| 1.10.112.714 I slot print_timing: id 0 | task 944 | graphs reused = 992 |
| 1.10.112.743 I slot release: id 0 | task 944 | stop processing: n_tokens = 203, truncated = 0 |
| 1.10.112.754 I srv update_slots: all slots are idle |
| 1.10.160.917 I srv params_from_: Chat format: peg-native |
| 1.10.161.149 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17536201499 |
| 1.10.161.153 I srv get_availabl: updating prompt cache |
| 1.10.161.208 W srv prompt_save: - saving prompt with length 203, total state size = 156.371 MiB (draft: 0.000 MiB) |
| 1.10.352.420 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.060 |
| 1.10.352.433 I srv update: - cache state: 15 prompts, 4563.539 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.10.352.434 I srv update: - prompt 0x60f4303137d0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.10.352.435 I srv update: - prompt 0x60f42fe5fe40: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.10.352.435 I srv update: - prompt 0x60f42f622220: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.10.352.436 I srv update: - prompt 0x60f43046fe10: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.10.352.437 I srv update: - prompt 0x60f42f89f6b0: 138 tokens, checkpoints: 1, 303.837 MiB |
| 1.10.352.437 I srv update: - prompt 0x60f4328f6e30: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.10.352.438 I srv update: - prompt 0x60f43027cdc0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.10.352.438 I srv update: - prompt 0x60f42fe60130: 114 tokens, checkpoints: 1, 303.040 MiB |
| 1.10.352.439 I srv update: - prompt 0x60f42f89f760: 242 tokens, checkpoints: 1, 307.292 MiB |
| 1.10.352.439 I srv update: - prompt 0x60f432a46d90: 186 tokens, checkpoints: 1, 305.432 MiB |
| 1.10.352.440 I srv update: - prompt 0x60f4328f04b0: 225 tokens, checkpoints: 1, 306.728 MiB |
| 1.10.352.440 I srv update: - prompt 0x60f42feac760: 214 tokens, checkpoints: 1, 306.362 MiB |
| 1.10.352.441 I srv update: - prompt 0x60f4328f6a70: 180 tokens, checkpoints: 1, 305.233 MiB |
| 1.10.352.441 I srv update: - prompt 0x60f4303523b0: 288 tokens, checkpoints: 1, 308.821 MiB |
| 1.10.352.442 I srv update: - prompt 0x60f4306c9cf0: 203 tokens, checkpoints: 1, 305.997 MiB |
| 1.10.352.443 I srv get_availabl: prompt cache update took 191.29 ms |
| 1.10.352.539 I slot launch_slot_: id 0 | task 1060 | processing task, is_child = 0 |
| 1.10.352.550 W slot update_slots: id 0 | task 1060 | erased invalidated context checkpoint (pos_min = 85, pos_max = 85, n_tokens = 86, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.10.731.131 I slot create_check: id 0 | task 1060 | created context checkpoint 1 of 32 (pos_min = 128, pos_max = 128, n_tokens = 129, size = 149.626 MiB) |
| 1.15.953.241 I slot print_timing: id 0 | task 1060 | n_decoded = 100, tg = 19.34 t/s |
| 1.18.797.213 I slot print_timing: id 0 | task 1060 | prompt eval time = 430.07 ms / 133 tokens ( 3.23 ms per token, 309.25 tokens per second) |
| 1.18.797.219 I slot print_timing: id 0 | task 1060 | eval time = 8014.59 ms / 156 tokens ( 51.38 ms per token, 19.46 tokens per second) |
| 1.18.797.219 I slot print_timing: id 0 | task 1060 | total time = 8444.66 ms / 289 tokens |
| 1.18.797.220 I slot print_timing: id 0 | task 1060 | graphs reused = 1145 |
| 1.18.797.255 I slot release: id 0 | task 1060 | stop processing: n_tokens = 288, truncated = 0 |
| 1.18.797.268 I srv update_slots: all slots are idle |
| 1.18.849.033 I srv params_from_: Chat format: peg-native |
| 1.18.849.261 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17544886011 |
| 1.18.849.265 I srv get_availabl: updating prompt cache |
| 1.18.849.297 W srv prompt_save: - saving prompt with length 288, total state size = 159.194 MiB (draft: 0.000 MiB) |
| 1.19.043.051 I srv load: - looking for better prompt, base f_keep = 0.028, sim = 0.075 |
| 1.19.043.067 I srv update: - cache state: 16 prompts, 4872.360 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.19.043.068 I srv update: - prompt 0x60f4303137d0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.19.043.069 I srv update: - prompt 0x60f42fe5fe40: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.19.043.070 I srv update: - prompt 0x60f42f622220: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.19.043.070 I srv update: - prompt 0x60f43046fe10: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.19.043.071 I srv update: - prompt 0x60f42f89f6b0: 138 tokens, checkpoints: 1, 303.837 MiB |
| 1.19.043.071 I srv update: - prompt 0x60f4328f6e30: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.19.043.072 I srv update: - prompt 0x60f43027cdc0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.19.043.072 I srv update: - prompt 0x60f42fe60130: 114 tokens, checkpoints: 1, 303.040 MiB |
| 1.19.043.073 I srv update: - prompt 0x60f42f89f760: 242 tokens, checkpoints: 1, 307.292 MiB |
| 1.19.043.074 I srv update: - prompt 0x60f432a46d90: 186 tokens, checkpoints: 1, 305.432 MiB |
| 1.19.043.074 I srv update: - prompt 0x60f4328f04b0: 225 tokens, checkpoints: 1, 306.728 MiB |
| 1.19.043.075 I srv update: - prompt 0x60f42feac760: 214 tokens, checkpoints: 1, 306.362 MiB |
| 1.19.043.075 I srv update: - prompt 0x60f4328f6a70: 180 tokens, checkpoints: 1, 305.233 MiB |
| 1.19.043.076 I srv update: - prompt 0x60f4303523b0: 288 tokens, checkpoints: 1, 308.821 MiB |
| 1.19.043.076 I srv update: - prompt 0x60f4306c9cf0: 203 tokens, checkpoints: 1, 305.997 MiB |
| 1.19.043.077 I srv update: - prompt 0x60f4301a2fe0: 288 tokens, checkpoints: 1, 308.821 MiB |
| 1.19.043.078 I srv get_availabl: prompt cache update took 193.81 ms |
| 1.19.043.176 I slot launch_slot_: id 0 | task 1218 | processing task, is_child = 0 |
| 1.19.043.187 W slot update_slots: id 0 | task 1218 | erased invalidated context checkpoint (pos_min = 128, pos_max = 128, n_tokens = 129, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.19.187.650 I slot create_check: id 0 | task 1218 | created context checkpoint 1 of 32 (pos_min = 101, pos_max = 101, n_tokens = 102, size = 149.626 MiB) |
| 1.22.189.291 I slot print_timing: id 0 | task 1218 | prompt eval time = 195.96 ms / 106 tokens ( 1.85 ms per token, 540.92 tokens per second) |
| 1.22.189.297 I slot print_timing: id 0 | task 1218 | eval time = 2950.13 ms / 54 tokens ( 54.63 ms per token, 18.30 tokens per second) |
| 1.22.189.297 I slot print_timing: id 0 | task 1218 | total time = 3146.09 ms / 160 tokens |
| 1.22.189.299 I slot print_timing: id 0 | task 1218 | graphs reused = 1197 |
| 1.22.189.333 I slot release: id 0 | task 1218 | stop processing: n_tokens = 159, truncated = 0 |
| 1.22.189.344 I srv update_slots: all slots are idle |
| 1.22.240.968 I srv params_from_: Chat format: peg-native |
| 1.22.241.198 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17548278089 |
| 1.22.241.201 I srv get_availabl: updating prompt cache |
| 1.22.241.232 W srv prompt_save: - saving prompt with length 159, total state size = 154.909 MiB (draft: 0.000 MiB) |
| 1.22.612.038 I srv load: - looking for better prompt, base f_keep = 0.075, sim = 0.094 |
| 1.22.612.053 I srv update: - cache state: 17 prompts, 5176.895 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.22.612.054 I srv update: - prompt 0x60f4303137d0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.22.612.055 I srv update: - prompt 0x60f42fe5fe40: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.22.612.057 I srv update: - prompt 0x60f42f622220: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.22.612.058 I srv update: - prompt 0x60f43046fe10: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.22.612.059 I srv update: - prompt 0x60f42f89f6b0: 138 tokens, checkpoints: 1, 303.837 MiB |
| 1.22.612.060 I srv update: - prompt 0x60f4328f6e30: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.22.612.060 I srv update: - prompt 0x60f43027cdc0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.22.612.061 I srv update: - prompt 0x60f42fe60130: 114 tokens, checkpoints: 1, 303.040 MiB |
| 1.22.612.062 I srv update: - prompt 0x60f42f89f760: 242 tokens, checkpoints: 1, 307.292 MiB |
| 1.22.612.062 I srv update: - prompt 0x60f432a46d90: 186 tokens, checkpoints: 1, 305.432 MiB |
| 1.22.612.063 I srv update: - prompt 0x60f4328f04b0: 225 tokens, checkpoints: 1, 306.728 MiB |
| 1.22.612.063 I srv update: - prompt 0x60f42feac760: 214 tokens, checkpoints: 1, 306.362 MiB |
| 1.22.612.064 I srv update: - prompt 0x60f4328f6a70: 180 tokens, checkpoints: 1, 305.233 MiB |
| 1.22.612.065 I srv update: - prompt 0x60f4303523b0: 288 tokens, checkpoints: 1, 308.821 MiB |
| 1.22.612.066 I srv update: - prompt 0x60f4306c9cf0: 203 tokens, checkpoints: 1, 305.997 MiB |
| 1.22.612.066 I srv update: - prompt 0x60f4301a2fe0: 288 tokens, checkpoints: 1, 308.821 MiB |
| 1.22.612.067 I srv update: - prompt 0x60f42f590b10: 159 tokens, checkpoints: 1, 304.535 MiB |
| 1.22.612.069 I srv get_availabl: prompt cache update took 370.87 ms |
| 1.22.612.186 I slot launch_slot_: id 0 | task 1274 | processing task, is_child = 0 |
| 1.22.612.200 W slot update_slots: id 0 | task 1274 | erased invalidated context checkpoint (pos_min = 101, pos_max = 101, n_tokens = 102, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.22.772.238 I slot create_check: id 0 | task 1274 | created context checkpoint 1 of 32 (pos_min = 122, pos_max = 122, n_tokens = 123, size = 149.626 MiB) |
| 1.28.079.496 I slot print_timing: id 0 | task 1274 | n_decoded = 100, tg = 19.03 t/s |
| 1.31.114.367 I slot print_timing: id 0 | task 1274 | n_decoded = 158, tg = 19.06 t/s |
| 1.33.588.807 I slot print_timing: id 0 | task 1274 | prompt eval time = 213.40 ms / 127 tokens ( 1.68 ms per token, 595.13 tokens per second) |
| 1.33.588.816 I slot print_timing: id 0 | task 1274 | eval time = 10763.20 ms / 204 tokens ( 52.76 ms per token, 18.95 tokens per second) |
| 1.33.588.817 I slot print_timing: id 0 | task 1274 | total time = 10976.60 ms / 331 tokens |
| 1.33.588.818 I slot print_timing: id 0 | task 1274 | graphs reused = 1398 |
| 1.33.588.853 I slot release: id 0 | task 1274 | stop processing: n_tokens = 330, truncated = 0 |
| 1.33.588.867 I srv update_slots: all slots are idle |
| 1.33.656.537 I srv operator(): operator(): cleaning up before exit... |
|
|
