| 0.00.370.500 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg) |
| 0.00.370.505 I device_info: |
| 0.00.662.709 I - CUDA0 : NVIDIA GeForce RTX 3080 (20052 MiB, 19814 MiB free) |
| 0.00.782.389 I - CUDA1 : NVIDIA GeForce RTX 3080 (20052 MiB, 19814 MiB free) |
| 0.00.908.017 I - CUDA2 : NVIDIA GeForce RTX 3080 (20052 MiB, 19814 MiB free) |
| 0.00.908.038 I - CPU : AMD EPYC 7F32 8-Core Processor (193092 MiB, 193092 MiB free) |
| 0.00.908.164 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CUDA : ARCHS = 860 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | |
| 0.00.908.286 I srv init: using 15 threads for HTTP server |
| 0.00.908.775 I srv start: binding port with default address family |
| 0.00.910.095 I srv llama_server: loading model |
| 0.00.910.105 I srv load_model: loading model '/mnt/models/PropellerA-models/qwen3.8-27b-Q6_K.gguf' |
| 0.00.910.197 I common_init_result: fitting params to device memory ... |
| 0.00.910.201 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on) |
| 0.06.785.259 W llama_context: n_ctx_seq (16384) < n_ctx_train (262144) -- the full capacity of the model will not be utilized |
| 0.06.880.957 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable) |
| 0.06.989.563 I srv load_model: initializing slots, n_slots = 1 |
| 0.07.035.248 W srv load_model: speculative decoding will use checkpoints |
| 0.07.035.265 W common_speculative_init: no implementations specified for speculative decoding |
| 0.07.035.267 I slot load_model: id 0 | task -1 | new slot, n_ctx = 16384 |
| 0.07.035.433 I srv load_model: prompt cache is enabled, size limit: 8192 MiB |
| 0.07.035.436 I srv load_model: use `--cache-ram 0` to disable the prompt cache |
| 0.07.035.437 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391 |
| 0.07.035.437 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256 |
| 0.07.035.465 I srv init: idle slots will be saved to prompt cache upon starting a new task |
| 0.07.055.856 I init: chat template, example_format: '<|im_start|>system |
| Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer. |
|
|
| You are a helpful assistant<|im_end|> |
| <|im_start|>user |
| Hello<|im_end|> |
| <|im_start|>assistant |
| <think> |
|
|
| </think> |
|
|
| Hi there<|im_end|> |
| <|im_start|>user |
| How are you?<|im_end|> |
| <|im_start|>assistant |
| <think> |
| ' |
| 0.07.071.432 I srv init: init: chat template, thinking = 1 |
| 0.07.071.480 I srv llama_server: model loaded |
| 0.07.071.485 I srv llama_server: server is listening on http://127.0.0.1:8291 |
| 0.07.071.492 I srv update_slots: all slots are idle |
| 0.08.126.574 I srv params_from_: Chat format: peg-native |
| 0.08.126.847 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = -1 |
| 0.08.126.852 I srv get_availabl: updating prompt cache |
| 0.08.126.863 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000 |
| 0.08.126.871 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 16384 tokens, 8589934592 est) |
| 0.08.126.872 I srv get_availabl: prompt cache update took 0.02 ms |
| 0.08.127.019 I slot launch_slot_: id 0 | task 0 | processing task, is_child = 0 |
| 0.08.307.164 I slot create_check: id 0 | task 0 | created context checkpoint 1 of 32 (pos_min = 62, pos_max = 62, n_tokens = 63, size = 149.626 MiB) |
| 0.08.856.605 I slot print_timing: id 0 | task 0 | prompt eval time = 234.43 ms / 67 tokens ( 3.50 ms per token, 285.80 tokens per second) |
| 0.08.856.611 I slot print_timing: id 0 | task 0 | eval time = 495.12 ms / 7 tokens ( 70.73 ms per token, 14.14 tokens per second) |
| 0.08.856.611 I slot print_timing: id 0 | task 0 | total time = 729.55 ms / 74 tokens |
| 0.08.856.617 I slot print_timing: id 0 | task 0 | graphs reused = 6 |
| 0.08.856.666 I slot release: id 0 | task 0 | stop processing: n_tokens = 73, truncated = 0 |
| 0.08.856.672 I srv update_slots: all slots are idle |
| 0.08.893.286 I srv params_from_: Chat format: peg-native |
| 0.08.893.538 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.238 (> 0.100 thold), f_keep = 0.205 |
| 0.08.893.541 I srv get_availabl: updating prompt cache |
| 0.08.893.573 W srv prompt_save: - saving prompt with length 73, total state size = 152.052 MiB (draft: 0.000 MiB) |
| 0.09.090.998 I srv load: - looking for better prompt, base f_keep = 0.205, sim = 0.238 |
| 0.09.091.009 I srv update: - cache state: 1 prompts, 301.678 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.09.091.010 I srv update: - prompt 0x5eec29761e40: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.09.091.012 I srv get_availabl: prompt cache update took 197.47 ms |
| 0.09.091.118 I slot launch_slot_: id 0 | task 9 | processing task, is_child = 0 |
| 0.09.091.131 W slot update_slots: id 0 | task 9 | erased invalidated context checkpoint (pos_min = 62, pos_max = 62, n_tokens = 63, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.09.216.395 I slot create_check: id 0 | task 9 | created context checkpoint 1 of 32 (pos_min = 58, pos_max = 58, n_tokens = 59, size = 149.626 MiB) |
| 0.09.788.613 I slot print_timing: id 0 | task 9 | prompt eval time = 177.51 ms / 63 tokens ( 2.82 ms per token, 354.91 tokens per second) |
| 0.09.788.619 I slot print_timing: id 0 | task 9 | eval time = 519.97 ms / 9 tokens ( 57.77 ms per token, 17.31 tokens per second) |
| 0.09.788.620 I slot print_timing: id 0 | task 9 | total time = 697.47 ms / 72 tokens |
| 0.09.788.621 I slot print_timing: id 0 | task 9 | graphs reused = 13 |
| 0.09.788.644 I slot release: id 0 | task 9 | stop processing: n_tokens = 71, truncated = 0 |
| 0.09.788.653 I srv update_slots: all slots are idle |
| 0.09.822.238 I srv params_from_: Chat format: peg-native |
| 0.09.822.484 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.227 (> 0.100 thold), f_keep = 0.211 |
| 0.09.822.487 I srv get_availabl: updating prompt cache |
| 0.09.822.515 W srv prompt_save: - saving prompt with length 71, total state size = 151.985 MiB (draft: 0.000 MiB) |
| 0.10.015.448 I srv load: - looking for better prompt, base f_keep = 0.211, sim = 0.227 |
| 0.10.015.461 I srv update: - cache state: 2 prompts, 603.289 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.10.015.462 I srv update: - prompt 0x5eec29761e40: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.10.015.463 I srv update: - prompt 0x5eec292ae2a0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.10.015.464 I srv get_availabl: prompt cache update took 192.98 ms |
| 0.10.015.567 I slot launch_slot_: id 0 | task 20 | processing task, is_child = 0 |
| 0.10.015.580 W slot update_slots: id 0 | task 20 | erased invalidated context checkpoint (pos_min = 58, pos_max = 58, n_tokens = 59, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.10.140.428 I slot create_check: id 0 | task 20 | created context checkpoint 1 of 32 (pos_min = 61, pos_max = 61, n_tokens = 62, size = 149.626 MiB) |
| 0.10.758.420 I slot print_timing: id 0 | task 20 | prompt eval time = 176.22 ms / 66 tokens ( 2.67 ms per token, 374.53 tokens per second) |
| 0.10.758.427 I slot print_timing: id 0 | task 20 | eval time = 566.61 ms / 10 tokens ( 56.66 ms per token, 17.65 tokens per second) |
| 0.10.758.428 I slot print_timing: id 0 | task 20 | total time = 742.83 ms / 76 tokens |
| 0.10.758.429 I slot print_timing: id 0 | task 20 | graphs reused = 21 |
| 0.10.758.456 I slot release: id 0 | task 20 | stop processing: n_tokens = 75, truncated = 0 |
| 0.10.758.465 I srv update_slots: all slots are idle |
| 0.10.791.667 I srv params_from_: Chat format: peg-native |
| 0.10.791.900 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17578849126 |
| 0.10.791.902 I srv get_availabl: updating prompt cache |
| 0.10.791.928 W srv prompt_save: - saving prompt with length 75, total state size = 152.118 MiB (draft: 0.000 MiB) |
| 0.10.985.721 I srv load: - looking for better prompt, base f_keep = 0.040, sim = 0.036 |
| 0.10.985.733 I srv update: - cache state: 3 prompts, 905.034 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.10.985.734 I srv update: - prompt 0x5eec29761e40: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.10.985.735 I srv update: - prompt 0x5eec292ae2a0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.10.985.735 I srv update: - prompt 0x5eec28a29f50: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.10.985.737 I srv get_availabl: prompt cache update took 193.83 ms |
| 0.10.985.838 I slot launch_slot_: id 0 | task 32 | processing task, is_child = 0 |
| 0.10.985.851 W slot update_slots: id 0 | task 32 | erased invalidated context checkpoint (pos_min = 61, pos_max = 61, n_tokens = 62, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.11.126.829 I slot create_check: id 0 | task 32 | created context checkpoint 1 of 32 (pos_min = 78, pos_max = 78, n_tokens = 79, size = 149.626 MiB) |
| 0.13.613.109 I slot print_timing: id 0 | task 32 | prompt eval time = 192.40 ms / 83 tokens ( 2.32 ms per token, 431.39 tokens per second) |
| 0.13.613.115 I slot print_timing: id 0 | task 32 | eval time = 2434.85 ms / 49 tokens ( 49.69 ms per token, 20.12 tokens per second) |
| 0.13.613.115 I slot print_timing: id 0 | task 32 | total time = 2627.25 ms / 132 tokens |
| 0.13.613.116 I slot print_timing: id 0 | task 32 | graphs reused = 68 |
| 0.13.613.143 I slot release: id 0 | task 32 | stop processing: n_tokens = 131, truncated = 0 |
| 0.13.613.153 I srv update_slots: all slots are idle |
| 0.13.639.459 I srv params_from_: Chat format: peg-native |
| 0.13.639.631 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17581703813 |
| 0.13.639.634 I srv get_availabl: updating prompt cache |
| 0.13.639.678 W srv prompt_save: - saving prompt with length 131, total state size = 153.979 MiB (draft: 0.000 MiB) |
| 0.13.837.864 I srv load: - looking for better prompt, base f_keep = 0.023, sim = 0.046 |
| 0.13.837.874 I srv update: - cache state: 4 prompts, 1208.638 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.13.837.875 I srv update: - prompt 0x5eec29761e40: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.13.837.876 I srv update: - prompt 0x5eec292ae2a0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.13.837.876 I srv update: - prompt 0x5eec28a29f50: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.13.837.877 I srv update: - prompt 0x5eec2be6aa20: 131 tokens, checkpoints: 1, 303.605 MiB |
| 0.13.837.878 I srv get_availabl: prompt cache update took 198.24 ms |
| 0.13.837.981 I slot launch_slot_: id 0 | task 83 | processing task, is_child = 0 |
| 0.13.837.993 W slot update_slots: id 0 | task 83 | erased invalidated context checkpoint (pos_min = 78, pos_max = 78, n_tokens = 79, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.13.962.032 I slot create_check: id 0 | task 83 | created context checkpoint 1 of 32 (pos_min = 60, pos_max = 60, n_tokens = 61, size = 149.626 MiB) |
| 0.14.600.180 I slot print_timing: id 0 | task 83 | prompt eval time = 175.58 ms / 65 tokens ( 2.70 ms per token, 370.20 tokens per second) |
| 0.14.600.187 I slot print_timing: id 0 | task 83 | eval time = 586.60 ms / 12 tokens ( 48.88 ms per token, 20.46 tokens per second) |
| 0.14.600.188 I slot print_timing: id 0 | task 83 | total time = 762.18 ms / 77 tokens |
| 0.14.600.189 I slot print_timing: id 0 | task 83 | graphs reused = 78 |
| 0.14.600.219 I slot release: id 0 | task 83 | stop processing: n_tokens = 76, truncated = 0 |
| 0.14.600.229 I srv update_slots: all slots are idle |
| 0.14.624.546 I srv params_from_: Chat format: peg-native |
| 0.14.624.711 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17582690889 |
| 0.14.624.713 I srv get_availabl: updating prompt cache |
| 0.14.624.741 W srv prompt_save: - saving prompt with length 76, total state size = 152.151 MiB (draft: 0.000 MiB) |
| 0.14.819.901 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.039 |
| 0.14.819.907 I srv load: - found better prompt with f_keep = 0.260, sim = 0.442 |
| 0.14.868.634 I srv update: - cache state: 4 prompts, 1206.811 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.14.868.640 I srv update: - prompt 0x5eec29761e40: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.14.868.641 I srv update: - prompt 0x5eec292ae2a0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.14.868.642 I srv update: - prompt 0x5eec28a29f50: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.14.868.642 I srv update: - prompt 0x5eec298be250: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.14.868.644 I srv get_availabl: prompt cache update took 243.93 ms |
| 0.14.868.755 I slot launch_slot_: id 0 | task 97 | processing task, is_child = 0 |
| 0.14.868.768 W slot update_slots: id 0 | task 97 | erased invalidated context checkpoint (pos_min = 78, pos_max = 78, n_tokens = 79, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.15.006.658 I slot create_check: id 0 | task 97 | created context checkpoint 1 of 32 (pos_min = 72, pos_max = 72, n_tokens = 73, size = 149.626 MiB) |
| 0.16.052.570 I slot print_timing: id 0 | task 97 | prompt eval time = 189.31 ms / 77 tokens ( 2.46 ms per token, 406.75 tokens per second) |
| 0.16.052.576 I slot print_timing: id 0 | task 97 | eval time = 994.49 ms / 24 tokens ( 41.44 ms per token, 24.13 tokens per second) |
| 0.16.052.576 I slot print_timing: id 0 | task 97 | total time = 1183.80 ms / 101 tokens |
| 0.16.052.577 I slot print_timing: id 0 | task 97 | graphs reused = 100 |
| 0.16.052.608 I slot release: id 0 | task 97 | stop processing: n_tokens = 100, truncated = 0 |
| 0.16.052.618 I srv update_slots: all slots are idle |
| 0.16.082.886 I srv params_from_: Chat format: peg-native |
| 0.16.083.091 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.393 (> 0.100 thold), f_keep = 0.350 |
| 0.16.083.095 I srv get_availabl: updating prompt cache |
| 0.16.083.139 W srv prompt_save: - saving prompt with length 100, total state size = 152.949 MiB (draft: 0.000 MiB) |
| 0.16.279.678 I srv load: - looking for better prompt, base f_keep = 0.350, sim = 0.393 |
| 0.16.279.692 I srv update: - cache state: 5 prompts, 1509.386 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.16.279.694 I srv update: - prompt 0x5eec29761e40: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.16.279.694 I srv update: - prompt 0x5eec292ae2a0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.16.279.695 I srv update: - prompt 0x5eec28a29f50: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.16.279.696 I srv update: - prompt 0x5eec298be250: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.16.279.696 I srv update: - prompt 0x5eec28c1ef60: 100 tokens, checkpoints: 1, 302.575 MiB |
| 0.16.279.698 I srv get_availabl: prompt cache update took 196.60 ms |
| 0.16.279.807 I slot launch_slot_: id 0 | task 123 | processing task, is_child = 0 |
| 0.16.279.820 W slot update_slots: id 0 | task 123 | erased invalidated context checkpoint (pos_min = 72, pos_max = 72, n_tokens = 73, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.16.571.284 I slot create_check: id 0 | task 123 | created context checkpoint 1 of 32 (pos_min = 84, pos_max = 84, n_tokens = 85, size = 149.626 MiB) |
| 0.18.716.297 I slot print_timing: id 0 | task 123 | prompt eval time = 342.94 ms / 89 tokens ( 3.85 ms per token, 259.52 tokens per second) |
| 0.18.716.302 I slot print_timing: id 0 | task 123 | eval time = 2093.53 ms / 50 tokens ( 41.87 ms per token, 23.88 tokens per second) |
| 0.18.716.303 I slot print_timing: id 0 | task 123 | total time = 2436.47 ms / 139 tokens |
| 0.18.716.304 I slot print_timing: id 0 | task 123 | graphs reused = 148 |
| 0.18.716.339 I slot release: id 0 | task 123 | stop processing: n_tokens = 138, truncated = 0 |
| 0.18.716.349 I srv update_slots: all slots are idle |
| 0.18.752.600 I srv params_from_: Chat format: peg-native |
| 0.18.752.838 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17586807009 |
| 0.18.752.841 I srv get_availabl: updating prompt cache |
| 0.18.752.875 W srv prompt_save: - saving prompt with length 138, total state size = 154.211 MiB (draft: 0.000 MiB) |
| 0.18.954.247 I srv load: - looking for better prompt, base f_keep = 0.022, sim = 0.050 |
| 0.18.954.263 I srv update: - cache state: 6 prompts, 1813.223 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.18.954.264 I srv update: - prompt 0x5eec29761e40: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.18.954.265 I srv update: - prompt 0x5eec292ae2a0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.18.954.265 I srv update: - prompt 0x5eec28a29f50: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.18.954.266 I srv update: - prompt 0x5eec298be250: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.18.954.267 I srv update: - prompt 0x5eec28c1ef60: 100 tokens, checkpoints: 1, 302.575 MiB |
| 0.18.954.267 I srv update: - prompt 0x5eec28c5de00: 138 tokens, checkpoints: 1, 303.837 MiB |
| 0.18.954.268 I srv get_availabl: prompt cache update took 201.43 ms |
| 0.18.954.380 I slot launch_slot_: id 0 | task 175 | processing task, is_child = 0 |
| 0.18.954.393 W slot update_slots: id 0 | task 175 | erased invalidated context checkpoint (pos_min = 84, pos_max = 84, n_tokens = 85, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.19.077.614 I slot create_check: id 0 | task 175 | created context checkpoint 1 of 32 (pos_min = 55, pos_max = 55, n_tokens = 56, size = 149.626 MiB) |
| 0.19.554.931 I slot print_timing: id 0 | task 175 | prompt eval time = 174.65 ms / 60 tokens ( 2.91 ms per token, 343.54 tokens per second) |
| 0.19.554.940 I slot print_timing: id 0 | task 175 | eval time = 425.88 ms / 9 tokens ( 47.32 ms per token, 21.13 tokens per second) |
| 0.19.554.941 I slot print_timing: id 0 | task 175 | total time = 600.53 ms / 69 tokens |
| 0.19.554.942 I slot print_timing: id 0 | task 175 | graphs reused = 155 |
| 0.19.554.977 I slot release: id 0 | task 175 | stop processing: n_tokens = 68, truncated = 0 |
| 0.19.554.993 I srv update_slots: all slots are idle |
| 0.19.588.503 I srv params_from_: Chat format: peg-native |
| 0.19.588.744 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.183 (> 0.100 thold), f_keep = 0.221 |
| 0.19.588.748 I srv get_availabl: updating prompt cache |
| 0.19.588.790 W srv prompt_save: - saving prompt with length 68, total state size = 151.886 MiB (draft: 0.000 MiB) |
| 0.19.782.428 I srv load: - looking for better prompt, base f_keep = 0.221, sim = 0.183 |
| 0.19.782.444 I srv update: - cache state: 7 prompts, 2114.735 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.19.782.446 I srv update: - prompt 0x5eec29761e40: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.19.782.446 I srv update: - prompt 0x5eec292ae2a0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.19.782.447 I srv update: - prompt 0x5eec28a29f50: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.19.782.448 I srv update: - prompt 0x5eec298be250: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.19.782.448 I srv update: - prompt 0x5eec28c1ef60: 100 tokens, checkpoints: 1, 302.575 MiB |
| 0.19.782.449 I srv update: - prompt 0x5eec28c5de00: 138 tokens, checkpoints: 1, 303.837 MiB |
| 0.19.782.449 I srv update: - prompt 0x5eec2be6d230: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.19.782.451 I srv get_availabl: prompt cache update took 193.70 ms |
| 0.19.782.552 I slot launch_slot_: id 0 | task 186 | processing task, is_child = 0 |
| 0.19.782.565 W slot update_slots: id 0 | task 186 | erased invalidated context checkpoint (pos_min = 55, pos_max = 55, n_tokens = 56, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.19.931.734 I slot create_check: id 0 | task 186 | created context checkpoint 1 of 32 (pos_min = 77, pos_max = 77, n_tokens = 78, size = 149.626 MiB) |
| 0.20.696.530 I slot print_timing: id 0 | task 186 | prompt eval time = 200.58 ms / 82 tokens ( 2.45 ms per token, 408.82 tokens per second) |
| 0.20.696.536 I slot print_timing: id 0 | task 186 | eval time = 713.38 ms / 16 tokens ( 44.59 ms per token, 22.43 tokens per second) |
| 0.20.696.537 I slot print_timing: id 0 | task 186 | total time = 913.96 ms / 98 tokens |
| 0.20.696.537 I slot print_timing: id 0 | task 186 | graphs reused = 169 |
| 0.20.696.570 I slot release: id 0 | task 186 | stop processing: n_tokens = 97, truncated = 0 |
| 0.20.696.584 I srv update_slots: all slots are idle |
| 0.20.723.059 I srv params_from_: Chat format: peg-native |
| 0.20.723.285 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17588787240 |
| 0.20.723.288 I srv get_availabl: updating prompt cache |
| 0.20.723.343 W srv prompt_save: - saving prompt with length 97, total state size = 152.849 MiB (draft: 0.000 MiB) |
| 0.20.930.627 I srv load: - looking for better prompt, base f_keep = 0.031, sim = 0.033 |
| 0.20.930.635 I srv load: - found better prompt with f_keep = 0.340, sim = 0.374 |
| 0.20.980.684 I srv update: - cache state: 7 prompts, 2114.635 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.20.980.690 I srv update: - prompt 0x5eec29761e40: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.20.980.691 I srv update: - prompt 0x5eec292ae2a0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.20.980.692 I srv update: - prompt 0x5eec28a29f50: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.20.980.692 I srv update: - prompt 0x5eec298be250: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.20.980.693 I srv update: - prompt 0x5eec28c5de00: 138 tokens, checkpoints: 1, 303.837 MiB |
| 0.20.980.693 I srv update: - prompt 0x5eec2be6d230: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.20.980.694 I srv update: - prompt 0x5eec2af70280: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.20.980.695 I srv get_availabl: prompt cache update took 257.41 ms |
| 0.20.980.807 I slot launch_slot_: id 0 | task 204 | processing task, is_child = 0 |
| 0.20.980.820 W slot update_slots: id 0 | task 204 | erased invalidated context checkpoint (pos_min = 72, pos_max = 72, n_tokens = 73, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.21.132.065 I slot create_check: id 0 | task 204 | created context checkpoint 1 of 32 (pos_min = 86, pos_max = 86, n_tokens = 87, size = 149.626 MiB) |
| 0.22.111.656 I slot print_timing: id 0 | task 204 | prompt eval time = 202.75 ms / 91 tokens ( 2.23 ms per token, 448.82 tokens per second) |
| 0.22.111.663 I slot print_timing: id 0 | task 204 | eval time = 928.08 ms / 23 tokens ( 40.35 ms per token, 24.78 tokens per second) |
| 0.22.111.664 I slot print_timing: id 0 | task 204 | total time = 1130.83 ms / 114 tokens |
| 0.22.111.665 I slot print_timing: id 0 | task 204 | graphs reused = 190 |
| 0.22.111.693 I slot release: id 0 | task 204 | stop processing: n_tokens = 113, truncated = 0 |
| 0.22.111.704 I srv update_slots: all slots are idle |
| 0.22.147.437 I srv params_from_: Chat format: peg-native |
| 0.22.147.666 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17590202363 |
| 0.22.147.668 I srv get_availabl: updating prompt cache |
| 0.22.147.696 W srv prompt_save: - saving prompt with length 113, total state size = 153.381 MiB (draft: 0.000 MiB) |
| 0.22.343.715 I srv load: - looking for better prompt, base f_keep = 0.027, sim = 0.024 |
| 0.22.343.729 I srv update: - cache state: 8 prompts, 2417.642 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.22.343.730 I srv update: - prompt 0x5eec29761e40: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.22.343.731 I srv update: - prompt 0x5eec292ae2a0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.22.343.731 I srv update: - prompt 0x5eec28a29f50: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.22.343.732 I srv update: - prompt 0x5eec298be250: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.22.343.733 I srv update: - prompt 0x5eec28c5de00: 138 tokens, checkpoints: 1, 303.837 MiB |
| 0.22.343.733 I srv update: - prompt 0x5eec2be6d230: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.22.343.734 I srv update: - prompt 0x5eec2af70280: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.22.343.734 I srv update: - prompt 0x5eec2953e190: 113 tokens, checkpoints: 1, 303.007 MiB |
| 0.22.343.736 I srv get_availabl: prompt cache update took 196.07 ms |
| 0.22.343.838 I slot launch_slot_: id 0 | task 229 | processing task, is_child = 0 |
| 0.22.343.848 W slot update_slots: id 0 | task 229 | erased invalidated context checkpoint (pos_min = 86, pos_max = 86, n_tokens = 87, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.22.651.838 I slot create_check: id 0 | task 229 | created context checkpoint 1 of 32 (pos_min = 119, pos_max = 119, n_tokens = 120, size = 149.626 MiB) |
| 0.26.963.729 I slot print_timing: id 0 | task 229 | n_decoded = 100, tg = 23.48 t/s |
| 0.28.897.555 I slot print_timing: id 0 | task 229 | prompt eval time = 360.65 ms / 124 tokens ( 2.91 ms per token, 343.82 tokens per second) |
| 0.28.897.561 I slot print_timing: id 0 | task 229 | eval time = 6193.05 ms / 140 tokens ( 44.24 ms per token, 22.61 tokens per second) |
| 0.28.897.561 I slot print_timing: id 0 | task 229 | total time = 6553.70 ms / 264 tokens |
| 0.28.897.562 I slot print_timing: id 0 | task 229 | graphs reused = 327 |
| 0.28.897.596 I slot release: id 0 | task 229 | stop processing: n_tokens = 263, truncated = 0 |
| 0.28.897.610 I srv update_slots: all slots are idle |
| 0.28.949.976 I srv params_from_: Chat format: peg-native |
| 0.28.950.212 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.129 (> 0.100 thold), f_keep = 0.046 |
| 0.28.950.216 I srv get_availabl: updating prompt cache |
| 0.28.950.274 W srv prompt_save: - saving prompt with length 263, total state size = 158.364 MiB (draft: 0.000 MiB) |
| 0.29.150.838 I srv load: - looking for better prompt, base f_keep = 0.046, sim = 0.129 |
| 0.29.150.850 I srv update: - cache state: 9 prompts, 2725.632 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.29.150.851 I srv update: - prompt 0x5eec29761e40: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.29.150.852 I srv update: - prompt 0x5eec292ae2a0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.29.150.852 I srv update: - prompt 0x5eec28a29f50: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.29.150.853 I srv update: - prompt 0x5eec298be250: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.29.150.854 I srv update: - prompt 0x5eec28c5de00: 138 tokens, checkpoints: 1, 303.837 MiB |
| 0.29.150.854 I srv update: - prompt 0x5eec2be6d230: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.29.150.855 I srv update: - prompt 0x5eec2af70280: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.29.150.855 I srv update: - prompt 0x5eec2953e190: 113 tokens, checkpoints: 1, 303.007 MiB |
| 0.29.150.856 I srv update: - prompt 0x5eec295fa0c0: 263 tokens, checkpoints: 1, 307.990 MiB |
| 0.29.150.858 I srv get_availabl: prompt cache update took 200.64 ms |
| 0.29.150.963 I slot launch_slot_: id 0 | task 371 | processing task, is_child = 0 |
| 0.29.150.973 W slot update_slots: id 0 | task 371 | erased invalidated context checkpoint (pos_min = 119, pos_max = 119, n_tokens = 120, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.29.301.693 I slot create_check: id 0 | task 371 | created context checkpoint 1 of 32 (pos_min = 88, pos_max = 88, n_tokens = 89, size = 149.626 MiB) |
| 0.33.180.349 I slot print_timing: id 0 | task 371 | prompt eval time = 202.20 ms / 93 tokens ( 2.17 ms per token, 459.94 tokens per second) |
| 0.33.180.355 I slot print_timing: id 0 | task 371 | eval time = 3827.16 ms / 94 tokens ( 40.71 ms per token, 24.56 tokens per second) |
| 0.33.180.356 I slot print_timing: id 0 | task 371 | total time = 4029.36 ms / 187 tokens |
| 0.33.180.356 I slot print_timing: id 0 | task 371 | graphs reused = 419 |
| 0.33.180.386 I slot release: id 0 | task 371 | stop processing: n_tokens = 186, truncated = 0 |
| 0.33.180.410 I srv update_slots: all slots are idle |
| 0.33.230.635 I srv params_from_: Chat format: peg-native |
| 0.33.230.863 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.138 (> 0.100 thold), f_keep = 0.065 |
| 0.33.230.866 I srv get_availabl: updating prompt cache |
| 0.33.230.903 W srv prompt_save: - saving prompt with length 186, total state size = 155.806 MiB (draft: 0.000 MiB) |
| 0.33.672.045 I srv load: - looking for better prompt, base f_keep = 0.065, sim = 0.138 |
| 0.33.672.057 I srv update: - cache state: 10 prompts, 3031.064 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.33.672.060 I srv update: - prompt 0x5eec29761e40: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.33.672.061 I srv update: - prompt 0x5eec292ae2a0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.33.672.061 I srv update: - prompt 0x5eec28a29f50: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.33.672.062 I srv update: - prompt 0x5eec298be250: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.33.672.062 I srv update: - prompt 0x5eec28c5de00: 138 tokens, checkpoints: 1, 303.837 MiB |
| 0.33.672.063 I srv update: - prompt 0x5eec2be6d230: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.33.672.064 I srv update: - prompt 0x5eec2af70280: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.33.672.064 I srv update: - prompt 0x5eec2953e190: 113 tokens, checkpoints: 1, 303.007 MiB |
| 0.33.672.065 I srv update: - prompt 0x5eec295fa0c0: 263 tokens, checkpoints: 1, 307.990 MiB |
| 0.33.672.065 I srv update: - prompt 0x5eec29a29190: 186 tokens, checkpoints: 1, 305.432 MiB |
| 0.33.672.066 I srv get_availabl: prompt cache update took 441.20 ms |
| 0.33.672.165 I slot launch_slot_: id 0 | task 467 | processing task, is_child = 0 |
| 0.33.672.179 W slot update_slots: id 0 | task 467 | erased invalidated context checkpoint (pos_min = 88, pos_max = 88, n_tokens = 89, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.33.820.761 I slot create_check: id 0 | task 467 | created context checkpoint 1 of 32 (pos_min = 82, pos_max = 82, n_tokens = 83, size = 149.626 MiB) |
| 0.37.870.758 I slot print_timing: id 0 | task 467 | n_decoded = 100, tg = 25.01 t/s |
| 0.39.506.677 I slot print_timing: id 0 | task 467 | prompt eval time = 200.04 ms / 87 tokens ( 2.30 ms per token, 434.92 tokens per second) |
| 0.39.506.683 I slot print_timing: id 0 | task 467 | eval time = 5634.45 ms / 139 tokens ( 40.54 ms per token, 24.67 tokens per second) |
| 0.39.506.683 I slot print_timing: id 0 | task 467 | total time = 5834.49 ms / 226 tokens |
| 0.39.506.684 I slot print_timing: id 0 | task 467 | graphs reused = 556 |
| 0.39.506.710 I slot release: id 0 | task 467 | stop processing: n_tokens = 225, truncated = 0 |
| 0.39.506.722 I srv update_slots: all slots are idle |
| 0.39.558.649 I srv params_from_: Chat format: peg-native |
| 0.39.558.880 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.053 |
| 0.39.558.883 I srv get_availabl: updating prompt cache |
| 0.39.558.915 W srv prompt_save: - saving prompt with length 225, total state size = 157.102 MiB (draft: 0.000 MiB) |
| 0.39.759.090 I srv load: - looking for better prompt, base f_keep = 0.053, sim = 0.107 |
| 0.39.759.102 I srv update: - cache state: 11 prompts, 3337.792 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.39.759.103 I srv update: - prompt 0x5eec29761e40: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.39.759.104 I srv update: - prompt 0x5eec292ae2a0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.39.759.104 I srv update: - prompt 0x5eec28a29f50: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.39.759.105 I srv update: - prompt 0x5eec298be250: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.39.759.105 I srv update: - prompt 0x5eec28c5de00: 138 tokens, checkpoints: 1, 303.837 MiB |
| 0.39.759.106 I srv update: - prompt 0x5eec2be6d230: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.39.759.106 I srv update: - prompt 0x5eec2af70280: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.39.759.107 I srv update: - prompt 0x5eec2953e190: 113 tokens, checkpoints: 1, 303.007 MiB |
| 0.39.759.108 I srv update: - prompt 0x5eec295fa0c0: 263 tokens, checkpoints: 1, 307.990 MiB |
| 0.39.759.109 I srv update: - prompt 0x5eec29a29190: 186 tokens, checkpoints: 1, 305.432 MiB |
| 0.39.759.110 I srv update: - prompt 0x5eec28a26500: 225 tokens, checkpoints: 1, 306.728 MiB |
| 0.39.759.116 I srv get_availabl: prompt cache update took 200.23 ms |
| 0.39.759.215 I slot launch_slot_: id 0 | task 608 | processing task, is_child = 0 |
| 0.39.759.226 W slot update_slots: id 0 | task 608 | erased invalidated context checkpoint (pos_min = 82, pos_max = 82, n_tokens = 83, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.39.924.360 I slot create_check: id 0 | task 608 | created context checkpoint 1 of 32 (pos_min = 107, pos_max = 107, n_tokens = 108, size = 149.626 MiB) |
| 0.44.222.228 I slot print_timing: id 0 | task 608 | n_decoded = 100, tg = 23.55 t/s |
| 0.44.330.043 I slot print_timing: id 0 | task 608 | prompt eval time = 216.55 ms / 112 tokens ( 1.93 ms per token, 517.20 tokens per second) |
| 0.44.330.048 I slot print_timing: id 0 | task 608 | eval time = 4354.26 ms / 103 tokens ( 42.27 ms per token, 23.65 tokens per second) |
| 0.44.330.049 I slot print_timing: id 0 | task 608 | total time = 4570.82 ms / 215 tokens |
| 0.44.330.050 I slot print_timing: id 0 | task 608 | graphs reused = 657 |
| 0.44.330.078 I slot release: id 0 | task 608 | stop processing: n_tokens = 214, truncated = 0 |
| 0.44.330.089 I srv update_slots: all slots are idle |
| 0.44.372.674 I srv params_from_: Chat format: peg-native |
| 0.44.372.871 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.136 (> 0.100 thold), f_keep = 0.056 |
| 0.44.372.875 I srv get_availabl: updating prompt cache |
| 0.44.372.919 W srv prompt_save: - saving prompt with length 214, total state size = 156.736 MiB (draft: 0.000 MiB) |
| 0.44.566.099 I srv load: - looking for better prompt, base f_keep = 0.056, sim = 0.136 |
| 0.44.566.113 I srv update: - cache state: 12 prompts, 3644.154 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.44.566.114 I srv update: - prompt 0x5eec29761e40: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.44.566.115 I srv update: - prompt 0x5eec292ae2a0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.44.566.115 I srv update: - prompt 0x5eec28a29f50: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.44.566.116 I srv update: - prompt 0x5eec298be250: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.44.566.117 I srv update: - prompt 0x5eec28c5de00: 138 tokens, checkpoints: 1, 303.837 MiB |
| 0.44.566.117 I srv update: - prompt 0x5eec2be6d230: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.44.566.118 I srv update: - prompt 0x5eec2af70280: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.44.566.119 I srv update: - prompt 0x5eec2953e190: 113 tokens, checkpoints: 1, 303.007 MiB |
| 0.44.566.119 I srv update: - prompt 0x5eec295fa0c0: 263 tokens, checkpoints: 1, 307.990 MiB |
| 0.44.566.120 I srv update: - prompt 0x5eec29a29190: 186 tokens, checkpoints: 1, 305.432 MiB |
| 0.44.566.120 I srv update: - prompt 0x5eec28a26500: 225 tokens, checkpoints: 1, 306.728 MiB |
| 0.44.566.121 I srv update: - prompt 0x5eec2c49e560: 214 tokens, checkpoints: 1, 306.362 MiB |
| 0.44.566.122 I srv get_availabl: prompt cache update took 193.25 ms |
| 0.44.566.221 I slot launch_slot_: id 0 | task 713 | processing task, is_child = 0 |
| 0.44.566.230 W slot update_slots: id 0 | task 713 | erased invalidated context checkpoint (pos_min = 107, pos_max = 107, n_tokens = 108, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.44.715.414 I slot create_check: id 0 | task 713 | created context checkpoint 1 of 32 (pos_min = 83, pos_max = 83, n_tokens = 84, size = 149.626 MiB) |
| 0.48.545.612 I slot print_timing: id 0 | task 713 | prompt eval time = 200.66 ms / 88 tokens ( 2.28 ms per token, 438.56 tokens per second) |
| 0.48.545.618 I slot print_timing: id 0 | task 713 | eval time = 3778.72 ms / 93 tokens ( 40.63 ms per token, 24.61 tokens per second) |
| 0.48.545.618 I slot print_timing: id 0 | task 713 | total time = 3979.38 ms / 181 tokens |
| 0.48.545.619 I slot print_timing: id 0 | task 713 | graphs reused = 748 |
| 0.48.545.644 I slot release: id 0 | task 713 | stop processing: n_tokens = 180, truncated = 0 |
| 0.48.545.655 I srv update_slots: all slots are idle |
| 0.48.598.404 I srv params_from_: Chat format: peg-native |
| 0.48.598.636 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.114 (> 0.100 thold), f_keep = 0.067 |
| 0.48.598.640 I srv get_availabl: updating prompt cache |
| 0.48.598.670 W srv prompt_save: - saving prompt with length 180, total state size = 155.607 MiB (draft: 0.000 MiB) |
| 0.48.794.061 I srv load: - looking for better prompt, base f_keep = 0.067, sim = 0.114 |
| 0.48.794.076 I srv update: - cache state: 13 prompts, 3949.387 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.48.794.077 I srv update: - prompt 0x5eec29761e40: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.48.794.078 I srv update: - prompt 0x5eec292ae2a0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.48.794.078 I srv update: - prompt 0x5eec28a29f50: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.48.794.079 I srv update: - prompt 0x5eec298be250: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.48.794.080 I srv update: - prompt 0x5eec28c5de00: 138 tokens, checkpoints: 1, 303.837 MiB |
| 0.48.794.080 I srv update: - prompt 0x5eec2be6d230: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.48.794.081 I srv update: - prompt 0x5eec2af70280: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.48.794.082 I srv update: - prompt 0x5eec2953e190: 113 tokens, checkpoints: 1, 303.007 MiB |
| 0.48.794.082 I srv update: - prompt 0x5eec295fa0c0: 263 tokens, checkpoints: 1, 307.990 MiB |
| 0.48.794.083 I srv update: - prompt 0x5eec29a29190: 186 tokens, checkpoints: 1, 305.432 MiB |
| 0.48.794.083 I srv update: - prompt 0x5eec28a26500: 225 tokens, checkpoints: 1, 306.728 MiB |
| 0.48.794.084 I srv update: - prompt 0x5eec2c49e560: 214 tokens, checkpoints: 1, 306.362 MiB |
| 0.48.794.084 I srv update: - prompt 0x5eec28f61f10: 180 tokens, checkpoints: 1, 305.233 MiB |
| 0.48.794.085 I srv get_availabl: prompt cache update took 195.44 ms |
| 0.48.794.183 I slot launch_slot_: id 0 | task 808 | processing task, is_child = 0 |
| 0.48.794.192 W slot update_slots: id 0 | task 808 | erased invalidated context checkpoint (pos_min = 83, pos_max = 83, n_tokens = 84, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.48.955.909 I slot create_check: id 0 | task 808 | created context checkpoint 1 of 32 (pos_min = 100, pos_max = 100, n_tokens = 101, size = 149.626 MiB) |
| 0.53.792.447 I slot print_timing: id 0 | task 808 | n_decoded = 100, tg = 20.90 t/s |
| 0.56.815.549 I slot print_timing: id 0 | task 808 | n_decoded = 170, tg = 21.77 t/s |
| 0.57.317.288 I slot print_timing: id 0 | task 808 | prompt eval time = 213.23 ms / 105 tokens ( 2.03 ms per token, 492.43 tokens per second) |
| 0.57.317.292 I slot print_timing: id 0 | task 808 | eval time = 8309.86 ms / 184 tokens ( 45.16 ms per token, 22.14 tokens per second) |
| 0.57.317.293 I slot print_timing: id 0 | task 808 | total time = 8523.09 ms / 289 tokens |
| 0.57.317.294 I slot print_timing: id 0 | task 808 | graphs reused = 929 |
| 0.57.317.322 I slot release: id 0 | task 808 | stop processing: n_tokens = 288, truncated = 0 |
| 0.57.317.340 I srv update_slots: all slots are idle |
| 0.57.368.563 I srv params_from_: Chat format: peg-native |
| 0.57.368.797 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.133 (> 0.100 thold), f_keep = 0.042 |
| 0.57.368.801 I srv get_availabl: updating prompt cache |
| 0.57.368.837 W srv prompt_save: - saving prompt with length 288, total state size = 159.194 MiB (draft: 0.000 MiB) |
| 0.57.826.960 I srv load: - looking for better prompt, base f_keep = 0.042, sim = 0.133 |
| 0.57.826.978 I srv update: - cache state: 14 prompts, 4258.207 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.57.826.979 I srv update: - prompt 0x5eec29761e40: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.57.826.980 I srv update: - prompt 0x5eec292ae2a0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.57.826.981 I srv update: - prompt 0x5eec28a29f50: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.57.826.982 I srv update: - prompt 0x5eec298be250: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.57.826.982 I srv update: - prompt 0x5eec28c5de00: 138 tokens, checkpoints: 1, 303.837 MiB |
| 0.57.826.983 I srv update: - prompt 0x5eec2be6d230: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.57.826.983 I srv update: - prompt 0x5eec2af70280: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.57.826.984 I srv update: - prompt 0x5eec2953e190: 113 tokens, checkpoints: 1, 303.007 MiB |
| 0.57.826.985 I srv update: - prompt 0x5eec295fa0c0: 263 tokens, checkpoints: 1, 307.990 MiB |
| 0.57.826.985 I srv update: - prompt 0x5eec29a29190: 186 tokens, checkpoints: 1, 305.432 MiB |
| 0.57.826.986 I srv update: - prompt 0x5eec28a26500: 225 tokens, checkpoints: 1, 306.728 MiB |
| 0.57.826.987 I srv update: - prompt 0x5eec2c49e560: 214 tokens, checkpoints: 1, 306.362 MiB |
| 0.57.826.987 I srv update: - prompt 0x5eec28f61f10: 180 tokens, checkpoints: 1, 305.233 MiB |
| 0.57.826.988 I srv update: - prompt 0x5eec29a26d40: 288 tokens, checkpoints: 1, 308.821 MiB |
| 0.57.826.989 I srv get_availabl: prompt cache update took 458.19 ms |
| 0.57.827.091 I slot launch_slot_: id 0 | task 994 | processing task, is_child = 0 |
| 0.57.827.101 W slot update_slots: id 0 | task 994 | erased invalidated context checkpoint (pos_min = 100, pos_max = 100, n_tokens = 101, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.57.976.790 I slot create_check: id 0 | task 994 | created context checkpoint 1 of 32 (pos_min = 85, pos_max = 85, n_tokens = 86, size = 149.626 MiB) |
| 1.02.257.651 I slot print_timing: id 0 | task 994 | n_decoded = 100, tg = 23.64 t/s |
| 1.02.758.853 I slot print_timing: id 0 | task 994 | prompt eval time = 201.23 ms / 90 tokens ( 2.24 ms per token, 447.26 tokens per second) |
| 1.02.758.858 I slot print_timing: id 0 | task 994 | eval time = 4730.52 ms / 114 tokens ( 41.50 ms per token, 24.10 tokens per second) |
| 1.02.758.859 I slot print_timing: id 0 | task 994 | total time = 4931.75 ms / 204 tokens |
| 1.02.758.860 I slot print_timing: id 0 | task 994 | graphs reused = 1041 |
| 1.02.758.885 I slot release: id 0 | task 994 | stop processing: n_tokens = 203, truncated = 0 |
| 1.02.758.896 I srv update_slots: all slots are idle |
| 1.02.804.537 I srv params_from_: Chat format: peg-native |
| 1.02.804.707 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17630849556 |
| 1.02.804.710 I srv get_availabl: updating prompt cache |
| 1.02.804.748 W srv prompt_save: - saving prompt with length 203, total state size = 156.371 MiB (draft: 0.000 MiB) |
| 1.03.003.785 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.060 |
| 1.03.003.803 I srv update: - cache state: 15 prompts, 4564.204 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.03.003.804 I srv update: - prompt 0x5eec29761e40: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.03.003.805 I srv update: - prompt 0x5eec292ae2a0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.03.003.805 I srv update: - prompt 0x5eec28a29f50: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.03.003.806 I srv update: - prompt 0x5eec298be250: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.03.003.806 I srv update: - prompt 0x5eec28c5de00: 138 tokens, checkpoints: 1, 303.837 MiB |
| 1.03.003.807 I srv update: - prompt 0x5eec2be6d230: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.03.003.807 I srv update: - prompt 0x5eec2af70280: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.03.003.808 I srv update: - prompt 0x5eec2953e190: 113 tokens, checkpoints: 1, 303.007 MiB |
| 1.03.003.809 I srv update: - prompt 0x5eec295fa0c0: 263 tokens, checkpoints: 1, 307.990 MiB |
| 1.03.003.809 I srv update: - prompt 0x5eec29a29190: 186 tokens, checkpoints: 1, 305.432 MiB |
| 1.03.003.810 I srv update: - prompt 0x5eec28a26500: 225 tokens, checkpoints: 1, 306.728 MiB |
| 1.03.003.810 I srv update: - prompt 0x5eec2c49e560: 214 tokens, checkpoints: 1, 306.362 MiB |
| 1.03.003.811 I srv update: - prompt 0x5eec28f61f10: 180 tokens, checkpoints: 1, 305.233 MiB |
| 1.03.003.811 I srv update: - prompt 0x5eec29a26d40: 288 tokens, checkpoints: 1, 308.821 MiB |
| 1.03.003.812 I srv update: - prompt 0x5eec29abb6c0: 203 tokens, checkpoints: 1, 305.997 MiB |
| 1.03.003.813 I srv get_availabl: prompt cache update took 199.10 ms |
| 1.03.003.917 I slot launch_slot_: id 0 | task 1110 | processing task, is_child = 0 |
| 1.03.003.927 W slot update_slots: id 0 | task 1110 | erased invalidated context checkpoint (pos_min = 85, pos_max = 85, n_tokens = 86, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.03.220.730 I slot create_check: id 0 | task 1110 | created context checkpoint 1 of 32 (pos_min = 128, pos_max = 128, n_tokens = 129, size = 149.626 MiB) |
| 1.07.987.942 I slot print_timing: id 0 | task 1110 | n_decoded = 100, tg = 21.21 t/s |
| 1.11.009.700 I slot print_timing: id 0 | task 1110 | n_decoded = 170, tg = 21.97 t/s |
| 1.13.992.799 I slot print_timing: id 0 | task 1110 | prompt eval time = 268.31 ms / 133 tokens ( 2.02 ms per token, 495.70 tokens per second) |
| 1.13.992.805 I slot print_timing: id 0 | task 1110 | eval time = 10720.56 ms / 240 tokens ( 44.67 ms per token, 22.39 tokens per second) |
| 1.13.992.806 I slot print_timing: id 0 | task 1110 | total time = 10988.87 ms / 373 tokens |
| 1.13.992.807 I slot print_timing: id 0 | task 1110 | graphs reused = 1278 |
| 1.13.992.841 I slot release: id 0 | task 1110 | stop processing: n_tokens = 372, truncated = 0 |
| 1.13.992.855 I srv update_slots: all slots are idle |
| 1.14.034.963 I srv params_from_: Chat format: peg-native |
| 1.14.035.131 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17642083512 |
| 1.14.035.134 I srv get_availabl: updating prompt cache |
| 1.14.035.176 W srv prompt_save: - saving prompt with length 372, total state size = 161.985 MiB (draft: 0.000 MiB) |
| 1.14.223.991 I srv load: - looking for better prompt, base f_keep = 0.022, sim = 0.075 |
| 1.14.224.007 I srv update: - cache state: 16 prompts, 4875.815 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.14.224.008 I srv update: - prompt 0x5eec29761e40: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.14.224.009 I srv update: - prompt 0x5eec292ae2a0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.14.224.009 I srv update: - prompt 0x5eec28a29f50: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.14.224.010 I srv update: - prompt 0x5eec298be250: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.14.224.010 I srv update: - prompt 0x5eec28c5de00: 138 tokens, checkpoints: 1, 303.837 MiB |
| 1.14.224.011 I srv update: - prompt 0x5eec2be6d230: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.14.224.012 I srv update: - prompt 0x5eec2af70280: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.14.224.012 I srv update: - prompt 0x5eec2953e190: 113 tokens, checkpoints: 1, 303.007 MiB |
| 1.14.224.013 I srv update: - prompt 0x5eec295fa0c0: 263 tokens, checkpoints: 1, 307.990 MiB |
| 1.14.224.013 I srv update: - prompt 0x5eec29a29190: 186 tokens, checkpoints: 1, 305.432 MiB |
| 1.14.224.014 I srv update: - prompt 0x5eec28a26500: 225 tokens, checkpoints: 1, 306.728 MiB |
| 1.14.224.015 I srv update: - prompt 0x5eec2c49e560: 214 tokens, checkpoints: 1, 306.362 MiB |
| 1.14.224.016 I srv update: - prompt 0x5eec28f61f10: 180 tokens, checkpoints: 1, 305.233 MiB |
| 1.14.224.019 I srv update: - prompt 0x5eec29a26d40: 288 tokens, checkpoints: 1, 308.821 MiB |
| 1.14.224.020 I srv update: - prompt 0x5eec29abb6c0: 203 tokens, checkpoints: 1, 305.997 MiB |
| 1.14.224.021 I srv update: - prompt 0x5eec289e49c0: 372 tokens, checkpoints: 1, 311.611 MiB |
| 1.14.224.022 I srv get_availabl: prompt cache update took 188.89 ms |
| 1.14.224.134 I slot launch_slot_: id 0 | task 1352 | processing task, is_child = 0 |
| 1.14.224.146 W slot update_slots: id 0 | task 1352 | erased invalidated context checkpoint (pos_min = 128, pos_max = 128, n_tokens = 129, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.14.620.133 I slot create_check: id 0 | task 1352 | created context checkpoint 1 of 32 (pos_min = 101, pos_max = 101, n_tokens = 102, size = 149.626 MiB) |
| 1.16.903.936 I slot print_timing: id 0 | task 1352 | prompt eval time = 447.84 ms / 106 tokens ( 4.22 ms per token, 236.69 tokens per second) |
| 1.16.903.941 I slot print_timing: id 0 | task 1352 | eval time = 2231.95 ms / 54 tokens ( 41.33 ms per token, 24.19 tokens per second) |
| 1.16.903.942 I slot print_timing: id 0 | task 1352 | total time = 2679.78 ms / 160 tokens |
| 1.16.903.943 I slot print_timing: id 0 | task 1352 | graphs reused = 1330 |
| 1.16.903.974 I slot release: id 0 | task 1352 | stop processing: n_tokens = 159, truncated = 0 |
| 1.16.903.986 I srv update_slots: all slots are idle |
| 1.16.947.893 I srv params_from_: Chat format: peg-native |
| 1.16.948.090 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17644994644 |
| 1.16.948.094 I srv get_availabl: updating prompt cache |
| 1.16.948.133 W srv prompt_save: - saving prompt with length 159, total state size = 154.909 MiB (draft: 0.000 MiB) |
| 1.17.142.930 I srv load: - looking for better prompt, base f_keep = 0.075, sim = 0.094 |
| 1.17.142.947 I srv update: - cache state: 17 prompts, 5180.350 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.17.142.948 I srv update: - prompt 0x5eec29761e40: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.17.142.949 I srv update: - prompt 0x5eec292ae2a0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.17.142.950 I srv update: - prompt 0x5eec28a29f50: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.17.142.950 I srv update: - prompt 0x5eec298be250: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.17.142.951 I srv update: - prompt 0x5eec28c5de00: 138 tokens, checkpoints: 1, 303.837 MiB |
| 1.17.142.952 I srv update: - prompt 0x5eec2be6d230: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.17.142.952 I srv update: - prompt 0x5eec2af70280: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.17.142.953 I srv update: - prompt 0x5eec2953e190: 113 tokens, checkpoints: 1, 303.007 MiB |
| 1.17.142.954 I srv update: - prompt 0x5eec295fa0c0: 263 tokens, checkpoints: 1, 307.990 MiB |
| 1.17.142.956 I srv update: - prompt 0x5eec29a29190: 186 tokens, checkpoints: 1, 305.432 MiB |
| 1.17.142.958 I srv update: - prompt 0x5eec28a26500: 225 tokens, checkpoints: 1, 306.728 MiB |
| 1.17.142.958 I srv update: - prompt 0x5eec2c49e560: 214 tokens, checkpoints: 1, 306.362 MiB |
| 1.17.142.959 I srv update: - prompt 0x5eec28f61f10: 180 tokens, checkpoints: 1, 305.233 MiB |
| 1.17.142.959 I srv update: - prompt 0x5eec29a26d40: 288 tokens, checkpoints: 1, 308.821 MiB |
| 1.17.142.960 I srv update: - prompt 0x5eec29abb6c0: 203 tokens, checkpoints: 1, 305.997 MiB |
| 1.17.142.960 I srv update: - prompt 0x5eec289e49c0: 372 tokens, checkpoints: 1, 311.611 MiB |
| 1.17.142.961 I srv update: - prompt 0x5eec28a68b20: 159 tokens, checkpoints: 1, 304.535 MiB |
| 1.17.142.962 I srv get_availabl: prompt cache update took 194.87 ms |
| 1.17.143.057 I slot launch_slot_: id 0 | task 1408 | processing task, is_child = 0 |
| 1.17.143.067 W slot update_slots: id 0 | task 1408 | erased invalidated context checkpoint (pos_min = 101, pos_max = 101, n_tokens = 102, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.17.320.236 I slot create_check: id 0 | task 1408 | created context checkpoint 1 of 32 (pos_min = 122, pos_max = 122, n_tokens = 123, size = 149.626 MiB) |
| 1.21.724.665 I slot print_timing: id 0 | task 1408 | n_decoded = 100, tg = 23.91 t/s |
| 1.24.758.060 I slot print_timing: id 0 | task 1408 | n_decoded = 171, tg = 23.70 t/s |
| 1.25.826.639 I slot print_timing: id 0 | task 1408 | prompt eval time = 399.19 ms / 127 tokens ( 3.14 ms per token, 318.15 tokens per second) |
| 1.25.826.644 I slot print_timing: id 0 | task 1408 | eval time = 8284.38 ms / 197 tokens ( 42.05 ms per token, 23.78 tokens per second) |
| 1.25.826.644 I slot print_timing: id 0 | task 1408 | total time = 8683.57 ms / 324 tokens |
| 1.25.826.645 I slot print_timing: id 0 | task 1408 | graphs reused = 1524 |
| 1.25.826.684 I slot release: id 0 | task 1408 | stop processing: n_tokens = 323, truncated = 0 |
| 1.25.826.696 I srv update_slots: all slots are idle |
| 1.25.906.963 I srv operator(): operator(): cleaning up before exit... |
|
|
