0.00.360.175 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg) 0.00.360.180 I device_info: 0.00.678.145 I - CUDA0 : NVIDIA GeForce RTX 3080 (20052 MiB, 19814 MiB free) 0.00.801.472 I - CUDA1 : NVIDIA GeForce RTX 3080 (20052 MiB, 19814 MiB free) 0.00.925.346 I - CUDA2 : NVIDIA GeForce RTX 3080 (20052 MiB, 19814 MiB free) 0.00.925.368 I - CPU : AMD EPYC 7F32 8-Core Processor (193092 MiB, 193092 MiB free) 0.00.925.506 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CUDA : ARCHS = 860 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 0.00.925.642 I srv init: using 15 threads for HTTP server 0.00.926.152 I srv start: binding port with default address family 0.00.927.473 I srv llama_server: loading model 0.00.927.481 I srv load_model: loading model '/mnt/models/PropellerA-models/Qwen3.6-27B-UD-Q6_K_XL.gguf' 0.00.927.577 I common_init_result: fitting params to device memory ... 0.00.927.580 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on) 0.26.698.273 W llama_context: n_ctx_seq (16384) < n_ctx_train (262144) -- the full capacity of the model will not be utilized 0.26.803.059 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable) 0.26.965.487 I srv load_model: initializing slots, n_slots = 1 0.27.016.741 W srv load_model: speculative decoding will use checkpoints 0.27.016.761 W common_speculative_init: no implementations specified for speculative decoding 0.27.016.763 I slot load_model: id 0 | task -1 | new slot, n_ctx = 16384 0.27.016.884 I srv load_model: prompt cache is enabled, size limit: 8192 MiB 0.27.016.886 I srv load_model: use `--cache-ram 0` to disable the prompt cache 0.27.016.886 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391 0.27.016.887 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256 0.27.016.929 I srv init: idle slots will be saved to prompt cache upon starting a new task 0.27.042.130 I init: chat template, example_format: '<|im_start|>system You are a helpful assistant<|im_end|> <|im_start|>user Hello<|im_end|> <|im_start|>assistant Hi there<|im_end|> <|im_start|>user How are you?<|im_end|> <|im_start|>assistant ' 0.27.057.914 I srv init: init: chat template, thinking = 1 0.27.057.968 I srv llama_server: model loaded 0.27.057.975 I srv llama_server: server is listening on http://127.0.0.1:8291 0.27.057.986 I srv update_slots: all slots are idle 0.28.239.485 I srv params_from_: Chat format: peg-native 0.28.239.795 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = -1 0.28.239.798 I srv get_availabl: updating prompt cache 0.28.239.811 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000 0.28.239.833 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 16384 tokens, 8589934592 est) 0.28.239.834 I srv get_availabl: prompt cache update took 0.04 ms 0.28.239.978 I slot launch_slot_: id 0 | task 0 | processing task, is_child = 0 0.28.734.104 I slot create_check: id 0 | task 0 | created context checkpoint 1 of 32 (pos_min = 62, pos_max = 62, n_tokens = 63, size = 149.626 MiB) 0.29.053.958 I slot print_timing: id 0 | task 0 | prompt eval time = 563.52 ms / 67 tokens ( 8.41 ms per token, 118.90 tokens per second) 0.29.053.964 I slot print_timing: id 0 | task 0 | eval time = 250.42 ms / 7 tokens ( 35.77 ms per token, 27.95 tokens per second) 0.29.053.965 I slot print_timing: id 0 | task 0 | total time = 813.95 ms / 74 tokens 0.29.053.972 I slot print_timing: id 0 | task 0 | graphs reused = 6 0.29.054.038 I slot release: id 0 | task 0 | stop processing: n_tokens = 73, truncated = 0 0.29.054.045 I srv update_slots: all slots are idle 0.29.079.811 I srv params_from_: Chat format: peg-native 0.29.080.032 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.238 (> 0.100 thold), f_keep = 0.205 0.29.080.033 I srv get_availabl: updating prompt cache 0.29.080.085 W srv prompt_save: - saving prompt with length 73, total state size = 152.052 MiB (draft: 0.000 MiB) 0.29.273.605 I srv load: - looking for better prompt, base f_keep = 0.205, sim = 0.238 0.29.273.619 I srv update: - cache state: 1 prompts, 301.678 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) 0.29.273.620 I srv update: - prompt 0x5ca5f42e1820: 73 tokens, checkpoints: 1, 301.678 MiB 0.29.273.621 I srv get_availabl: prompt cache update took 193.59 ms 0.29.273.722 I slot launch_slot_: id 0 | task 9 | processing task, is_child = 0 0.29.273.734 W slot update_slots: id 0 | task 9 | erased invalidated context checkpoint (pos_min = 62, pos_max = 62, n_tokens = 63, n_swa = 0, pos_next = 0, size = 149.626 MiB) 0.29.396.398 I slot create_check: id 0 | task 9 | created context checkpoint 1 of 32 (pos_min = 58, pos_max = 58, n_tokens = 59, size = 149.626 MiB) 0.29.995.918 I slot print_timing: id 0 | task 9 | prompt eval time = 396.69 ms / 63 tokens ( 6.30 ms per token, 158.81 tokens per second) 0.29.995.925 I slot print_timing: id 0 | task 9 | eval time = 325.49 ms / 9 tokens ( 36.17 ms per token, 27.65 tokens per second) 0.29.995.926 I slot print_timing: id 0 | task 9 | total time = 722.17 ms / 72 tokens 0.29.995.927 I slot print_timing: id 0 | task 9 | graphs reused = 13 0.29.995.962 I slot release: id 0 | task 9 | stop processing: n_tokens = 71, truncated = 0 0.29.995.976 I srv update_slots: all slots are idle 0.30.029.333 I srv params_from_: Chat format: peg-native 0.30.029.563 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.227 (> 0.100 thold), f_keep = 0.211 0.30.029.565 I srv get_availabl: updating prompt cache 0.30.029.602 W srv prompt_save: - saving prompt with length 71, total state size = 151.985 MiB (draft: 0.000 MiB) 0.30.221.860 I srv load: - looking for better prompt, base f_keep = 0.211, sim = 0.227 0.30.221.870 I srv update: - cache state: 2 prompts, 603.289 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) 0.30.221.872 I srv update: - prompt 0x5ca5f42e1820: 73 tokens, checkpoints: 1, 301.678 MiB 0.30.221.873 I srv update: - prompt 0x5ca5f3dff200: 71 tokens, checkpoints: 1, 301.611 MiB 0.30.221.874 I srv get_availabl: prompt cache update took 192.31 ms 0.30.221.991 I slot launch_slot_: id 0 | task 20 | processing task, is_child = 0 0.30.222.006 W slot update_slots: id 0 | task 20 | erased invalidated context checkpoint (pos_min = 58, pos_max = 58, n_tokens = 59, n_swa = 0, pos_next = 0, size = 149.626 MiB) 0.30.342.574 I slot create_check: id 0 | task 20 | created context checkpoint 1 of 32 (pos_min = 61, pos_max = 61, n_tokens = 62, size = 149.626 MiB) 0.31.008.747 I slot print_timing: id 0 | task 20 | prompt eval time = 175.22 ms / 66 tokens ( 2.65 ms per token, 376.66 tokens per second) 0.31.008.754 I slot print_timing: id 0 | task 20 | eval time = 611.51 ms / 10 tokens ( 61.15 ms per token, 16.35 tokens per second) 0.31.008.755 I slot print_timing: id 0 | task 20 | total time = 786.73 ms / 76 tokens 0.31.008.756 I slot print_timing: id 0 | task 20 | graphs reused = 21 0.31.008.791 I slot release: id 0 | task 20 | stop processing: n_tokens = 75, truncated = 0 0.31.008.804 I srv update_slots: all slots are idle 0.31.042.634 I srv params_from_: Chat format: peg-native 0.31.042.880 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17906065980 0.31.042.883 I srv get_availabl: updating prompt cache 0.31.042.916 W srv prompt_save: - saving prompt with length 75, total state size = 152.118 MiB (draft: 0.000 MiB) 0.31.233.650 I srv load: - looking for better prompt, base f_keep = 0.040, sim = 0.036 0.31.233.662 I srv update: - cache state: 3 prompts, 905.034 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) 0.31.233.664 I srv update: - prompt 0x5ca5f42e1820: 73 tokens, checkpoints: 1, 301.678 MiB 0.31.233.664 I srv update: - prompt 0x5ca5f3dff200: 71 tokens, checkpoints: 1, 301.611 MiB 0.31.233.665 I srv update: - prompt 0x5ca5f384c3e0: 75 tokens, checkpoints: 1, 301.744 MiB 0.31.233.667 I srv get_availabl: prompt cache update took 190.78 ms 0.31.233.788 I slot launch_slot_: id 0 | task 32 | processing task, is_child = 0 0.31.233.803 W slot update_slots: id 0 | task 32 | erased invalidated context checkpoint (pos_min = 61, pos_max = 61, n_tokens = 62, n_swa = 0, pos_next = 0, size = 149.626 MiB) 0.31.657.518 I slot create_check: id 0 | task 32 | created context checkpoint 1 of 32 (pos_min = 78, pos_max = 78, n_tokens = 79, size = 149.626 MiB) 0.32.396.263 I slot print_timing: id 0 | task 32 | prompt eval time = 478.43 ms / 83 tokens ( 5.76 ms per token, 173.48 tokens per second) 0.32.396.269 I slot print_timing: id 0 | task 32 | eval time = 684.02 ms / 18 tokens ( 38.00 ms per token, 26.32 tokens per second) 0.32.396.270 I slot print_timing: id 0 | task 32 | total time = 1162.45 ms / 101 tokens 0.32.396.271 I slot print_timing: id 0 | task 32 | graphs reused = 37 0.32.396.305 I slot release: id 0 | task 32 | stop processing: n_tokens = 100, truncated = 0 0.32.396.320 I srv update_slots: all slots are idle 0.32.627.079 I srv params_from_: Chat format: peg-native 0.32.627.261 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17907453494 0.32.627.264 I srv get_availabl: updating prompt cache 0.32.627.317 W srv prompt_save: - saving prompt with length 100, total state size = 152.949 MiB (draft: 0.000 MiB) 0.32.819.050 I srv load: - looking for better prompt, base f_keep = 0.030, sim = 0.046 0.32.819.064 I srv update: - cache state: 4 prompts, 1207.608 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) 0.32.819.065 I srv update: - prompt 0x5ca5f42e1820: 73 tokens, checkpoints: 1, 301.678 MiB 0.32.819.066 I srv update: - prompt 0x5ca5f3dff200: 71 tokens, checkpoints: 1, 301.611 MiB 0.32.819.067 I srv update: - prompt 0x5ca5f384c3e0: 75 tokens, checkpoints: 1, 301.744 MiB 0.32.819.068 I srv update: - prompt 0x5ca6072c6d60: 100 tokens, checkpoints: 1, 302.575 MiB 0.32.819.070 I srv get_availabl: prompt cache update took 191.81 ms 0.32.819.184 I slot launch_slot_: id 0 | task 52 | processing task, is_child = 0 0.32.819.199 W slot update_slots: id 0 | task 52 | erased invalidated context checkpoint (pos_min = 78, pos_max = 78, n_tokens = 79, n_swa = 0, pos_next = 0, size = 149.626 MiB) 0.32.940.211 I slot create_check: id 0 | task 52 | created context checkpoint 1 of 32 (pos_min = 60, pos_max = 60, n_tokens = 61, size = 149.626 MiB) 0.33.443.573 I slot print_timing: id 0 | task 52 | prompt eval time = 175.68 ms / 65 tokens ( 2.70 ms per token, 369.99 tokens per second) 0.33.443.578 I slot print_timing: id 0 | task 52 | eval time = 448.69 ms / 12 tokens ( 37.39 ms per token, 26.74 tokens per second) 0.33.443.579 I slot print_timing: id 0 | task 52 | total time = 624.37 ms / 77 tokens 0.33.443.580 I slot print_timing: id 0 | task 52 | graphs reused = 47 0.33.443.611 I slot release: id 0 | task 52 | stop processing: n_tokens = 76, truncated = 0 0.33.443.623 I srv update_slots: all slots are idle 0.33.735.435 I srv params_from_: Chat format: peg-native 0.33.735.606 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17908500800 0.33.735.610 I srv get_availabl: updating prompt cache 0.33.735.658 W srv prompt_save: - saving prompt with length 76, total state size = 152.151 MiB (draft: 0.000 MiB) 0.33.920.732 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.039 0.33.920.740 I srv load: - found better prompt with f_keep = 0.340, sim = 0.442 0.33.966.123 I srv update: - cache state: 4 prompts, 1206.811 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) 0.33.966.129 I srv update: - prompt 0x5ca5f42e1820: 73 tokens, checkpoints: 1, 301.678 MiB 0.33.966.130 I srv update: - prompt 0x5ca5f3dff200: 71 tokens, checkpoints: 1, 301.611 MiB 0.33.966.130 I srv update: - prompt 0x5ca5f384c3e0: 75 tokens, checkpoints: 1, 301.744 MiB 0.33.966.131 I srv update: - prompt 0x5ca5f444bb00: 76 tokens, checkpoints: 1, 301.778 MiB 0.33.966.132 I srv get_availabl: prompt cache update took 230.52 ms 0.33.966.248 I slot launch_slot_: id 0 | task 66 | processing task, is_child = 0 0.33.966.262 W slot update_slots: id 0 | task 66 | erased invalidated context checkpoint (pos_min = 78, pos_max = 78, n_tokens = 79, n_swa = 0, pos_next = 0, size = 149.626 MiB) 0.34.100.268 I slot create_check: id 0 | task 66 | created context checkpoint 1 of 32 (pos_min = 72, pos_max = 72, n_tokens = 73, size = 149.626 MiB) 0.35.258.704 I slot print_timing: id 0 | task 66 | prompt eval time = 188.76 ms / 77 tokens ( 2.45 ms per token, 407.93 tokens per second) 0.35.258.710 I slot print_timing: id 0 | task 66 | eval time = 1103.68 ms / 23 tokens ( 47.99 ms per token, 20.84 tokens per second) 0.35.258.711 I slot print_timing: id 0 | task 66 | total time = 1292.44 ms / 100 tokens 0.35.258.712 I slot print_timing: id 0 | task 66 | graphs reused = 68 0.35.258.740 I slot release: id 0 | task 66 | stop processing: n_tokens = 99, truncated = 0 0.35.258.753 I srv update_slots: all slots are idle 0.35.292.712 I srv params_from_: Chat format: peg-native 0.35.292.999 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.393 (> 0.100 thold), f_keep = 0.354 0.35.293.003 I srv get_availabl: updating prompt cache 0.35.293.055 W srv prompt_save: - saving prompt with length 99, total state size = 152.916 MiB (draft: 0.000 MiB) 0.35.723.359 I srv load: - looking for better prompt, base f_keep = 0.354, sim = 0.393 0.35.723.371 I srv update: - cache state: 5 prompts, 1509.353 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) 0.35.723.372 I srv update: - prompt 0x5ca5f42e1820: 73 tokens, checkpoints: 1, 301.678 MiB 0.35.723.373 I srv update: - prompt 0x5ca5f3dff200: 71 tokens, checkpoints: 1, 301.611 MiB 0.35.723.374 I srv update: - prompt 0x5ca5f384c3e0: 75 tokens, checkpoints: 1, 301.744 MiB 0.35.723.374 I srv update: - prompt 0x5ca5f444bb00: 76 tokens, checkpoints: 1, 301.778 MiB 0.35.723.375 I srv update: - prompt 0x5ca5f3e01a70: 99 tokens, checkpoints: 1, 302.542 MiB 0.35.723.376 I srv get_availabl: prompt cache update took 430.37 ms 0.35.723.478 I slot launch_slot_: id 0 | task 91 | processing task, is_child = 0 0.35.723.493 W slot update_slots: id 0 | task 91 | erased invalidated context checkpoint (pos_min = 72, pos_max = 72, n_tokens = 73, n_swa = 0, pos_next = 0, size = 149.626 MiB) 0.35.906.678 I slot create_check: id 0 | task 91 | created context checkpoint 1 of 32 (pos_min = 84, pos_max = 84, n_tokens = 85, size = 149.626 MiB) 0.36.902.301 I slot print_timing: id 0 | task 91 | prompt eval time = 238.91 ms / 89 tokens ( 2.68 ms per token, 372.53 tokens per second) 0.36.902.307 I slot print_timing: id 0 | task 91 | eval time = 939.89 ms / 19 tokens ( 49.47 ms per token, 20.22 tokens per second) 0.36.902.308 I slot print_timing: id 0 | task 91 | total time = 1178.80 ms / 108 tokens 0.36.902.309 I slot print_timing: id 0 | task 91 | graphs reused = 85 0.36.902.338 I slot release: id 0 | task 91 | stop processing: n_tokens = 107, truncated = 0 0.36.902.349 I srv update_slots: all slots are idle 0.36.937.757 I srv params_from_: Chat format: peg-native 0.36.938.008 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17911959527 0.36.938.011 I srv get_availabl: updating prompt cache 0.36.938.040 W srv prompt_save: - saving prompt with length 107, total state size = 153.181 MiB (draft: 0.000 MiB) 0.37.126.563 I srv load: - looking for better prompt, base f_keep = 0.028, sim = 0.050 0.37.126.578 I srv update: - cache state: 6 prompts, 1812.160 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) 0.37.126.579 I srv update: - prompt 0x5ca5f42e1820: 73 tokens, checkpoints: 1, 301.678 MiB 0.37.126.580 I srv update: - prompt 0x5ca5f3dff200: 71 tokens, checkpoints: 1, 301.611 MiB 0.37.126.581 I srv update: - prompt 0x5ca5f384c3e0: 75 tokens, checkpoints: 1, 301.744 MiB 0.37.126.581 I srv update: - prompt 0x5ca5f444bb00: 76 tokens, checkpoints: 1, 301.778 MiB 0.37.126.582 I srv update: - prompt 0x5ca5f3e01a70: 99 tokens, checkpoints: 1, 302.542 MiB 0.37.126.583 I srv update: - prompt 0x5ca5f388d6b0: 107 tokens, checkpoints: 1, 302.807 MiB 0.37.126.584 I srv get_availabl: prompt cache update took 188.57 ms 0.37.126.689 I slot launch_slot_: id 0 | task 112 | processing task, is_child = 0 0.37.126.699 W slot update_slots: id 0 | task 112 | erased invalidated context checkpoint (pos_min = 84, pos_max = 84, n_tokens = 85, n_swa = 0, pos_next = 0, size = 149.626 MiB) 0.37.243.288 I slot create_check: id 0 | task 112 | created context checkpoint 1 of 32 (pos_min = 55, pos_max = 55, n_tokens = 56, size = 149.626 MiB) 0.37.872.488 I slot print_timing: id 0 | task 112 | prompt eval time = 171.14 ms / 60 tokens ( 2.85 ms per token, 350.59 tokens per second) 0.37.872.495 I slot print_timing: id 0 | task 112 | eval time = 574.64 ms / 9 tokens ( 63.85 ms per token, 15.66 tokens per second) 0.37.872.496 I slot print_timing: id 0 | task 112 | total time = 745.78 ms / 69 tokens 0.37.872.497 I slot print_timing: id 0 | task 112 | graphs reused = 92 0.37.872.522 I slot release: id 0 | task 112 | stop processing: n_tokens = 68, truncated = 0 0.37.872.533 I srv update_slots: all slots are idle 0.37.909.843 I srv params_from_: Chat format: peg-native 0.37.910.215 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.183 (> 0.100 thold), f_keep = 0.221 0.37.910.219 I srv get_availabl: updating prompt cache 0.37.910.246 W srv prompt_save: - saving prompt with length 68, total state size = 151.886 MiB (draft: 0.000 MiB) 0.38.096.209 I srv load: - looking for better prompt, base f_keep = 0.221, sim = 0.183 0.38.096.221 I srv update: - cache state: 7 prompts, 2113.672 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) 0.38.096.222 I srv update: - prompt 0x5ca5f42e1820: 73 tokens, checkpoints: 1, 301.678 MiB 0.38.096.223 I srv update: - prompt 0x5ca5f3dff200: 71 tokens, checkpoints: 1, 301.611 MiB 0.38.096.224 I srv update: - prompt 0x5ca5f384c3e0: 75 tokens, checkpoints: 1, 301.744 MiB 0.38.096.225 I srv update: - prompt 0x5ca5f444bb00: 76 tokens, checkpoints: 1, 301.778 MiB 0.38.096.225 I srv update: - prompt 0x5ca5f3e01a70: 99 tokens, checkpoints: 1, 302.542 MiB 0.38.096.226 I srv update: - prompt 0x5ca5f388d6b0: 107 tokens, checkpoints: 1, 302.807 MiB 0.38.096.226 I srv update: - prompt 0x5ca5f3e055f0: 68 tokens, checkpoints: 1, 301.512 MiB 0.38.096.228 I srv get_availabl: prompt cache update took 186.01 ms 0.38.096.329 I slot launch_slot_: id 0 | task 123 | processing task, is_child = 0 0.38.096.341 W slot update_slots: id 0 | task 123 | erased invalidated context checkpoint (pos_min = 55, pos_max = 55, n_tokens = 56, n_swa = 0, pos_next = 0, size = 149.626 MiB) 0.38.228.195 I slot create_check: id 0 | task 123 | created context checkpoint 1 of 32 (pos_min = 77, pos_max = 77, n_tokens = 78, size = 149.626 MiB) 0.39.120.496 I slot print_timing: id 0 | task 123 | prompt eval time = 186.38 ms / 82 tokens ( 2.27 ms per token, 439.95 tokens per second) 0.39.120.503 I slot print_timing: id 0 | task 123 | eval time = 837.76 ms / 16 tokens ( 52.36 ms per token, 19.10 tokens per second) 0.39.120.504 I slot print_timing: id 0 | task 123 | total time = 1024.15 ms / 98 tokens 0.39.120.504 I slot print_timing: id 0 | task 123 | graphs reused = 106 0.39.120.529 I slot release: id 0 | task 123 | stop processing: n_tokens = 97, truncated = 0 0.39.120.541 I srv update_slots: all slots are idle 0.39.153.717 I srv params_from_: Chat format: peg-native 0.39.153.963 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17914177718 0.39.153.966 I srv get_availabl: updating prompt cache 0.39.153.994 W srv prompt_save: - saving prompt with length 97, total state size = 152.849 MiB (draft: 0.000 MiB) 0.39.344.007 I srv load: - looking for better prompt, base f_keep = 0.031, sim = 0.033 0.39.344.017 I srv load: - found better prompt with f_keep = 0.343, sim = 0.374 0.39.389.325 I srv update: - cache state: 7 prompts, 2113.605 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) 0.39.389.332 I srv update: - prompt 0x5ca5f42e1820: 73 tokens, checkpoints: 1, 301.678 MiB 0.39.389.333 I srv update: - prompt 0x5ca5f3dff200: 71 tokens, checkpoints: 1, 301.611 MiB 0.39.389.334 I srv update: - prompt 0x5ca5f384c3e0: 75 tokens, checkpoints: 1, 301.744 MiB 0.39.389.335 I srv update: - prompt 0x5ca5f444bb00: 76 tokens, checkpoints: 1, 301.778 MiB 0.39.389.335 I srv update: - prompt 0x5ca5f388d6b0: 107 tokens, checkpoints: 1, 302.807 MiB 0.39.389.336 I srv update: - prompt 0x5ca5f3e055f0: 68 tokens, checkpoints: 1, 301.512 MiB 0.39.389.337 I srv update: - prompt 0x5ca5f417c820: 97 tokens, checkpoints: 1, 302.475 MiB 0.39.389.338 I srv get_availabl: prompt cache update took 235.37 ms 0.39.389.449 I slot launch_slot_: id 0 | task 141 | processing task, is_child = 0 0.39.389.463 W slot update_slots: id 0 | task 141 | erased invalidated context checkpoint (pos_min = 72, pos_max = 72, n_tokens = 73, n_swa = 0, pos_next = 0, size = 149.626 MiB) 0.39.776.668 I slot create_check: id 0 | task 141 | created context checkpoint 1 of 32 (pos_min = 86, pos_max = 86, n_tokens = 87, size = 149.626 MiB) 0.41.119.507 I slot print_timing: id 0 | task 141 | prompt eval time = 441.76 ms / 91 tokens ( 4.85 ms per token, 205.99 tokens per second) 0.41.119.513 I slot print_timing: id 0 | task 141 | eval time = 1288.28 ms / 27 tokens ( 47.71 ms per token, 20.96 tokens per second) 0.41.119.514 I slot print_timing: id 0 | task 141 | total time = 1730.04 ms / 118 tokens 0.41.119.515 I slot print_timing: id 0 | task 141 | graphs reused = 131 0.41.119.542 I slot release: id 0 | task 141 | stop processing: n_tokens = 117, truncated = 0 0.41.119.552 I srv update_slots: all slots are idle 0.41.153.041 I srv params_from_: Chat format: peg-native 0.41.153.278 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17916176731 0.41.153.281 I srv get_availabl: updating prompt cache 0.41.153.308 W srv prompt_save: - saving prompt with length 117, total state size = 153.514 MiB (draft: 0.000 MiB) 0.41.342.933 I srv load: - looking for better prompt, base f_keep = 0.026, sim = 0.024 0.41.342.947 I srv update: - cache state: 8 prompts, 2416.745 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) 0.41.342.948 I srv update: - prompt 0x5ca5f42e1820: 73 tokens, checkpoints: 1, 301.678 MiB 0.41.342.949 I srv update: - prompt 0x5ca5f3dff200: 71 tokens, checkpoints: 1, 301.611 MiB 0.41.342.950 I srv update: - prompt 0x5ca5f384c3e0: 75 tokens, checkpoints: 1, 301.744 MiB 0.41.342.950 I srv update: - prompt 0x5ca5f444bb00: 76 tokens, checkpoints: 1, 301.778 MiB 0.41.342.951 I srv update: - prompt 0x5ca5f388d6b0: 107 tokens, checkpoints: 1, 302.807 MiB 0.41.342.952 I srv update: - prompt 0x5ca5f3e055f0: 68 tokens, checkpoints: 1, 301.512 MiB 0.41.342.952 I srv update: - prompt 0x5ca5f417c820: 97 tokens, checkpoints: 1, 302.475 MiB 0.41.342.953 I srv update: - prompt 0x5ca5f3dbc470: 117 tokens, checkpoints: 1, 303.140 MiB 0.41.342.954 I srv get_availabl: prompt cache update took 189.67 ms 0.41.343.057 I slot launch_slot_: id 0 | task 170 | processing task, is_child = 0 0.41.343.068 W slot update_slots: id 0 | task 170 | erased invalidated context checkpoint (pos_min = 86, pos_max = 86, n_tokens = 87, n_swa = 0, pos_next = 0, size = 149.626 MiB) 0.41.798.147 I slot create_check: id 0 | task 170 | created context checkpoint 1 of 32 (pos_min = 119, pos_max = 119, n_tokens = 120, size = 149.626 MiB) 0.46.738.910 I slot print_timing: id 0 | task 170 | n_decoded = 100, tg = 20.47 t/s 0.47.702.056 I slot print_timing: id 0 | task 170 | prompt eval time = 509.61 ms / 124 tokens ( 4.11 ms per token, 243.32 tokens per second) 0.47.702.061 I slot print_timing: id 0 | task 170 | eval time = 5849.38 ms / 118 tokens ( 49.57 ms per token, 20.17 tokens per second) 0.47.702.062 I slot print_timing: id 0 | task 170 | total time = 6358.99 ms / 242 tokens 0.47.702.063 I slot print_timing: id 0 | task 170 | graphs reused = 247 0.47.702.092 I slot release: id 0 | task 170 | stop processing: n_tokens = 241, truncated = 0 0.47.702.105 I srv update_slots: all slots are idle 0.47.744.055 I srv params_from_: Chat format: peg-native 0.47.744.225 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.129 (> 0.100 thold), f_keep = 0.050 0.47.744.229 I srv get_availabl: updating prompt cache 0.47.744.265 W srv prompt_save: - saving prompt with length 241, total state size = 157.633 MiB (draft: 0.000 MiB) 0.47.940.639 I srv load: - looking for better prompt, base f_keep = 0.050, sim = 0.129 0.47.940.655 I srv update: - cache state: 9 prompts, 2724.004 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) 0.47.940.656 I srv update: - prompt 0x5ca5f42e1820: 73 tokens, checkpoints: 1, 301.678 MiB 0.47.940.657 I srv update: - prompt 0x5ca5f3dff200: 71 tokens, checkpoints: 1, 301.611 MiB 0.47.940.658 I srv update: - prompt 0x5ca5f384c3e0: 75 tokens, checkpoints: 1, 301.744 MiB 0.47.940.658 I srv update: - prompt 0x5ca5f444bb00: 76 tokens, checkpoints: 1, 301.778 MiB 0.47.940.659 I srv update: - prompt 0x5ca5f388d6b0: 107 tokens, checkpoints: 1, 302.807 MiB 0.47.940.659 I srv update: - prompt 0x5ca5f3e055f0: 68 tokens, checkpoints: 1, 301.512 MiB 0.47.940.660 I srv update: - prompt 0x5ca5f417c820: 97 tokens, checkpoints: 1, 302.475 MiB 0.47.940.661 I srv update: - prompt 0x5ca5f3dbc470: 117 tokens, checkpoints: 1, 303.140 MiB 0.47.940.661 I srv update: - prompt 0x5ca5f3dbe640: 241 tokens, checkpoints: 1, 307.259 MiB 0.47.940.662 I srv get_availabl: prompt cache update took 196.43 ms 0.47.940.761 I slot launch_slot_: id 0 | task 290 | processing task, is_child = 0 0.47.940.771 W slot update_slots: id 0 | task 290 | erased invalidated context checkpoint (pos_min = 119, pos_max = 119, n_tokens = 120, n_swa = 0, pos_next = 0, size = 149.626 MiB) 0.48.083.544 I slot create_check: id 0 | task 290 | created context checkpoint 1 of 32 (pos_min = 88, pos_max = 88, n_tokens = 89, size = 149.626 MiB) 0.52.345.406 I slot print_timing: id 0 | task 290 | prompt eval time = 197.37 ms / 93 tokens ( 2.12 ms per token, 471.21 tokens per second) 0.52.345.412 I slot print_timing: id 0 | task 290 | eval time = 4207.26 ms / 88 tokens ( 47.81 ms per token, 20.92 tokens per second) 0.52.345.413 I slot print_timing: id 0 | task 290 | total time = 4404.63 ms / 181 tokens 0.52.345.414 I slot print_timing: id 0 | task 290 | graphs reused = 333 0.52.345.456 I slot release: id 0 | task 290 | stop processing: n_tokens = 180, truncated = 0 0.52.345.473 I srv update_slots: all slots are idle 0.52.398.619 I srv params_from_: Chat format: peg-native 0.52.398.905 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.138 (> 0.100 thold), f_keep = 0.067 0.52.398.908 I srv get_availabl: updating prompt cache 0.52.398.950 W srv prompt_save: - saving prompt with length 180, total state size = 155.607 MiB (draft: 0.000 MiB) 0.52.867.722 I srv load: - looking for better prompt, base f_keep = 0.067, sim = 0.138 0.52.867.736 I srv update: - cache state: 10 prompts, 3029.237 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) 0.52.867.737 I srv update: - prompt 0x5ca5f42e1820: 73 tokens, checkpoints: 1, 301.678 MiB 0.52.867.738 I srv update: - prompt 0x5ca5f3dff200: 71 tokens, checkpoints: 1, 301.611 MiB 0.52.867.739 I srv update: - prompt 0x5ca5f384c3e0: 75 tokens, checkpoints: 1, 301.744 MiB 0.52.867.739 I srv update: - prompt 0x5ca5f444bb00: 76 tokens, checkpoints: 1, 301.778 MiB 0.52.867.740 I srv update: - prompt 0x5ca5f388d6b0: 107 tokens, checkpoints: 1, 302.807 MiB 0.52.867.741 I srv update: - prompt 0x5ca5f3e055f0: 68 tokens, checkpoints: 1, 301.512 MiB 0.52.867.741 I srv update: - prompt 0x5ca5f417c820: 97 tokens, checkpoints: 1, 302.475 MiB 0.52.867.742 I srv update: - prompt 0x5ca5f3dbc470: 117 tokens, checkpoints: 1, 303.140 MiB 0.52.867.744 I srv update: - prompt 0x5ca5f3dbe640: 241 tokens, checkpoints: 1, 307.259 MiB 0.52.867.745 I srv update: - prompt 0x5ca6072cd250: 180 tokens, checkpoints: 1, 305.233 MiB 0.52.867.746 I srv get_availabl: prompt cache update took 468.84 ms 0.52.867.855 I slot launch_slot_: id 0 | task 380 | processing task, is_child = 0 0.52.867.868 W slot update_slots: id 0 | task 380 | erased invalidated context checkpoint (pos_min = 88, pos_max = 88, n_tokens = 89, n_swa = 0, pos_next = 0, size = 149.626 MiB) 0.53.009.600 I slot create_check: id 0 | task 380 | created context checkpoint 1 of 32 (pos_min = 82, pos_max = 82, n_tokens = 83, size = 149.626 MiB) 0.57.615.839 I slot print_timing: id 0 | task 380 | n_decoded = 100, tg = 21.97 t/s 1.00.620.009 I slot print_timing: id 0 | task 380 | n_decoded = 164, tg = 21.71 t/s 1.03.654.364 I slot print_timing: id 0 | task 380 | n_decoded = 228, tg = 21.53 t/s 1.04.924.405 I slot print_timing: id 0 | task 380 | prompt eval time = 196.42 ms / 87 tokens ( 2.26 ms per token, 442.93 tokens per second) 1.04.924.411 I slot print_timing: id 0 | task 380 | eval time = 11860.11 ms / 256 tokens ( 46.33 ms per token, 21.58 tokens per second) 1.04.924.412 I slot print_timing: id 0 | task 380 | total time = 12056.53 ms / 343 tokens 1.04.924.412 I slot print_timing: id 0 | task 380 | graphs reused = 586 1.04.924.442 I slot release: id 0 | task 380 | stop processing: n_tokens = 342, truncated = 0 1.04.924.456 I srv update_slots: all slots are idle 1.04.968.825 I srv params_from_: Chat format: peg-native 1.04.968.994 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.035 1.04.968.996 I srv get_availabl: updating prompt cache 1.04.969.038 W srv prompt_save: - saving prompt with length 342, total state size = 160.988 MiB (draft: 0.000 MiB) 1.05.165.222 I srv load: - looking for better prompt, base f_keep = 0.035, sim = 0.107 1.05.165.238 I srv update: - cache state: 11 prompts, 3339.851 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) 1.05.165.239 I srv update: - prompt 0x5ca5f42e1820: 73 tokens, checkpoints: 1, 301.678 MiB 1.05.165.240 I srv update: - prompt 0x5ca5f3dff200: 71 tokens, checkpoints: 1, 301.611 MiB 1.05.165.240 I srv update: - prompt 0x5ca5f384c3e0: 75 tokens, checkpoints: 1, 301.744 MiB 1.05.165.241 I srv update: - prompt 0x5ca5f444bb00: 76 tokens, checkpoints: 1, 301.778 MiB 1.05.165.242 I srv update: - prompt 0x5ca5f388d6b0: 107 tokens, checkpoints: 1, 302.807 MiB 1.05.165.243 I srv update: - prompt 0x5ca5f3e055f0: 68 tokens, checkpoints: 1, 301.512 MiB 1.05.165.243 I srv update: - prompt 0x5ca5f417c820: 97 tokens, checkpoints: 1, 302.475 MiB 1.05.165.244 I srv update: - prompt 0x5ca5f3dbc470: 117 tokens, checkpoints: 1, 303.140 MiB 1.05.165.244 I srv update: - prompt 0x5ca5f3dbe640: 241 tokens, checkpoints: 1, 307.259 MiB 1.05.165.245 I srv update: - prompt 0x5ca6072cd250: 180 tokens, checkpoints: 1, 305.233 MiB 1.05.165.245 I srv update: - prompt 0x5ca5f3e4d2b0: 342 tokens, checkpoints: 1, 310.615 MiB 1.05.165.247 I srv get_availabl: prompt cache update took 196.25 ms 1.05.165.355 I slot launch_slot_: id 0 | task 638 | processing task, is_child = 0 1.05.165.366 W slot update_slots: id 0 | task 638 | erased invalidated context checkpoint (pos_min = 82, pos_max = 82, n_tokens = 83, n_swa = 0, pos_next = 0, size = 149.626 MiB) 1.05.362.404 I slot create_check: id 0 | task 638 | created context checkpoint 1 of 32 (pos_min = 107, pos_max = 107, n_tokens = 108, size = 149.626 MiB) 1.09.709.718 I slot print_timing: id 0 | task 638 | prompt eval time = 251.71 ms / 112 tokens ( 2.25 ms per token, 444.95 tokens per second) 1.09.709.724 I slot print_timing: id 0 | task 638 | eval time = 4292.63 ms / 80 tokens ( 53.66 ms per token, 18.64 tokens per second) 1.09.709.724 I slot print_timing: id 0 | task 638 | total time = 4544.34 ms / 192 tokens 1.09.709.725 I slot print_timing: id 0 | task 638 | graphs reused = 664 1.09.709.754 I slot release: id 0 | task 638 | stop processing: n_tokens = 191, truncated = 0 1.09.709.767 I srv update_slots: all slots are idle 1.09.745.828 I srv params_from_: Chat format: peg-native 1.09.746.009 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.136 (> 0.100 thold), f_keep = 0.063 1.09.746.013 I srv get_availabl: updating prompt cache 1.09.746.053 W srv prompt_save: - saving prompt with length 191, total state size = 155.972 MiB (draft: 0.000 MiB) 1.09.933.395 I srv load: - looking for better prompt, base f_keep = 0.063, sim = 0.136 1.09.933.409 I srv update: - cache state: 12 prompts, 3645.450 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) 1.09.933.410 I srv update: - prompt 0x5ca5f42e1820: 73 tokens, checkpoints: 1, 301.678 MiB 1.09.933.411 I srv update: - prompt 0x5ca5f3dff200: 71 tokens, checkpoints: 1, 301.611 MiB 1.09.933.412 I srv update: - prompt 0x5ca5f384c3e0: 75 tokens, checkpoints: 1, 301.744 MiB 1.09.933.412 I srv update: - prompt 0x5ca5f444bb00: 76 tokens, checkpoints: 1, 301.778 MiB 1.09.933.413 I srv update: - prompt 0x5ca5f388d6b0: 107 tokens, checkpoints: 1, 302.807 MiB 1.09.933.413 I srv update: - prompt 0x5ca5f3e055f0: 68 tokens, checkpoints: 1, 301.512 MiB 1.09.933.414 I srv update: - prompt 0x5ca5f417c820: 97 tokens, checkpoints: 1, 302.475 MiB 1.09.933.414 I srv update: - prompt 0x5ca5f3dbc470: 117 tokens, checkpoints: 1, 303.140 MiB 1.09.933.415 I srv update: - prompt 0x5ca5f3dbe640: 241 tokens, checkpoints: 1, 307.259 MiB 1.09.933.415 I srv update: - prompt 0x5ca6072cd250: 180 tokens, checkpoints: 1, 305.233 MiB 1.09.933.416 I srv update: - prompt 0x5ca5f3e4d2b0: 342 tokens, checkpoints: 1, 310.615 MiB 1.09.933.416 I srv update: - prompt 0x5ca5f420b7b0: 191 tokens, checkpoints: 1, 305.598 MiB 1.09.933.418 I srv get_availabl: prompt cache update took 187.40 ms 1.09.933.520 I slot launch_slot_: id 0 | task 720 | processing task, is_child = 0 1.09.933.531 W slot update_slots: id 0 | task 720 | erased invalidated context checkpoint (pos_min = 107, pos_max = 107, n_tokens = 108, n_swa = 0, pos_next = 0, size = 149.626 MiB) 1.10.074.225 I slot create_check: id 0 | task 720 | created context checkpoint 1 of 32 (pos_min = 83, pos_max = 83, n_tokens = 84, size = 149.626 MiB) 1.14.651.797 I slot print_timing: id 0 | task 720 | prompt eval time = 195.25 ms / 88 tokens ( 2.22 ms per token, 450.71 tokens per second) 1.14.651.803 I slot print_timing: id 0 | task 720 | eval time = 4523.01 ms / 93 tokens ( 48.63 ms per token, 20.56 tokens per second) 1.14.651.804 I slot print_timing: id 0 | task 720 | total time = 4718.26 ms / 181 tokens 1.14.651.805 I slot print_timing: id 0 | task 720 | graphs reused = 755 1.14.651.837 I slot release: id 0 | task 720 | stop processing: n_tokens = 180, truncated = 0 1.14.651.848 I srv update_slots: all slots are idle 1.14.698.704 I srv params_from_: Chat format: peg-native 1.14.698.943 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.114 (> 0.100 thold), f_keep = 0.067 1.14.698.949 I srv get_availabl: updating prompt cache 1.14.699.002 W srv prompt_save: - saving prompt with length 180, total state size = 155.607 MiB (draft: 0.000 MiB) 1.14.889.239 I srv load: - looking for better prompt, base f_keep = 0.067, sim = 0.114 1.14.889.252 I srv update: - cache state: 13 prompts, 3950.682 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) 1.14.889.253 I srv update: - prompt 0x5ca5f42e1820: 73 tokens, checkpoints: 1, 301.678 MiB 1.14.889.254 I srv update: - prompt 0x5ca5f3dff200: 71 tokens, checkpoints: 1, 301.611 MiB 1.14.889.255 I srv update: - prompt 0x5ca5f384c3e0: 75 tokens, checkpoints: 1, 301.744 MiB 1.14.889.255 I srv update: - prompt 0x5ca5f444bb00: 76 tokens, checkpoints: 1, 301.778 MiB 1.14.889.256 I srv update: - prompt 0x5ca5f388d6b0: 107 tokens, checkpoints: 1, 302.807 MiB 1.14.889.257 I srv update: - prompt 0x5ca5f3e055f0: 68 tokens, checkpoints: 1, 301.512 MiB 1.14.889.257 I srv update: - prompt 0x5ca5f417c820: 97 tokens, checkpoints: 1, 302.475 MiB 1.14.889.258 I srv update: - prompt 0x5ca5f3dbc470: 117 tokens, checkpoints: 1, 303.140 MiB 1.14.889.258 I srv update: - prompt 0x5ca5f3dbe640: 241 tokens, checkpoints: 1, 307.259 MiB 1.14.889.259 I srv update: - prompt 0x5ca6072cd250: 180 tokens, checkpoints: 1, 305.233 MiB 1.14.889.259 I srv update: - prompt 0x5ca5f3e4d2b0: 342 tokens, checkpoints: 1, 310.615 MiB 1.14.889.260 I srv update: - prompt 0x5ca5f420b7b0: 191 tokens, checkpoints: 1, 305.598 MiB 1.14.889.260 I srv update: - prompt 0x5ca5f3bc5bf0: 180 tokens, checkpoints: 1, 305.233 MiB 1.14.889.262 I srv get_availabl: prompt cache update took 190.31 ms 1.14.889.364 I slot launch_slot_: id 0 | task 815 | processing task, is_child = 0 1.14.889.376 W slot update_slots: id 0 | task 815 | erased invalidated context checkpoint (pos_min = 83, pos_max = 83, n_tokens = 84, n_swa = 0, pos_next = 0, size = 149.626 MiB) 1.15.041.199 I slot create_check: id 0 | task 815 | created context checkpoint 1 of 32 (pos_min = 100, pos_max = 100, n_tokens = 101, size = 149.626 MiB) 1.19.857.314 I slot print_timing: id 0 | task 815 | n_decoded = 100, tg = 21.00 t/s 1.22.891.401 I slot print_timing: id 0 | task 815 | n_decoded = 165, tg = 21.17 t/s 1.24.092.037 I slot print_timing: id 0 | task 815 | prompt eval time = 206.43 ms / 105 tokens ( 1.97 ms per token, 508.64 tokens per second) 1.24.092.042 I slot print_timing: id 0 | task 815 | eval time = 8996.23 ms / 191 tokens ( 47.10 ms per token, 21.23 tokens per second) 1.24.092.043 I slot print_timing: id 0 | task 815 | total time = 9202.66 ms / 296 tokens 1.24.092.044 I slot print_timing: id 0 | task 815 | graphs reused = 943 1.24.092.075 I slot release: id 0 | task 815 | stop processing: n_tokens = 295, truncated = 0 1.24.092.086 I srv update_slots: all slots are idle 1.24.143.760 I srv params_from_: Chat format: peg-native 1.24.144.048 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.133 (> 0.100 thold), f_keep = 0.041 1.24.144.053 I srv get_availabl: updating prompt cache 1.24.144.138 W srv prompt_save: - saving prompt with length 295, total state size = 159.427 MiB (draft: 0.000 MiB) 1.24.338.495 I srv load: - looking for better prompt, base f_keep = 0.041, sim = 0.133 1.24.338.508 I srv update: - cache state: 14 prompts, 4259.735 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) 1.24.338.510 I srv update: - prompt 0x5ca5f42e1820: 73 tokens, checkpoints: 1, 301.678 MiB 1.24.338.510 I srv update: - prompt 0x5ca5f3dff200: 71 tokens, checkpoints: 1, 301.611 MiB 1.24.338.511 I srv update: - prompt 0x5ca5f384c3e0: 75 tokens, checkpoints: 1, 301.744 MiB 1.24.338.512 I srv update: - prompt 0x5ca5f444bb00: 76 tokens, checkpoints: 1, 301.778 MiB 1.24.338.512 I srv update: - prompt 0x5ca5f388d6b0: 107 tokens, checkpoints: 1, 302.807 MiB 1.24.338.513 I srv update: - prompt 0x5ca5f3e055f0: 68 tokens, checkpoints: 1, 301.512 MiB 1.24.338.514 I srv update: - prompt 0x5ca5f417c820: 97 tokens, checkpoints: 1, 302.475 MiB 1.24.338.514 I srv update: - prompt 0x5ca5f3dbc470: 117 tokens, checkpoints: 1, 303.140 MiB 1.24.338.515 I srv update: - prompt 0x5ca5f3dbe640: 241 tokens, checkpoints: 1, 307.259 MiB 1.24.338.516 I srv update: - prompt 0x5ca6072cd250: 180 tokens, checkpoints: 1, 305.233 MiB 1.24.338.516 I srv update: - prompt 0x5ca5f3e4d2b0: 342 tokens, checkpoints: 1, 310.615 MiB 1.24.338.517 I srv update: - prompt 0x5ca5f420b7b0: 191 tokens, checkpoints: 1, 305.598 MiB 1.24.338.517 I srv update: - prompt 0x5ca5f3bc5bf0: 180 tokens, checkpoints: 1, 305.233 MiB 1.24.338.518 I srv update: - prompt 0x5ca5f44005d0: 295 tokens, checkpoints: 1, 309.053 MiB 1.24.338.519 I srv get_availabl: prompt cache update took 194.46 ms 1.24.338.623 I slot launch_slot_: id 0 | task 1008 | processing task, is_child = 0 1.24.338.634 W slot update_slots: id 0 | task 1008 | erased invalidated context checkpoint (pos_min = 100, pos_max = 100, n_tokens = 101, n_swa = 0, pos_next = 0, size = 149.626 MiB) 1.24.639.576 I slot create_check: id 0 | task 1008 | created context checkpoint 1 of 32 (pos_min = 85, pos_max = 85, n_tokens = 86, size = 149.626 MiB) 1.29.319.211 I slot print_timing: id 0 | task 1008 | n_decoded = 100, tg = 21.62 t/s 1.32.334.346 I slot print_timing: id 0 | task 1008 | n_decoded = 164, tg = 21.47 t/s 1.32.908.620 I slot print_timing: id 0 | task 1008 | prompt eval time = 355.58 ms / 90 tokens ( 3.95 ms per token, 253.11 tokens per second) 1.32.908.639 I slot print_timing: id 0 | task 1008 | eval time = 8214.40 ms / 174 tokens ( 47.21 ms per token, 21.18 tokens per second) 1.32.908.640 I slot print_timing: id 0 | task 1008 | total time = 8569.97 ms / 264 tokens 1.32.908.642 I slot print_timing: id 0 | task 1008 | graphs reused = 1114 1.32.908.697 I slot release: id 0 | task 1008 | stop processing: n_tokens = 263, truncated = 0 1.32.908.718 I srv update_slots: all slots are idle 1.32.965.136 I srv params_from_: Chat format: peg-native 1.32.965.374 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17967965886 1.32.965.377 I srv get_availabl: updating prompt cache 1.32.965.446 W srv prompt_save: - saving prompt with length 263, total state size = 158.364 MiB (draft: 0.000 MiB) 1.33.157.721 I srv load: - looking for better prompt, base f_keep = 0.030, sim = 0.060 1.33.157.740 I srv update: - cache state: 15 prompts, 4567.725 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) 1.33.157.741 I srv update: - prompt 0x5ca5f42e1820: 73 tokens, checkpoints: 1, 301.678 MiB 1.33.157.742 I srv update: - prompt 0x5ca5f3dff200: 71 tokens, checkpoints: 1, 301.611 MiB 1.33.157.743 I srv update: - prompt 0x5ca5f384c3e0: 75 tokens, checkpoints: 1, 301.744 MiB 1.33.157.744 I srv update: - prompt 0x5ca5f444bb00: 76 tokens, checkpoints: 1, 301.778 MiB 1.33.157.745 I srv update: - prompt 0x5ca5f388d6b0: 107 tokens, checkpoints: 1, 302.807 MiB 1.33.157.745 I srv update: - prompt 0x5ca5f3e055f0: 68 tokens, checkpoints: 1, 301.512 MiB 1.33.157.746 I srv update: - prompt 0x5ca5f417c820: 97 tokens, checkpoints: 1, 302.475 MiB 1.33.157.747 I srv update: - prompt 0x5ca5f3dbc470: 117 tokens, checkpoints: 1, 303.140 MiB 1.33.157.747 I srv update: - prompt 0x5ca5f3dbe640: 241 tokens, checkpoints: 1, 307.259 MiB 1.33.157.748 I srv update: - prompt 0x5ca6072cd250: 180 tokens, checkpoints: 1, 305.233 MiB 1.33.157.748 I srv update: - prompt 0x5ca5f3e4d2b0: 342 tokens, checkpoints: 1, 310.615 MiB 1.33.157.749 I srv update: - prompt 0x5ca5f420b7b0: 191 tokens, checkpoints: 1, 305.598 MiB 1.33.157.750 I srv update: - prompt 0x5ca5f3bc5bf0: 180 tokens, checkpoints: 1, 305.233 MiB 1.33.157.751 I srv update: - prompt 0x5ca5f44005d0: 295 tokens, checkpoints: 1, 309.053 MiB 1.33.157.751 I srv update: - prompt 0x5ca5f42ec410: 263 tokens, checkpoints: 1, 307.990 MiB 1.33.157.753 I srv get_availabl: prompt cache update took 192.38 ms 1.33.157.890 I slot launch_slot_: id 0 | task 1184 | processing task, is_child = 0 1.33.157.904 W slot update_slots: id 0 | task 1184 | erased invalidated context checkpoint (pos_min = 85, pos_max = 85, n_tokens = 86, n_swa = 0, pos_next = 0, size = 149.626 MiB) 1.33.363.833 I slot create_check: id 0 | task 1184 | created context checkpoint 1 of 32 (pos_min = 128, pos_max = 128, n_tokens = 129, size = 149.626 MiB) 1.38.262.295 I slot print_timing: id 0 | task 1184 | n_decoded = 100, tg = 21.36 t/s 1.40.978.931 I slot print_timing: id 0 | task 1184 | prompt eval time = 423.27 ms / 133 tokens ( 3.18 ms per token, 314.22 tokens per second) 1.40.978.936 I slot print_timing: id 0 | task 1184 | eval time = 7397.75 ms / 156 tokens ( 47.42 ms per token, 21.09 tokens per second) 1.40.978.937 I slot print_timing: id 0 | task 1184 | total time = 7821.02 ms / 289 tokens 1.40.978.938 I slot print_timing: id 0 | task 1184 | graphs reused = 1267 1.40.978.972 I slot release: id 0 | task 1184 | stop processing: n_tokens = 288, truncated = 0 1.40.978.983 I srv update_slots: all slots are idle 1.41.021.035 I srv params_from_: Chat format: peg-native 1.41.021.205 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17976036161 1.41.021.208 I srv get_availabl: updating prompt cache 1.41.021.272 W srv prompt_save: - saving prompt with length 288, total state size = 159.194 MiB (draft: 0.000 MiB) 1.41.218.849 I srv load: - looking for better prompt, base f_keep = 0.028, sim = 0.075 1.41.218.865 I srv update: - cache state: 16 prompts, 4876.546 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) 1.41.218.867 I srv update: - prompt 0x5ca5f42e1820: 73 tokens, checkpoints: 1, 301.678 MiB 1.41.218.868 I srv update: - prompt 0x5ca5f3dff200: 71 tokens, checkpoints: 1, 301.611 MiB 1.41.218.868 I srv update: - prompt 0x5ca5f384c3e0: 75 tokens, checkpoints: 1, 301.744 MiB 1.41.218.870 I srv update: - prompt 0x5ca5f444bb00: 76 tokens, checkpoints: 1, 301.778 MiB 1.41.218.871 I srv update: - prompt 0x5ca5f388d6b0: 107 tokens, checkpoints: 1, 302.807 MiB 1.41.218.872 I srv update: - prompt 0x5ca5f3e055f0: 68 tokens, checkpoints: 1, 301.512 MiB 1.41.218.873 I srv update: - prompt 0x5ca5f417c820: 97 tokens, checkpoints: 1, 302.475 MiB 1.41.218.873 I srv update: - prompt 0x5ca5f3dbc470: 117 tokens, checkpoints: 1, 303.140 MiB 1.41.218.874 I srv update: - prompt 0x5ca5f3dbe640: 241 tokens, checkpoints: 1, 307.259 MiB 1.41.218.875 I srv update: - prompt 0x5ca6072cd250: 180 tokens, checkpoints: 1, 305.233 MiB 1.41.218.875 I srv update: - prompt 0x5ca5f3e4d2b0: 342 tokens, checkpoints: 1, 310.615 MiB 1.41.218.876 I srv update: - prompt 0x5ca5f420b7b0: 191 tokens, checkpoints: 1, 305.598 MiB 1.41.218.876 I srv update: - prompt 0x5ca5f3bc5bf0: 180 tokens, checkpoints: 1, 305.233 MiB 1.41.218.877 I srv update: - prompt 0x5ca5f44005d0: 295 tokens, checkpoints: 1, 309.053 MiB 1.41.218.878 I srv update: - prompt 0x5ca5f42ec410: 263 tokens, checkpoints: 1, 307.990 MiB 1.41.218.878 I srv update: - prompt 0x5ca5e0197a50: 288 tokens, checkpoints: 1, 308.821 MiB 1.41.218.880 I srv get_availabl: prompt cache update took 197.67 ms 1.41.218.999 I slot launch_slot_: id 0 | task 1342 | processing task, is_child = 0 1.41.219.017 W slot update_slots: id 0 | task 1342 | erased invalidated context checkpoint (pos_min = 128, pos_max = 128, n_tokens = 129, n_swa = 0, pos_next = 0, size = 149.626 MiB) 1.41.371.937 I slot create_check: id 0 | task 1342 | created context checkpoint 1 of 32 (pos_min = 101, pos_max = 101, n_tokens = 102, size = 149.626 MiB) 1.44.092.749 I slot print_timing: id 0 | task 1342 | prompt eval time = 362.24 ms / 106 tokens ( 3.42 ms per token, 292.63 tokens per second) 1.44.092.757 I slot print_timing: id 0 | task 1342 | eval time = 2511.49 ms / 56 tokens ( 44.85 ms per token, 22.30 tokens per second) 1.44.092.757 I slot print_timing: id 0 | task 1342 | total time = 2873.72 ms / 162 tokens 1.44.092.758 I slot print_timing: id 0 | task 1342 | graphs reused = 1321 1.44.092.803 I slot release: id 0 | task 1342 | stop processing: n_tokens = 161, truncated = 0 1.44.092.826 I srv update_slots: all slots are idle 1.44.135.759 I srv params_from_: Chat format: peg-native 1.44.135.933 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17979149992 1.44.135.935 I srv get_availabl: updating prompt cache 1.44.135.971 W srv prompt_save: - saving prompt with length 161, total state size = 154.975 MiB (draft: 0.000 MiB) 1.44.329.897 I srv load: - looking for better prompt, base f_keep = 0.075, sim = 0.094 1.44.329.917 I srv update: - cache state: 17 prompts, 5181.147 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) 1.44.329.919 I srv update: - prompt 0x5ca5f42e1820: 73 tokens, checkpoints: 1, 301.678 MiB 1.44.329.920 I srv update: - prompt 0x5ca5f3dff200: 71 tokens, checkpoints: 1, 301.611 MiB 1.44.329.921 I srv update: - prompt 0x5ca5f384c3e0: 75 tokens, checkpoints: 1, 301.744 MiB 1.44.329.922 I srv update: - prompt 0x5ca5f444bb00: 76 tokens, checkpoints: 1, 301.778 MiB 1.44.329.922 I srv update: - prompt 0x5ca5f388d6b0: 107 tokens, checkpoints: 1, 302.807 MiB 1.44.329.923 I srv update: - prompt 0x5ca5f3e055f0: 68 tokens, checkpoints: 1, 301.512 MiB 1.44.329.924 I srv update: - prompt 0x5ca5f417c820: 97 tokens, checkpoints: 1, 302.475 MiB 1.44.329.924 I srv update: - prompt 0x5ca5f3dbc470: 117 tokens, checkpoints: 1, 303.140 MiB 1.44.329.925 I srv update: - prompt 0x5ca5f3dbe640: 241 tokens, checkpoints: 1, 307.259 MiB 1.44.329.925 I srv update: - prompt 0x5ca6072cd250: 180 tokens, checkpoints: 1, 305.233 MiB 1.44.329.926 I srv update: - prompt 0x5ca5f3e4d2b0: 342 tokens, checkpoints: 1, 310.615 MiB 1.44.329.927 I srv update: - prompt 0x5ca5f420b7b0: 191 tokens, checkpoints: 1, 305.598 MiB 1.44.329.927 I srv update: - prompt 0x5ca5f3bc5bf0: 180 tokens, checkpoints: 1, 305.233 MiB 1.44.329.928 I srv update: - prompt 0x5ca5f44005d0: 295 tokens, checkpoints: 1, 309.053 MiB 1.44.329.929 I srv update: - prompt 0x5ca5f42ec410: 263 tokens, checkpoints: 1, 307.990 MiB 1.44.329.933 I srv update: - prompt 0x5ca5e0197a50: 288 tokens, checkpoints: 1, 308.821 MiB 1.44.329.935 I srv update: - prompt 0x5ca6073946a0: 161 tokens, checkpoints: 1, 304.601 MiB 1.44.329.937 I srv get_availabl: prompt cache update took 194.00 ms 1.44.330.063 I slot launch_slot_: id 0 | task 1400 | processing task, is_child = 0 1.44.330.076 W slot update_slots: id 0 | task 1400 | erased invalidated context checkpoint (pos_min = 101, pos_max = 101, n_tokens = 102, n_swa = 0, pos_next = 0, size = 149.626 MiB) 1.44.632.385 I slot create_check: id 0 | task 1400 | created context checkpoint 1 of 32 (pos_min = 122, pos_max = 122, n_tokens = 123, size = 149.626 MiB) 1.49.258.273 I slot print_timing: id 0 | task 1400 | n_decoded = 100, tg = 21.88 t/s 1.52.282.525 I slot print_timing: id 0 | task 1400 | n_decoded = 164, tg = 21.59 t/s 1.53.691.988 I slot print_timing: id 0 | task 1400 | prompt eval time = 357.14 ms / 127 tokens ( 2.81 ms per token, 355.61 tokens per second) 1.53.691.993 I slot print_timing: id 0 | task 1400 | eval time = 9004.77 ms / 192 tokens ( 46.90 ms per token, 21.32 tokens per second) 1.53.691.994 I slot print_timing: id 0 | task 1400 | total time = 9361.91 ms / 319 tokens 1.53.691.995 I slot print_timing: id 0 | task 1400 | graphs reused = 1510 1.53.692.027 I slot release: id 0 | task 1400 | stop processing: n_tokens = 318, truncated = 0 1.53.692.040 I srv update_slots: all slots are idle 1.53.763.825 I srv operator(): operator(): cleaning up before exit...