| 0.00.365.021 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg) |
| 0.00.365.026 I device_info: |
| 0.00.745.434 I - CUDA0 : NVIDIA GeForce RTX 3080 (20052 MiB, 19814 MiB free) |
| 0.00.879.113 I - CUDA1 : NVIDIA GeForce RTX 3080 (20052 MiB, 19814 MiB free) |
| 0.01.017.386 I - CUDA2 : NVIDIA GeForce RTX 3080 (20052 MiB, 19814 MiB free) |
| 0.01.017.407 I - CPU : AMD EPYC 7F32 8-Core Processor (193092 MiB, 193092 MiB free) |
| 0.01.017.540 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CUDA : ARCHS = 860 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | |
| 0.01.017.665 I srv init: using 15 threads for HTTP server |
| 0.01.018.168 I srv start: binding port with default address family |
| 0.01.019.525 I srv llama_server: loading model |
| 0.01.019.534 I srv load_model: loading model '/mnt/models/PropellerA-models/Qwen3.6-27B-Q5_K_M.gguf' |
| 0.01.019.618 I common_init_result: fitting params to device memory ... |
| 0.01.019.621 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on) |
| 0.20.765.039 W llama_context: n_ctx_seq (16384) < n_ctx_train (262144) -- the full capacity of the model will not be utilized |
| 0.20.875.663 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable) |
| 0.20.998.765 I srv load_model: initializing slots, n_slots = 1 |
| 0.21.041.608 W srv load_model: speculative decoding will use checkpoints |
| 0.21.041.627 W common_speculative_init: no implementations specified for speculative decoding |
| 0.21.041.628 I slot load_model: id 0 | task -1 | new slot, n_ctx = 16384 |
| 0.21.041.728 I srv load_model: prompt cache is enabled, size limit: 8192 MiB |
| 0.21.041.732 I srv load_model: use `--cache-ram 0` to disable the prompt cache |
| 0.21.041.732 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391 |
| 0.21.041.732 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256 |
| 0.21.041.762 I srv init: idle slots will be saved to prompt cache upon starting a new task |
| 0.21.062.284 I init: chat template, example_format: '<|im_start|>system |
| You are a helpful assistant<|im_end|> |
| <|im_start|>user |
| Hello<|im_end|> |
| <|im_start|>assistant |
| Hi there<|im_end|> |
| <|im_start|>user |
| How are you?<|im_end|> |
| <|im_start|>assistant |
| <think> |
| ' |
| 0.21.077.775 I srv init: init: chat template, thinking = 1 |
| 0.21.078.129 I srv llama_server: model loaded |
| 0.21.078.138 I srv llama_server: server is listening on http://127.0.0.1:8291 |
| 0.21.078.145 I srv update_slots: all slots are idle |
| 0.22.237.927 I srv params_from_: Chat format: peg-native |
| 0.22.238.197 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = -1 |
| 0.22.238.201 I srv get_availabl: updating prompt cache |
| 0.22.238.209 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000 |
| 0.22.238.215 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 16384 tokens, 8589934592 est) |
| 0.22.238.216 I srv get_availabl: prompt cache update took 0.01 ms |
| 0.22.238.285 I slot launch_slot_: id 0 | task 0 | processing task, is_child = 0 |
| 0.22.700.386 I slot create_check: id 0 | task 0 | created context checkpoint 1 of 32 (pos_min = 62, pos_max = 62, n_tokens = 63, size = 149.626 MiB) |
| 0.22.963.877 I slot print_timing: id 0 | task 0 | prompt eval time = 522.55 ms / 67 tokens ( 7.80 ms per token, 128.22 tokens per second) |
| 0.22.963.883 I slot print_timing: id 0 | task 0 | eval time = 203.02 ms / 7 tokens ( 29.00 ms per token, 34.48 tokens per second) |
| 0.22.963.884 I slot print_timing: id 0 | task 0 | total time = 725.56 ms / 74 tokens |
| 0.22.963.891 I slot print_timing: id 0 | task 0 | graphs reused = 6 |
| 0.22.963.946 I slot release: id 0 | task 0 | stop processing: n_tokens = 73, truncated = 0 |
| 0.22.963.952 I srv update_slots: all slots are idle |
| 0.22.989.652 I srv params_from_: Chat format: peg-native |
| 0.22.989.841 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.238 (> 0.100 thold), f_keep = 0.205 |
| 0.22.989.843 I srv get_availabl: updating prompt cache |
| 0.22.989.879 W srv prompt_save: - saving prompt with length 73, total state size = 152.052 MiB (draft: 0.000 MiB) |
| 0.23.188.172 I srv load: - looking for better prompt, base f_keep = 0.205, sim = 0.238 |
| 0.23.188.181 I srv update: - cache state: 1 prompts, 301.678 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.23.188.182 I srv update: - prompt 0x58812c6d7be0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.23.188.183 I srv get_availabl: prompt cache update took 198.34 ms |
| 0.23.188.299 I slot launch_slot_: id 0 | task 9 | processing task, is_child = 0 |
| 0.23.188.311 W slot update_slots: id 0 | task 9 | erased invalidated context checkpoint (pos_min = 62, pos_max = 62, n_tokens = 63, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.23.309.399 I slot create_check: id 0 | task 9 | created context checkpoint 1 of 32 (pos_min = 58, pos_max = 58, n_tokens = 59, size = 149.626 MiB) |
| 0.23.802.902 I slot print_timing: id 0 | task 9 | prompt eval time = 173.16 ms / 63 tokens ( 2.75 ms per token, 363.83 tokens per second) |
| 0.23.802.907 I slot print_timing: id 0 | task 9 | eval time = 441.42 ms / 9 tokens ( 49.05 ms per token, 20.39 tokens per second) |
| 0.23.802.908 I slot print_timing: id 0 | task 9 | total time = 614.58 ms / 72 tokens |
| 0.23.802.909 I slot print_timing: id 0 | task 9 | graphs reused = 13 |
| 0.23.802.939 I slot release: id 0 | task 9 | stop processing: n_tokens = 71, truncated = 0 |
| 0.23.802.952 I srv update_slots: all slots are idle |
| 0.23.837.324 I srv params_from_: Chat format: peg-native |
| 0.23.837.581 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.227 (> 0.100 thold), f_keep = 0.211 |
| 0.23.837.584 I srv get_availabl: updating prompt cache |
| 0.23.837.618 W srv prompt_save: - saving prompt with length 71, total state size = 151.985 MiB (draft: 0.000 MiB) |
| 0.24.049.652 I srv load: - looking for better prompt, base f_keep = 0.211, sim = 0.227 |
| 0.24.049.667 I srv update: - cache state: 2 prompts, 603.289 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.24.049.668 I srv update: - prompt 0x58812c6d7be0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.24.049.668 I srv update: - prompt 0x5881220e9d80: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.24.049.670 I srv get_availabl: prompt cache update took 212.09 ms |
| 0.24.049.772 I slot launch_slot_: id 0 | task 20 | processing task, is_child = 0 |
| 0.24.049.784 W slot update_slots: id 0 | task 20 | erased invalidated context checkpoint (pos_min = 58, pos_max = 58, n_tokens = 59, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.24.170.262 I slot create_check: id 0 | task 20 | created context checkpoint 1 of 32 (pos_min = 61, pos_max = 61, n_tokens = 62, size = 149.626 MiB) |
| 0.24.713.725 I slot print_timing: id 0 | task 20 | prompt eval time = 172.38 ms / 66 tokens ( 2.61 ms per token, 382.88 tokens per second) |
| 0.24.713.732 I slot print_timing: id 0 | task 20 | eval time = 491.56 ms / 10 tokens ( 49.16 ms per token, 20.34 tokens per second) |
| 0.24.713.732 I slot print_timing: id 0 | task 20 | total time = 663.93 ms / 76 tokens |
| 0.24.713.733 I slot print_timing: id 0 | task 20 | graphs reused = 21 |
| 0.24.713.764 I slot release: id 0 | task 20 | stop processing: n_tokens = 75, truncated = 0 |
| 0.24.713.775 I srv update_slots: all slots are idle |
| 0.24.746.896 I srv params_from_: Chat format: peg-native |
| 0.24.747.155 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17796796355 |
| 0.24.747.161 I srv get_availabl: updating prompt cache |
| 0.24.747.238 W srv prompt_save: - saving prompt with length 75, total state size = 152.118 MiB (draft: 0.000 MiB) |
| 0.24.953.835 I srv load: - looking for better prompt, base f_keep = 0.040, sim = 0.036 |
| 0.24.953.847 I srv update: - cache state: 3 prompts, 905.034 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.24.953.848 I srv update: - prompt 0x58812c6d7be0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.24.953.849 I srv update: - prompt 0x5881220e9d80: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.24.953.850 I srv update: - prompt 0x588121da99b0: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.24.953.852 I srv get_availabl: prompt cache update took 206.69 ms |
| 0.24.953.963 I slot launch_slot_: id 0 | task 32 | processing task, is_child = 0 |
| 0.24.953.975 W slot update_slots: id 0 | task 32 | erased invalidated context checkpoint (pos_min = 61, pos_max = 61, n_tokens = 62, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.25.133.982 I slot create_check: id 0 | task 32 | created context checkpoint 1 of 32 (pos_min = 78, pos_max = 78, n_tokens = 79, size = 149.626 MiB) |
| 0.25.905.675 I slot print_timing: id 0 | task 32 | prompt eval time = 231.90 ms / 83 tokens ( 2.79 ms per token, 357.91 tokens per second) |
| 0.25.905.681 I slot print_timing: id 0 | task 32 | eval time = 719.79 ms / 18 tokens ( 39.99 ms per token, 25.01 tokens per second) |
| 0.25.905.682 I slot print_timing: id 0 | task 32 | total time = 951.69 ms / 101 tokens |
| 0.25.905.683 I slot print_timing: id 0 | task 32 | graphs reused = 37 |
| 0.25.905.712 I slot release: id 0 | task 32 | stop processing: n_tokens = 100, truncated = 0 |
| 0.25.905.723 I srv update_slots: all slots are idle |
| 0.25.938.744 I srv params_from_: Chat format: peg-native |
| 0.25.938.988 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17797988303 |
| 0.25.938.993 I srv get_availabl: updating prompt cache |
| 0.25.939.034 W srv prompt_save: - saving prompt with length 100, total state size = 152.949 MiB (draft: 0.000 MiB) |
| 0.26.133.618 I srv load: - looking for better prompt, base f_keep = 0.030, sim = 0.046 |
| 0.26.133.632 I srv update: - cache state: 4 prompts, 1207.608 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.26.133.633 I srv update: - prompt 0x58812c6d7be0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.26.133.634 I srv update: - prompt 0x5881220e9d80: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.26.133.635 I srv update: - prompt 0x588121da99b0: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.26.133.635 I srv update: - prompt 0x58812e27ae50: 100 tokens, checkpoints: 1, 302.575 MiB |
| 0.26.133.637 I srv get_availabl: prompt cache update took 194.64 ms |
| 0.26.133.741 I slot launch_slot_: id 0 | task 52 | processing task, is_child = 0 |
| 0.26.133.753 W slot update_slots: id 0 | task 52 | erased invalidated context checkpoint (pos_min = 78, pos_max = 78, n_tokens = 79, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.26.256.868 I slot create_check: id 0 | task 52 | created context checkpoint 1 of 32 (pos_min = 60, pos_max = 60, n_tokens = 61, size = 149.626 MiB) |
| 0.26.830.015 I slot print_timing: id 0 | task 52 | prompt eval time = 174.99 ms / 65 tokens ( 2.69 ms per token, 371.45 tokens per second) |
| 0.26.830.021 I slot print_timing: id 0 | task 52 | eval time = 521.26 ms / 12 tokens ( 43.44 ms per token, 23.02 tokens per second) |
| 0.26.830.022 I slot print_timing: id 0 | task 52 | total time = 696.25 ms / 77 tokens |
| 0.26.830.023 I slot print_timing: id 0 | task 52 | graphs reused = 47 |
| 0.26.830.048 I slot release: id 0 | task 52 | stop processing: n_tokens = 76, truncated = 0 |
| 0.26.830.059 I srv update_slots: all slots are idle |
| 0.26.865.703 I srv params_from_: Chat format: peg-native |
| 0.26.865.958 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17798912639 |
| 0.26.865.960 I srv get_availabl: updating prompt cache |
| 0.26.865.987 W srv prompt_save: - saving prompt with length 76, total state size = 152.151 MiB (draft: 0.000 MiB) |
| 0.27.058.170 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.039 |
| 0.27.058.182 I srv load: - found better prompt with f_keep = 0.340, sim = 0.442 |
| 0.27.106.866 I srv update: - cache state: 4 prompts, 1206.811 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.27.106.873 I srv update: - prompt 0x58812c6d7be0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.27.106.874 I srv update: - prompt 0x5881220e9d80: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.27.106.875 I srv update: - prompt 0x588121da99b0: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.27.106.875 I srv update: - prompt 0x58812277eb80: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.27.106.877 I srv get_availabl: prompt cache update took 240.92 ms |
| 0.27.106.993 I slot launch_slot_: id 0 | task 66 | processing task, is_child = 0 |
| 0.27.107.006 W slot update_slots: id 0 | task 66 | erased invalidated context checkpoint (pos_min = 78, pos_max = 78, n_tokens = 79, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.27.233.471 I slot create_check: id 0 | task 66 | created context checkpoint 1 of 32 (pos_min = 72, pos_max = 72, n_tokens = 73, size = 149.626 MiB) |
| 0.28.138.497 I slot print_timing: id 0 | task 66 | prompt eval time = 178.36 ms / 77 tokens ( 2.32 ms per token, 431.70 tokens per second) |
| 0.28.138.503 I slot print_timing: id 0 | task 66 | eval time = 853.12 ms / 23 tokens ( 37.09 ms per token, 26.96 tokens per second) |
| 0.28.138.504 I slot print_timing: id 0 | task 66 | total time = 1031.48 ms / 100 tokens |
| 0.28.138.505 I slot print_timing: id 0 | task 66 | graphs reused = 68 |
| 0.28.138.531 I slot release: id 0 | task 66 | stop processing: n_tokens = 99, truncated = 0 |
| 0.28.138.542 I srv update_slots: all slots are idle |
| 0.28.172.442 I srv params_from_: Chat format: peg-native |
| 0.28.172.710 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.393 (> 0.100 thold), f_keep = 0.354 |
| 0.28.172.714 I srv get_availabl: updating prompt cache |
| 0.28.172.764 W srv prompt_save: - saving prompt with length 99, total state size = 152.916 MiB (draft: 0.000 MiB) |
| 0.28.364.086 I srv load: - looking for better prompt, base f_keep = 0.354, sim = 0.393 |
| 0.28.364.101 I srv update: - cache state: 5 prompts, 1509.353 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.28.364.102 I srv update: - prompt 0x58812c6d7be0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.28.364.103 I srv update: - prompt 0x5881220e9d80: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.28.364.104 I srv update: - prompt 0x588121da99b0: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.28.364.104 I srv update: - prompt 0x58812277eb80: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.28.364.105 I srv update: - prompt 0x5881220ec6d0: 99 tokens, checkpoints: 1, 302.542 MiB |
| 0.28.364.106 I srv get_availabl: prompt cache update took 191.39 ms |
| 0.28.364.217 I slot launch_slot_: id 0 | task 91 | processing task, is_child = 0 |
| 0.28.364.231 W slot update_slots: id 0 | task 91 | erased invalidated context checkpoint (pos_min = 72, pos_max = 72, n_tokens = 73, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.28.709.281 I slot create_check: id 0 | task 91 | created context checkpoint 1 of 32 (pos_min = 84, pos_max = 84, n_tokens = 85, size = 149.626 MiB) |
| 0.29.343.408 I slot print_timing: id 0 | task 91 | prompt eval time = 398.16 ms / 89 tokens ( 4.47 ms per token, 223.53 tokens per second) |
| 0.29.343.419 I slot print_timing: id 0 | task 91 | eval time = 581.01 ms / 19 tokens ( 30.58 ms per token, 32.70 tokens per second) |
| 0.29.343.420 I slot print_timing: id 0 | task 91 | total time = 979.17 ms / 108 tokens |
| 0.29.343.422 I slot print_timing: id 0 | task 91 | graphs reused = 85 |
| 0.29.343.450 I slot release: id 0 | task 91 | stop processing: n_tokens = 107, truncated = 0 |
| 0.29.343.461 I srv update_slots: all slots are idle |
| 0.29.377.061 I srv params_from_: Chat format: peg-native |
| 0.29.377.301 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17801426042 |
| 0.29.377.304 I srv get_availabl: updating prompt cache |
| 0.29.377.332 W srv prompt_save: - saving prompt with length 107, total state size = 153.181 MiB (draft: 0.000 MiB) |
| 0.29.749.969 I srv load: - looking for better prompt, base f_keep = 0.028, sim = 0.050 |
| 0.29.749.984 I srv update: - cache state: 6 prompts, 1812.160 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.29.749.985 I srv update: - prompt 0x58812c6d7be0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.29.749.987 I srv update: - prompt 0x5881220e9d80: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.29.749.989 I srv update: - prompt 0x588121da99b0: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.29.749.990 I srv update: - prompt 0x58812277eb80: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.29.749.991 I srv update: - prompt 0x5881220ec6d0: 99 tokens, checkpoints: 1, 302.542 MiB |
| 0.29.749.991 I srv update: - prompt 0x588122a333d0: 107 tokens, checkpoints: 1, 302.807 MiB |
| 0.29.749.993 I srv get_availabl: prompt cache update took 372.69 ms |
| 0.29.750.100 I slot launch_slot_: id 0 | task 112 | processing task, is_child = 0 |
| 0.29.750.112 W slot update_slots: id 0 | task 112 | erased invalidated context checkpoint (pos_min = 84, pos_max = 84, n_tokens = 85, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.29.866.956 I slot create_check: id 0 | task 112 | created context checkpoint 1 of 32 (pos_min = 55, pos_max = 55, n_tokens = 56, size = 149.626 MiB) |
| 0.30.180.627 I slot print_timing: id 0 | task 112 | prompt eval time = 168.76 ms / 60 tokens ( 2.81 ms per token, 355.53 tokens per second) |
| 0.30.180.634 I slot print_timing: id 0 | task 112 | eval time = 261.74 ms / 9 tokens ( 29.08 ms per token, 34.38 tokens per second) |
| 0.30.180.635 I slot print_timing: id 0 | task 112 | total time = 430.51 ms / 69 tokens |
| 0.30.180.636 I slot print_timing: id 0 | task 112 | graphs reused = 92 |
| 0.30.180.661 I slot release: id 0 | task 112 | stop processing: n_tokens = 68, truncated = 0 |
| 0.30.180.673 I srv update_slots: all slots are idle |
| 0.30.215.175 I srv params_from_: Chat format: peg-native |
| 0.30.215.478 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.183 (> 0.100 thold), f_keep = 0.221 |
| 0.30.215.479 I srv get_availabl: updating prompt cache |
| 0.30.215.536 W srv prompt_save: - saving prompt with length 68, total state size = 151.886 MiB (draft: 0.000 MiB) |
| 0.30.595.473 I srv load: - looking for better prompt, base f_keep = 0.221, sim = 0.183 |
| 0.30.595.486 I srv update: - cache state: 7 prompts, 2113.672 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.30.595.487 I srv update: - prompt 0x58812c6d7be0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.30.595.488 I srv update: - prompt 0x5881220e9d80: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.30.595.489 I srv update: - prompt 0x588121da99b0: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.30.595.490 I srv update: - prompt 0x58812277eb80: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.30.595.490 I srv update: - prompt 0x5881220ec6d0: 99 tokens, checkpoints: 1, 302.542 MiB |
| 0.30.595.491 I srv update: - prompt 0x588122a333d0: 107 tokens, checkpoints: 1, 302.807 MiB |
| 0.30.595.493 I srv update: - prompt 0x5881220f0e90: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.30.595.494 I srv get_availabl: prompt cache update took 380.01 ms |
| 0.30.595.598 I slot launch_slot_: id 0 | task 123 | processing task, is_child = 0 |
| 0.30.595.611 W slot update_slots: id 0 | task 123 | erased invalidated context checkpoint (pos_min = 55, pos_max = 55, n_tokens = 56, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.30.723.059 I slot create_check: id 0 | task 123 | created context checkpoint 1 of 32 (pos_min = 77, pos_max = 77, n_tokens = 78, size = 149.626 MiB) |
| 0.31.260.070 I slot print_timing: id 0 | task 123 | prompt eval time = 179.34 ms / 82 tokens ( 2.19 ms per token, 457.22 tokens per second) |
| 0.31.260.077 I slot print_timing: id 0 | task 123 | eval time = 485.11 ms / 16 tokens ( 30.32 ms per token, 32.98 tokens per second) |
| 0.31.260.078 I slot print_timing: id 0 | task 123 | total time = 664.45 ms / 98 tokens |
| 0.31.260.079 I slot print_timing: id 0 | task 123 | graphs reused = 106 |
| 0.31.260.108 I slot release: id 0 | task 123 | stop processing: n_tokens = 97, truncated = 0 |
| 0.31.260.120 I srv update_slots: all slots are idle |
| 0.31.297.303 I srv params_from_: Chat format: peg-native |
| 0.31.297.565 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17803342699 |
| 0.31.297.568 I srv get_availabl: updating prompt cache |
| 0.31.297.595 W srv prompt_save: - saving prompt with length 97, total state size = 152.849 MiB (draft: 0.000 MiB) |
| 0.31.664.716 I srv load: - looking for better prompt, base f_keep = 0.031, sim = 0.033 |
| 0.31.664.727 I srv load: - found better prompt with f_keep = 0.343, sim = 0.374 |
| 0.31.711.838 I srv update: - cache state: 7 prompts, 2113.605 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.31.711.847 I srv update: - prompt 0x58812c6d7be0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.31.711.847 I srv update: - prompt 0x5881220e9d80: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.31.711.848 I srv update: - prompt 0x588121da99b0: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.31.711.849 I srv update: - prompt 0x58812277eb80: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.31.711.849 I srv update: - prompt 0x588122a333d0: 107 tokens, checkpoints: 1, 302.807 MiB |
| 0.31.711.850 I srv update: - prompt 0x5881220f0e90: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.31.711.851 I srv update: - prompt 0x58812c6d7ea0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.31.711.852 I srv get_availabl: prompt cache update took 414.28 ms |
| 0.31.711.964 I slot launch_slot_: id 0 | task 141 | processing task, is_child = 0 |
| 0.31.711.977 W slot update_slots: id 0 | task 141 | erased invalidated context checkpoint (pos_min = 72, pos_max = 72, n_tokens = 73, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.31.853.651 I slot create_check: id 0 | task 141 | created context checkpoint 1 of 32 (pos_min = 86, pos_max = 86, n_tokens = 87, size = 149.626 MiB) |
| 0.32.906.701 I slot print_timing: id 0 | task 141 | prompt eval time = 193.84 ms / 91 tokens ( 2.13 ms per token, 469.46 tokens per second) |
| 0.32.906.707 I slot print_timing: id 0 | task 141 | eval time = 1000.88 ms / 27 tokens ( 37.07 ms per token, 26.98 tokens per second) |
| 0.32.906.708 I slot print_timing: id 0 | task 141 | total time = 1194.72 ms / 118 tokens |
| 0.32.906.709 I slot print_timing: id 0 | task 141 | graphs reused = 131 |
| 0.32.906.739 I slot release: id 0 | task 141 | stop processing: n_tokens = 117, truncated = 0 |
| 0.32.906.751 I srv update_slots: all slots are idle |
| 0.32.940.248 I srv params_from_: Chat format: peg-native |
| 0.32.940.484 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17804989331 |
| 0.32.940.487 I srv get_availabl: updating prompt cache |
| 0.32.940.516 W srv prompt_save: - saving prompt with length 117, total state size = 153.514 MiB (draft: 0.000 MiB) |
| 0.33.136.295 I srv load: - looking for better prompt, base f_keep = 0.026, sim = 0.024 |
| 0.33.136.306 I srv update: - cache state: 8 prompts, 2416.745 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.33.136.307 I srv update: - prompt 0x58812c6d7be0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.33.136.308 I srv update: - prompt 0x5881220e9d80: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.33.136.309 I srv update: - prompt 0x588121da99b0: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.33.136.309 I srv update: - prompt 0x58812277eb80: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.33.136.310 I srv update: - prompt 0x588122a333d0: 107 tokens, checkpoints: 1, 302.807 MiB |
| 0.33.136.311 I srv update: - prompt 0x5881220f0e90: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.33.136.312 I srv update: - prompt 0x58812c6d7ea0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.33.136.312 I srv update: - prompt 0x5881220ea940: 117 tokens, checkpoints: 1, 303.140 MiB |
| 0.33.136.314 I srv get_availabl: prompt cache update took 195.83 ms |
| 0.33.136.429 I slot launch_slot_: id 0 | task 170 | processing task, is_child = 0 |
| 0.33.136.443 W slot update_slots: id 0 | task 170 | erased invalidated context checkpoint (pos_min = 86, pos_max = 86, n_tokens = 87, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.33.341.592 I slot create_check: id 0 | task 170 | created context checkpoint 1 of 32 (pos_min = 119, pos_max = 119, n_tokens = 120, size = 149.626 MiB) |
| 0.37.194.134 I slot print_timing: id 0 | task 170 | n_decoded = 100, tg = 27.42 t/s |
| 0.40.088.448 I slot print_timing: id 0 | task 170 | prompt eval time = 410.88 ms / 124 tokens ( 3.31 ms per token, 301.79 tokens per second) |
| 0.40.088.454 I slot print_timing: id 0 | task 170 | eval time = 6541.12 ms / 173 tokens ( 37.81 ms per token, 26.45 tokens per second) |
| 0.40.088.455 I slot print_timing: id 0 | task 170 | total time = 6952.00 ms / 297 tokens |
| 0.40.088.456 I slot print_timing: id 0 | task 170 | graphs reused = 301 |
| 0.40.088.489 I slot release: id 0 | task 170 | stop processing: n_tokens = 296, truncated = 0 |
| 0.40.088.501 I srv update_slots: all slots are idle |
| 0.40.139.676 I srv params_from_: Chat format: peg-native |
| 0.40.139.905 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.129 (> 0.100 thold), f_keep = 0.041 |
| 0.40.139.908 I srv get_availabl: updating prompt cache |
| 0.40.139.943 W srv prompt_save: - saving prompt with length 296, total state size = 159.460 MiB (draft: 0.000 MiB) |
| 0.40.338.176 I srv load: - looking for better prompt, base f_keep = 0.041, sim = 0.129 |
| 0.40.338.187 I srv update: - cache state: 9 prompts, 2725.831 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.40.338.189 I srv update: - prompt 0x58812c6d7be0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.40.338.190 I srv update: - prompt 0x5881220e9d80: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.40.338.190 I srv update: - prompt 0x588121da99b0: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.40.338.191 I srv update: - prompt 0x58812277eb80: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.40.338.192 I srv update: - prompt 0x588122a333d0: 107 tokens, checkpoints: 1, 302.807 MiB |
| 0.40.338.192 I srv update: - prompt 0x5881220f0e90: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.40.338.193 I srv update: - prompt 0x58812c6d7ea0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.40.338.194 I srv update: - prompt 0x5881220ea940: 117 tokens, checkpoints: 1, 303.140 MiB |
| 0.40.338.194 I srv update: - prompt 0x5881226f3ca0: 296 tokens, checkpoints: 1, 309.086 MiB |
| 0.40.338.196 I srv get_availabl: prompt cache update took 198.29 ms |
| 0.40.338.303 I slot launch_slot_: id 0 | task 345 | processing task, is_child = 0 |
| 0.40.338.314 W slot update_slots: id 0 | task 345 | erased invalidated context checkpoint (pos_min = 119, pos_max = 119, n_tokens = 120, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.40.627.378 I slot create_check: id 0 | task 345 | created context checkpoint 1 of 32 (pos_min = 88, pos_max = 88, n_tokens = 89, size = 149.626 MiB) |
| 0.43.946.319 I slot print_timing: id 0 | task 345 | prompt eval time = 341.14 ms / 93 tokens ( 3.67 ms per token, 272.61 tokens per second) |
| 0.43.946.325 I slot print_timing: id 0 | task 345 | eval time = 3266.85 ms / 88 tokens ( 37.12 ms per token, 26.94 tokens per second) |
| 0.43.946.326 I slot print_timing: id 0 | task 345 | total time = 3608.00 ms / 181 tokens |
| 0.43.946.327 I slot print_timing: id 0 | task 345 | graphs reused = 387 |
| 0.43.946.358 I slot release: id 0 | task 345 | stop processing: n_tokens = 180, truncated = 0 |
| 0.43.946.371 I srv update_slots: all slots are idle |
| 0.43.998.024 I srv params_from_: Chat format: peg-native |
| 0.43.998.258 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.138 (> 0.100 thold), f_keep = 0.067 |
| 0.43.998.261 I srv get_availabl: updating prompt cache |
| 0.43.998.292 W srv prompt_save: - saving prompt with length 180, total state size = 155.607 MiB (draft: 0.000 MiB) |
| 0.44.193.733 I srv load: - looking for better prompt, base f_keep = 0.067, sim = 0.138 |
| 0.44.193.746 I srv update: - cache state: 10 prompts, 3031.064 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.44.193.748 I srv update: - prompt 0x58812c6d7be0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.44.193.748 I srv update: - prompt 0x5881220e9d80: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.44.193.749 I srv update: - prompt 0x588121da99b0: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.44.193.749 I srv update: - prompt 0x58812277eb80: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.44.193.750 I srv update: - prompt 0x588122a333d0: 107 tokens, checkpoints: 1, 302.807 MiB |
| 0.44.193.751 I srv update: - prompt 0x5881220f0e90: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.44.193.751 I srv update: - prompt 0x58812c6d7ea0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.44.193.752 I srv update: - prompt 0x5881220ea940: 117 tokens, checkpoints: 1, 303.140 MiB |
| 0.44.193.752 I srv update: - prompt 0x5881226f3ca0: 296 tokens, checkpoints: 1, 309.086 MiB |
| 0.44.193.753 I srv update: - prompt 0x58812e27cbd0: 180 tokens, checkpoints: 1, 305.233 MiB |
| 0.44.193.754 I srv get_availabl: prompt cache update took 195.49 ms |
| 0.44.193.862 I slot launch_slot_: id 0 | task 435 | processing task, is_child = 0 |
| 0.44.193.876 W slot update_slots: id 0 | task 435 | erased invalidated context checkpoint (pos_min = 88, pos_max = 88, n_tokens = 89, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.44.330.599 I slot create_check: id 0 | task 435 | created context checkpoint 1 of 32 (pos_min = 82, pos_max = 82, n_tokens = 83, size = 149.626 MiB) |
| 0.48.233.652 I slot print_timing: id 0 | task 435 | n_decoded = 100, tg = 25.97 t/s |
| 0.51.235.959 I slot print_timing: id 0 | task 435 | n_decoded = 177, tg = 25.83 t/s |
| 0.53.194.755 I slot print_timing: id 0 | task 435 | prompt eval time = 188.85 ms / 87 tokens ( 2.17 ms per token, 460.69 tokens per second) |
| 0.53.194.760 I slot print_timing: id 0 | task 435 | eval time = 8812.03 ms / 225 tokens ( 39.16 ms per token, 25.53 tokens per second) |
| 0.53.194.761 I slot print_timing: id 0 | task 435 | total time = 9000.88 ms / 312 tokens |
| 0.53.194.762 I slot print_timing: id 0 | task 435 | graphs reused = 609 |
| 0.53.194.791 I slot release: id 0 | task 435 | stop processing: n_tokens = 311, truncated = 0 |
| 0.53.194.804 I srv update_slots: all slots are idle |
| 0.53.237.827 I srv params_from_: Chat format: peg-native |
| 0.53.237.993 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.039 |
| 0.53.237.996 I srv get_availabl: updating prompt cache |
| 0.53.238.050 W srv prompt_save: - saving prompt with length 311, total state size = 159.959 MiB (draft: 0.000 MiB) |
| 0.53.609.818 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.107 |
| 0.53.609.832 I srv update: - cache state: 11 prompts, 3340.649 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.53.609.833 I srv update: - prompt 0x58812c6d7be0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.53.609.834 I srv update: - prompt 0x5881220e9d80: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.53.609.837 I srv update: - prompt 0x588121da99b0: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.53.609.839 I srv update: - prompt 0x58812277eb80: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.53.609.839 I srv update: - prompt 0x588122a333d0: 107 tokens, checkpoints: 1, 302.807 MiB |
| 0.53.609.840 I srv update: - prompt 0x5881220f0e90: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.53.609.840 I srv update: - prompt 0x58812c6d7ea0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.53.609.841 I srv update: - prompt 0x5881220ea940: 117 tokens, checkpoints: 1, 303.140 MiB |
| 0.53.609.841 I srv update: - prompt 0x5881226f3ca0: 296 tokens, checkpoints: 1, 309.086 MiB |
| 0.53.609.842 I srv update: - prompt 0x58812e27cbd0: 180 tokens, checkpoints: 1, 305.233 MiB |
| 0.53.609.843 I srv update: - prompt 0x58812e27f170: 311 tokens, checkpoints: 1, 309.585 MiB |
| 0.53.609.844 I srv get_availabl: prompt cache update took 371.85 ms |
| 0.53.609.954 I slot launch_slot_: id 0 | task 662 | processing task, is_child = 0 |
| 0.53.609.967 W slot update_slots: id 0 | task 662 | erased invalidated context checkpoint (pos_min = 82, pos_max = 82, n_tokens = 83, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.53.805.187 I slot create_check: id 0 | task 662 | created context checkpoint 1 of 32 (pos_min = 107, pos_max = 107, n_tokens = 108, size = 149.626 MiB) |
| 0.56.874.720 I slot print_timing: id 0 | task 662 | prompt eval time = 247.37 ms / 112 tokens ( 2.21 ms per token, 452.77 tokens per second) |
| 0.56.874.725 I slot print_timing: id 0 | task 662 | eval time = 3017.38 ms / 80 tokens ( 37.72 ms per token, 26.51 tokens per second) |
| 0.56.874.726 I slot print_timing: id 0 | task 662 | total time = 3264.75 ms / 192 tokens |
| 0.56.874.727 I slot print_timing: id 0 | task 662 | graphs reused = 687 |
| 0.56.874.757 I slot release: id 0 | task 662 | stop processing: n_tokens = 191, truncated = 0 |
| 0.56.874.770 I srv update_slots: all slots are idle |
| 0.56.926.037 I srv params_from_: Chat format: peg-native |
| 0.56.927.474 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.136 (> 0.100 thold), f_keep = 0.063 |
| 0.56.927.478 I srv get_availabl: updating prompt cache |
| 0.56.927.513 W srv prompt_save: - saving prompt with length 191, total state size = 155.972 MiB (draft: 0.000 MiB) |
| 0.57.122.022 I srv load: - looking for better prompt, base f_keep = 0.063, sim = 0.136 |
| 0.57.122.036 I srv update: - cache state: 12 prompts, 3646.247 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.57.122.037 I srv update: - prompt 0x58812c6d7be0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.57.122.038 I srv update: - prompt 0x5881220e9d80: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.57.122.039 I srv update: - prompt 0x588121da99b0: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.57.122.040 I srv update: - prompt 0x58812277eb80: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.57.122.040 I srv update: - prompt 0x588122a333d0: 107 tokens, checkpoints: 1, 302.807 MiB |
| 0.57.122.041 I srv update: - prompt 0x5881220f0e90: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.57.122.042 I srv update: - prompt 0x58812c6d7ea0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.57.122.042 I srv update: - prompt 0x5881220ea940: 117 tokens, checkpoints: 1, 303.140 MiB |
| 0.57.122.043 I srv update: - prompt 0x5881226f3ca0: 296 tokens, checkpoints: 1, 309.086 MiB |
| 0.57.122.044 I srv update: - prompt 0x58812e27cbd0: 180 tokens, checkpoints: 1, 305.233 MiB |
| 0.57.122.044 I srv update: - prompt 0x58812e27f170: 311 tokens, checkpoints: 1, 309.585 MiB |
| 0.57.122.045 I srv update: - prompt 0x588121c43570: 191 tokens, checkpoints: 1, 305.598 MiB |
| 0.57.122.046 I srv get_availabl: prompt cache update took 194.57 ms |
| 0.57.122.154 I slot launch_slot_: id 0 | task 744 | processing task, is_child = 0 |
| 0.57.122.165 W slot update_slots: id 0 | task 744 | erased invalidated context checkpoint (pos_min = 107, pos_max = 107, n_tokens = 108, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.57.259.170 I slot create_check: id 0 | task 744 | created context checkpoint 1 of 32 (pos_min = 83, pos_max = 83, n_tokens = 84, size = 149.626 MiB) |
| 1.00.874.689 I slot print_timing: id 0 | task 744 | prompt eval time = 189.12 ms / 88 tokens ( 2.15 ms per token, 465.33 tokens per second) |
| 1.00.874.696 I slot print_timing: id 0 | task 744 | eval time = 3563.40 ms / 91 tokens ( 39.16 ms per token, 25.54 tokens per second) |
| 1.00.874.696 I slot print_timing: id 0 | task 744 | total time = 3752.52 ms / 179 tokens |
| 1.00.874.697 I slot print_timing: id 0 | task 744 | graphs reused = 776 |
| 1.00.874.729 I slot release: id 0 | task 744 | stop processing: n_tokens = 178, truncated = 0 |
| 1.00.874.740 I srv update_slots: all slots are idle |
| 1.00.916.513 I srv params_from_: Chat format: peg-native |
| 1.00.916.689 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.114 (> 0.100 thold), f_keep = 0.067 |
| 1.00.916.692 I srv get_availabl: updating prompt cache |
| 1.00.916.724 W srv prompt_save: - saving prompt with length 178, total state size = 155.540 MiB (draft: 0.000 MiB) |
| 1.01.111.698 I srv load: - looking for better prompt, base f_keep = 0.067, sim = 0.114 |
| 1.01.111.713 I srv update: - cache state: 13 prompts, 3951.413 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.01.111.714 I srv update: - prompt 0x58812c6d7be0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.01.111.715 I srv update: - prompt 0x5881220e9d80: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.01.111.716 I srv update: - prompt 0x588121da99b0: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.01.111.717 I srv update: - prompt 0x58812277eb80: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.01.111.718 I srv update: - prompt 0x588122a333d0: 107 tokens, checkpoints: 1, 302.807 MiB |
| 1.01.111.718 I srv update: - prompt 0x5881220f0e90: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.01.111.719 I srv update: - prompt 0x58812c6d7ea0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.01.111.720 I srv update: - prompt 0x5881220ea940: 117 tokens, checkpoints: 1, 303.140 MiB |
| 1.01.111.720 I srv update: - prompt 0x5881226f3ca0: 296 tokens, checkpoints: 1, 309.086 MiB |
| 1.01.111.721 I srv update: - prompt 0x58812e27cbd0: 180 tokens, checkpoints: 1, 305.233 MiB |
| 1.01.111.722 I srv update: - prompt 0x58812e27f170: 311 tokens, checkpoints: 1, 309.585 MiB |
| 1.01.111.723 I srv update: - prompt 0x588121c43570: 191 tokens, checkpoints: 1, 305.598 MiB |
| 1.01.111.725 I srv update: - prompt 0x588121f392c0: 178 tokens, checkpoints: 1, 305.166 MiB |
| 1.01.111.727 I srv get_availabl: prompt cache update took 195.03 ms |
| 1.01.111.838 I slot launch_slot_: id 0 | task 837 | processing task, is_child = 0 |
| 1.01.111.850 W slot update_slots: id 0 | task 837 | erased invalidated context checkpoint (pos_min = 83, pos_max = 83, n_tokens = 84, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.01.263.433 I slot create_check: id 0 | task 837 | created context checkpoint 1 of 32 (pos_min = 100, pos_max = 100, n_tokens = 101, size = 149.626 MiB) |
| 1.05.137.666 I slot print_timing: id 0 | task 837 | n_decoded = 100, tg = 26.16 t/s |
| 1.08.141.914 I slot print_timing: id 0 | task 837 | n_decoded = 178, tg = 26.08 t/s |
| 1.09.002.680 I slot print_timing: id 0 | task 837 | prompt eval time = 203.77 ms / 105 tokens ( 1.94 ms per token, 515.28 tokens per second) |
| 1.09.002.686 I slot print_timing: id 0 | task 837 | eval time = 7687.05 ms / 200 tokens ( 38.44 ms per token, 26.02 tokens per second) |
| 1.09.002.687 I slot print_timing: id 0 | task 837 | total time = 7890.83 ms / 305 tokens |
| 1.09.002.687 I slot print_timing: id 0 | task 837 | graphs reused = 973 |
| 1.09.002.716 I slot release: id 0 | task 837 | stop processing: n_tokens = 304, truncated = 0 |
| 1.09.002.726 I srv update_slots: all slots are idle |
| 1.09.054.728 I srv params_from_: Chat format: peg-native |
| 1.09.054.971 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.133 (> 0.100 thold), f_keep = 0.039 |
| 1.09.054.974 I srv get_availabl: updating prompt cache |
| 1.09.055.020 W srv prompt_save: - saving prompt with length 304, total state size = 159.726 MiB (draft: 0.000 MiB) |
| 1.09.255.125 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.133 |
| 1.09.255.140 I srv update: - cache state: 14 prompts, 4260.765 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.09.255.142 I srv update: - prompt 0x58812c6d7be0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.09.255.142 I srv update: - prompt 0x5881220e9d80: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.09.255.143 I srv update: - prompt 0x588121da99b0: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.09.255.144 I srv update: - prompt 0x58812277eb80: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.09.255.144 I srv update: - prompt 0x588122a333d0: 107 tokens, checkpoints: 1, 302.807 MiB |
| 1.09.255.145 I srv update: - prompt 0x5881220f0e90: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.09.255.146 I srv update: - prompt 0x58812c6d7ea0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.09.255.146 I srv update: - prompt 0x5881220ea940: 117 tokens, checkpoints: 1, 303.140 MiB |
| 1.09.255.147 I srv update: - prompt 0x5881226f3ca0: 296 tokens, checkpoints: 1, 309.086 MiB |
| 1.09.255.147 I srv update: - prompt 0x58812e27cbd0: 180 tokens, checkpoints: 1, 305.233 MiB |
| 1.09.255.148 I srv update: - prompt 0x58812e27f170: 311 tokens, checkpoints: 1, 309.585 MiB |
| 1.09.255.148 I srv update: - prompt 0x588121c43570: 191 tokens, checkpoints: 1, 305.598 MiB |
| 1.09.255.149 I srv update: - prompt 0x588121f392c0: 178 tokens, checkpoints: 1, 305.166 MiB |
| 1.09.255.149 I srv update: - prompt 0x588121fc8ea0: 304 tokens, checkpoints: 1, 309.352 MiB |
| 1.09.255.151 I srv get_availabl: prompt cache update took 200.18 ms |
| 1.09.255.254 I slot launch_slot_: id 0 | task 1039 | processing task, is_child = 0 |
| 1.09.255.266 W slot update_slots: id 0 | task 1039 | erased invalidated context checkpoint (pos_min = 100, pos_max = 100, n_tokens = 101, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.09.394.070 I slot create_check: id 0 | task 1039 | created context checkpoint 1 of 32 (pos_min = 85, pos_max = 85, n_tokens = 86, size = 149.626 MiB) |
| 1.13.546.789 I slot print_timing: id 0 | task 1039 | n_decoded = 100, tg = 25.44 t/s |
| 1.14.747.617 I slot print_timing: id 0 | task 1039 | prompt eval time = 361.16 ms / 90 tokens ( 4.01 ms per token, 249.19 tokens per second) |
| 1.14.747.623 I slot print_timing: id 0 | task 1039 | eval time = 5131.18 ms / 132 tokens ( 38.87 ms per token, 25.73 tokens per second) |
| 1.14.747.624 I slot print_timing: id 0 | task 1039 | total time = 5492.34 ms / 222 tokens |
| 1.14.747.625 I slot print_timing: id 0 | task 1039 | graphs reused = 1103 |
| 1.14.747.654 I slot release: id 0 | task 1039 | stop processing: n_tokens = 221, truncated = 0 |
| 1.14.747.666 I srv update_slots: all slots are idle |
| 1.14.805.843 I srv params_from_: Chat format: peg-native |
| 1.14.806.088 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17846830246 |
| 1.14.806.092 I srv get_availabl: updating prompt cache |
| 1.14.806.154 W srv prompt_save: - saving prompt with length 221, total state size = 156.969 MiB (draft: 0.000 MiB) |
| 1.15.006.839 I srv load: - looking for better prompt, base f_keep = 0.036, sim = 0.060 |
| 1.15.006.854 I srv update: - cache state: 15 prompts, 4567.360 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.15.006.855 I srv update: - prompt 0x58812c6d7be0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.15.006.856 I srv update: - prompt 0x5881220e9d80: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.15.006.856 I srv update: - prompt 0x588121da99b0: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.15.006.857 I srv update: - prompt 0x58812277eb80: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.15.006.858 I srv update: - prompt 0x588122a333d0: 107 tokens, checkpoints: 1, 302.807 MiB |
| 1.15.006.859 I srv update: - prompt 0x5881220f0e90: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.15.006.859 I srv update: - prompt 0x58812c6d7ea0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.15.006.860 I srv update: - prompt 0x5881220ea940: 117 tokens, checkpoints: 1, 303.140 MiB |
| 1.15.006.860 I srv update: - prompt 0x5881226f3ca0: 296 tokens, checkpoints: 1, 309.086 MiB |
| 1.15.006.861 I srv update: - prompt 0x58812e27cbd0: 180 tokens, checkpoints: 1, 305.233 MiB |
| 1.15.006.861 I srv update: - prompt 0x58812e27f170: 311 tokens, checkpoints: 1, 309.585 MiB |
| 1.15.006.862 I srv update: - prompt 0x588121c43570: 191 tokens, checkpoints: 1, 305.598 MiB |
| 1.15.006.863 I srv update: - prompt 0x588121f392c0: 178 tokens, checkpoints: 1, 305.166 MiB |
| 1.15.006.863 I srv update: - prompt 0x588121fc8ea0: 304 tokens, checkpoints: 1, 309.352 MiB |
| 1.15.006.864 I srv update: - prompt 0x588121a092f0: 221 tokens, checkpoints: 1, 306.595 MiB |
| 1.15.006.865 I srv get_availabl: prompt cache update took 200.77 ms |
| 1.15.006.967 I slot launch_slot_: id 0 | task 1173 | processing task, is_child = 0 |
| 1.15.006.978 W slot update_slots: id 0 | task 1173 | erased invalidated context checkpoint (pos_min = 85, pos_max = 85, n_tokens = 86, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.15.207.288 I slot create_check: id 0 | task 1173 | created context checkpoint 1 of 32 (pos_min = 128, pos_max = 128, n_tokens = 129, size = 149.626 MiB) |
| 1.19.060.360 I slot print_timing: id 0 | task 1173 | n_decoded = 100, tg = 26.31 t/s |
| 1.22.090.578 I slot print_timing: id 0 | task 1173 | n_decoded = 179, tg = 26.20 t/s |
| 1.23.786.759 I slot print_timing: id 0 | task 1173 | prompt eval time = 252.38 ms / 133 tokens ( 1.90 ms per token, 526.98 tokens per second) |
| 1.23.786.764 I slot print_timing: id 0 | task 1173 | eval time = 8527.39 ms / 222 tokens ( 38.41 ms per token, 26.03 tokens per second) |
| 1.23.786.765 I slot print_timing: id 0 | task 1173 | total time = 8779.77 ms / 355 tokens |
| 1.23.786.766 I slot print_timing: id 0 | task 1173 | graphs reused = 1322 |
| 1.23.786.798 I slot release: id 0 | task 1173 | stop processing: n_tokens = 354, truncated = 0 |
| 1.23.786.810 I srv update_slots: all slots are idle |
| 1.23.828.853 I srv params_from_: Chat format: peg-native |
| 1.23.829.022 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17855869389 |
| 1.23.829.025 I srv get_availabl: updating prompt cache |
| 1.23.829.065 W srv prompt_save: - saving prompt with length 354, total state size = 161.387 MiB (draft: 0.000 MiB) |
| 1.24.051.047 I srv load: - looking for better prompt, base f_keep = 0.023, sim = 0.075 |
| 1.24.051.063 I srv update: - cache state: 16 prompts, 4878.373 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.24.051.065 I srv update: - prompt 0x58812c6d7be0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.24.051.065 I srv update: - prompt 0x5881220e9d80: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.24.051.066 I srv update: - prompt 0x588121da99b0: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.24.051.067 I srv update: - prompt 0x58812277eb80: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.24.051.067 I srv update: - prompt 0x588122a333d0: 107 tokens, checkpoints: 1, 302.807 MiB |
| 1.24.051.068 I srv update: - prompt 0x5881220f0e90: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.24.051.068 I srv update: - prompt 0x58812c6d7ea0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.24.051.069 I srv update: - prompt 0x5881220ea940: 117 tokens, checkpoints: 1, 303.140 MiB |
| 1.24.051.070 I srv update: - prompt 0x5881226f3ca0: 296 tokens, checkpoints: 1, 309.086 MiB |
| 1.24.051.070 I srv update: - prompt 0x58812e27cbd0: 180 tokens, checkpoints: 1, 305.233 MiB |
| 1.24.051.071 I srv update: - prompt 0x58812e27f170: 311 tokens, checkpoints: 1, 309.585 MiB |
| 1.24.051.071 I srv update: - prompt 0x588121c43570: 191 tokens, checkpoints: 1, 305.598 MiB |
| 1.24.051.072 I srv update: - prompt 0x588121f392c0: 178 tokens, checkpoints: 1, 305.166 MiB |
| 1.24.051.072 I srv update: - prompt 0x588121fc8ea0: 304 tokens, checkpoints: 1, 309.352 MiB |
| 1.24.051.073 I srv update: - prompt 0x588121a092f0: 221 tokens, checkpoints: 1, 306.595 MiB |
| 1.24.051.073 I srv update: - prompt 0x58812e27b790: 354 tokens, checkpoints: 1, 311.013 MiB |
| 1.24.051.075 I srv get_availabl: prompt cache update took 222.05 ms |
| 1.24.051.175 I slot launch_slot_: id 0 | task 1397 | processing task, is_child = 0 |
| 1.24.051.186 W slot update_slots: id 0 | task 1397 | erased invalidated context checkpoint (pos_min = 128, pos_max = 128, n_tokens = 129, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.24.206.658 I slot create_check: id 0 | task 1397 | created context checkpoint 1 of 32 (pos_min = 101, pos_max = 101, n_tokens = 102, size = 149.626 MiB) |
| 1.26.613.633 I slot print_timing: id 0 | task 1397 | prompt eval time = 209.61 ms / 106 tokens ( 1.98 ms per token, 505.71 tokens per second) |
| 1.26.613.639 I slot print_timing: id 0 | task 1397 | eval time = 2352.83 ms / 56 tokens ( 42.01 ms per token, 23.80 tokens per second) |
| 1.26.613.640 I slot print_timing: id 0 | task 1397 | total time = 2562.44 ms / 162 tokens |
| 1.26.613.641 I slot print_timing: id 0 | task 1397 | graphs reused = 1376 |
| 1.26.613.677 I slot release: id 0 | task 1397 | stop processing: n_tokens = 161, truncated = 0 |
| 1.26.613.689 I srv update_slots: all slots are idle |
| 1.26.665.972 I srv params_from_: Chat format: peg-native |
| 1.26.666.238 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 17858696268 |
| 1.26.666.243 I srv get_availabl: updating prompt cache |
| 1.26.666.313 W srv prompt_save: - saving prompt with length 161, total state size = 154.975 MiB (draft: 0.000 MiB) |
| 1.26.866.809 I srv load: - looking for better prompt, base f_keep = 0.075, sim = 0.094 |
| 1.26.866.823 I srv update: - cache state: 17 prompts, 5182.975 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.26.866.825 I srv update: - prompt 0x58812c6d7be0: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.26.866.826 I srv update: - prompt 0x5881220e9d80: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.26.866.826 I srv update: - prompt 0x588121da99b0: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.26.866.827 I srv update: - prompt 0x58812277eb80: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.26.866.828 I srv update: - prompt 0x588122a333d0: 107 tokens, checkpoints: 1, 302.807 MiB |
| 1.26.866.828 I srv update: - prompt 0x5881220f0e90: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.26.866.829 I srv update: - prompt 0x58812c6d7ea0: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.26.866.830 I srv update: - prompt 0x5881220ea940: 117 tokens, checkpoints: 1, 303.140 MiB |
| 1.26.866.830 I srv update: - prompt 0x5881226f3ca0: 296 tokens, checkpoints: 1, 309.086 MiB |
| 1.26.866.831 I srv update: - prompt 0x58812e27cbd0: 180 tokens, checkpoints: 1, 305.233 MiB |
| 1.26.866.831 I srv update: - prompt 0x58812e27f170: 311 tokens, checkpoints: 1, 309.585 MiB |
| 1.26.866.832 I srv update: - prompt 0x588121c43570: 191 tokens, checkpoints: 1, 305.598 MiB |
| 1.26.866.832 I srv update: - prompt 0x588121f392c0: 178 tokens, checkpoints: 1, 305.166 MiB |
| 1.26.866.833 I srv update: - prompt 0x588121fc8ea0: 304 tokens, checkpoints: 1, 309.352 MiB |
| 1.26.866.833 I srv update: - prompt 0x588121a092f0: 221 tokens, checkpoints: 1, 306.595 MiB |
| 1.26.866.834 I srv update: - prompt 0x58812e27b790: 354 tokens, checkpoints: 1, 311.013 MiB |
| 1.26.866.834 I srv update: - prompt 0x58812e391070: 161 tokens, checkpoints: 1, 304.601 MiB |
| 1.26.866.836 I srv get_availabl: prompt cache update took 200.59 ms |
| 1.26.866.946 I slot launch_slot_: id 0 | task 1455 | processing task, is_child = 0 |
| 1.26.866.958 W slot update_slots: id 0 | task 1455 | erased invalidated context checkpoint (pos_min = 101, pos_max = 101, n_tokens = 102, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.27.033.840 I slot create_check: id 0 | task 1455 | created context checkpoint 1 of 32 (pos_min = 122, pos_max = 122, n_tokens = 123, size = 149.626 MiB) |
| 1.30.907.736 I slot print_timing: id 0 | task 1455 | n_decoded = 100, tg = 26.17 t/s |
| 1.33.909.139 I slot print_timing: id 0 | task 1455 | n_decoded = 179, tg = 26.23 t/s |
| 1.34.325.262 I slot print_timing: id 0 | task 1455 | prompt eval time = 219.07 ms / 127 tokens ( 1.72 ms per token, 579.72 tokens per second) |
| 1.34.325.267 I slot print_timing: id 0 | task 1455 | eval time = 7239.23 ms / 192 tokens ( 37.70 ms per token, 26.52 tokens per second) |
| 1.34.325.268 I slot print_timing: id 0 | task 1455 | total time = 7458.30 ms / 319 tokens |
| 1.34.325.269 I slot print_timing: id 0 | task 1455 | graphs reused = 1565 |
| 1.34.325.303 I slot release: id 0 | task 1455 | stop processing: n_tokens = 318, truncated = 0 |
| 1.34.325.315 I srv update_slots: all slots are idle |
| 1.34.575.654 I srv operator(): operator(): cleaning up before exit... |
|
|
