| 0.00.508.759 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg) |
| 0.00.508.764 I device_info: |
| 0.00.634.857 I - CUDA0 : NVIDIA GeForce RTX 3080 (20052 MiB, 19814 MiB free) |
| 0.00.761.526 I - CUDA1 : NVIDIA GeForce RTX 3080 (20052 MiB, 19814 MiB free) |
| 0.00.904.217 I - CUDA2 : NVIDIA GeForce RTX 3080 (20052 MiB, 19814 MiB free) |
| 0.00.904.239 I - CPU : AMD EPYC 7F32 8-Core Processor (193092 MiB, 193092 MiB free) |
| 0.00.904.364 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CUDA : ARCHS = 860 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | |
| 0.00.904.481 I srv init: using 15 threads for HTTP server |
| 0.00.904.970 I srv start: binding port with default address family |
| 0.00.906.270 I srv llama_server: loading model |
| 0.00.906.278 I srv load_model: loading model '/mnt/models/modelle/qwen36-27b-uncensored/Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-Q6_K_P.gguf' |
| 0.00.906.384 I common_init_result: fitting params to device memory ... |
| 0.00.906.387 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on) |
| 0.24.282.573 W llama_context: n_ctx_seq (16384) < n_ctx_train (262144) -- the full capacity of the model will not be utilized |
| 0.24.648.533 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable) |
| 0.24.757.230 I srv load_model: initializing slots, n_slots = 1 |
| 0.24.803.431 W srv load_model: speculative decoding will use checkpoints |
| 0.24.803.451 W common_speculative_init: no implementations specified for speculative decoding |
| 0.24.803.453 I slot load_model: id 0 | task -1 | new slot, n_ctx = 16384 |
| 0.24.803.548 I srv load_model: prompt cache is enabled, size limit: 8192 MiB |
| 0.24.803.551 I srv load_model: use `--cache-ram 0` to disable the prompt cache |
| 0.24.803.551 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391 |
| 0.24.803.552 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256 |
| 0.24.803.581 I srv init: idle slots will be saved to prompt cache upon starting a new task |
| 0.24.824.411 I init: chat template, example_format: '<|im_start|>system |
| You are a helpful assistant<|im_end|> |
| <|im_start|>user |
| Hello<|im_end|> |
| <|im_start|>assistant |
| Hi there<|im_end|> |
| <|im_start|>user |
| How are you?<|im_end|> |
| <|im_start|>assistant |
| <think> |
| ' |
| 0.24.838.984 I srv init: init: chat template, thinking = 1 |
| 0.24.839.035 I srv llama_server: model loaded |
| 0.24.839.041 I srv llama_server: server is listening on http://127.0.0.1:8291 |
| 0.24.839.049 I srv update_slots: all slots are idle |
| 0.26.259.583 I srv params_from_: Chat format: peg-native |
| 0.26.260.254 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = -1 |
| 0.26.260.260 I srv get_availabl: updating prompt cache |
| 0.26.260.274 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000 |
| 0.26.260.286 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 16384 tokens, 8589934592 est) |
| 0.26.260.290 I srv get_availabl: prompt cache update took 0.03 ms |
| 0.26.260.455 I slot launch_slot_: id 0 | task 0 | processing task, is_child = 0 |
| 0.26.612.180 I slot create_check: id 0 | task 0 | created context checkpoint 1 of 32 (pos_min = 62, pos_max = 62, n_tokens = 63, size = 149.626 MiB) |
| 0.26.899.854 I slot print_timing: id 0 | task 0 | prompt eval time = 406.49 ms / 67 tokens ( 6.07 ms per token, 164.83 tokens per second) |
| 0.26.899.860 I slot print_timing: id 0 | task 0 | eval time = 232.88 ms / 7 tokens ( 33.27 ms per token, 30.06 tokens per second) |
| 0.26.899.861 I slot print_timing: id 0 | task 0 | total time = 639.36 ms / 74 tokens |
| 0.26.899.868 I slot print_timing: id 0 | task 0 | graphs reused = 6 |
| 0.26.899.921 I slot release: id 0 | task 0 | stop processing: n_tokens = 73, truncated = 0 |
| 0.26.899.928 I srv update_slots: all slots are idle |
| 0.26.936.283 I srv params_from_: Chat format: peg-native |
| 0.26.936.520 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.238 (> 0.100 thold), f_keep = 0.205 |
| 0.26.936.523 I srv get_availabl: updating prompt cache |
| 0.26.936.557 W srv prompt_save: - saving prompt with length 73, total state size = 152.052 MiB (draft: 0.000 MiB) |
| 0.27.119.549 I srv load: - looking for better prompt, base f_keep = 0.205, sim = 0.238 |
| 0.27.119.561 I srv update: - cache state: 1 prompts, 301.678 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.27.119.561 I srv update: - prompt 0x5dccfc719990: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.27.119.563 I srv get_availabl: prompt cache update took 183.04 ms |
| 0.27.119.970 I slot launch_slot_: id 0 | task 9 | processing task, is_child = 0 |
| 0.27.119.984 W slot update_slots: id 0 | task 9 | erased invalidated context checkpoint (pos_min = 62, pos_max = 62, n_tokens = 63, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.27.239.142 I slot create_check: id 0 | task 9 | created context checkpoint 1 of 32 (pos_min = 58, pos_max = 58, n_tokens = 59, size = 149.626 MiB) |
| 0.27.826.446 I slot print_timing: id 0 | task 9 | prompt eval time = 170.54 ms / 63 tokens ( 2.71 ms per token, 369.40 tokens per second) |
| 0.27.826.452 I slot print_timing: id 0 | task 9 | eval time = 535.91 ms / 9 tokens ( 59.55 ms per token, 16.79 tokens per second) |
| 0.27.826.453 I slot print_timing: id 0 | task 9 | total time = 706.45 ms / 72 tokens |
| 0.27.826.454 I slot print_timing: id 0 | task 9 | graphs reused = 13 |
| 0.27.826.479 I slot release: id 0 | task 9 | stop processing: n_tokens = 71, truncated = 0 |
| 0.27.826.490 I srv update_slots: all slots are idle |
| 0.27.857.588 I srv params_from_: Chat format: peg-native |
| 0.27.857.870 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.227 (> 0.100 thold), f_keep = 0.211 |
| 0.27.857.872 I srv get_availabl: updating prompt cache |
| 0.27.857.902 W srv prompt_save: - saving prompt with length 71, total state size = 151.985 MiB (draft: 0.000 MiB) |
| 0.28.040.174 I srv load: - looking for better prompt, base f_keep = 0.211, sim = 0.227 |
| 0.28.040.188 I srv update: - cache state: 2 prompts, 603.289 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.28.040.189 I srv update: - prompt 0x5dccfc719990: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.28.040.190 I srv update: - prompt 0x5dccfc266bc0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.28.040.191 I srv get_availabl: prompt cache update took 182.32 ms |
| 0.28.040.290 I slot launch_slot_: id 0 | task 20 | processing task, is_child = 0 |
| 0.28.040.303 W slot update_slots: id 0 | task 20 | erased invalidated context checkpoint (pos_min = 58, pos_max = 58, n_tokens = 59, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.28.160.531 I slot create_check: id 0 | task 20 | created context checkpoint 1 of 32 (pos_min = 61, pos_max = 61, n_tokens = 62, size = 149.626 MiB) |
| 0.28.728.198 I slot print_timing: id 0 | task 20 | prompt eval time = 171.63 ms / 66 tokens ( 2.60 ms per token, 384.55 tokens per second) |
| 0.28.728.205 I slot print_timing: id 0 | task 20 | eval time = 516.26 ms / 10 tokens ( 51.63 ms per token, 19.37 tokens per second) |
| 0.28.728.205 I slot print_timing: id 0 | task 20 | total time = 687.89 ms / 76 tokens |
| 0.28.728.206 I slot print_timing: id 0 | task 20 | graphs reused = 21 |
| 0.28.728.231 I slot release: id 0 | task 20 | stop processing: n_tokens = 75, truncated = 0 |
| 0.28.728.243 I srv update_slots: all slots are idle |
| 0.28.760.648 I srv params_from_: Chat format: peg-native |
| 0.28.760.913 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 18025894504 |
| 0.28.760.916 I srv get_availabl: updating prompt cache |
| 0.28.760.977 W srv prompt_save: - saving prompt with length 75, total state size = 152.118 MiB (draft: 0.000 MiB) |
| 0.28.944.820 I srv load: - looking for better prompt, base f_keep = 0.040, sim = 0.036 |
| 0.28.944.832 I srv update: - cache state: 3 prompts, 905.034 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.28.944.833 I srv update: - prompt 0x5dccfc719990: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.28.944.834 I srv update: - prompt 0x5dccfc266bc0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.28.944.835 I srv update: - prompt 0x5dccfb9e1b90: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.28.944.836 I srv get_availabl: prompt cache update took 183.92 ms |
| 0.28.944.939 I slot launch_slot_: id 0 | task 32 | processing task, is_child = 0 |
| 0.28.944.950 W slot update_slots: id 0 | task 32 | erased invalidated context checkpoint (pos_min = 61, pos_max = 61, n_tokens = 62, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.29.126.938 I slot create_check: id 0 | task 32 | created context checkpoint 1 of 32 (pos_min = 78, pos_max = 78, n_tokens = 79, size = 149.626 MiB) |
| 0.30.062.942 I slot print_timing: id 0 | task 32 | prompt eval time = 233.36 ms / 83 tokens ( 2.81 ms per token, 355.67 tokens per second) |
| 0.30.062.948 I slot print_timing: id 0 | task 32 | eval time = 884.62 ms / 18 tokens ( 49.15 ms per token, 20.35 tokens per second) |
| 0.30.062.949 I slot print_timing: id 0 | task 32 | total time = 1117.98 ms / 101 tokens |
| 0.30.062.950 I slot print_timing: id 0 | task 32 | graphs reused = 37 |
| 0.30.062.977 I slot release: id 0 | task 32 | stop processing: n_tokens = 100, truncated = 0 |
| 0.30.062.988 I srv update_slots: all slots are idle |
| 0.30.094.392 I srv params_from_: Chat format: peg-native |
| 0.30.094.622 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 18027229249 |
| 0.30.094.625 I srv get_availabl: updating prompt cache |
| 0.30.094.652 W srv prompt_save: - saving prompt with length 100, total state size = 152.949 MiB (draft: 0.000 MiB) |
| 0.30.299.586 I srv load: - looking for better prompt, base f_keep = 0.030, sim = 0.046 |
| 0.30.299.596 I srv update: - cache state: 4 prompts, 1207.608 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.30.299.598 I srv update: - prompt 0x5dccfc719990: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.30.299.599 I srv update: - prompt 0x5dccfc266bc0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.30.299.599 I srv update: - prompt 0x5dccfb9e1b90: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.30.299.600 I srv update: - prompt 0x5dccff98ed10: 100 tokens, checkpoints: 1, 302.575 MiB |
| 0.30.299.601 I srv get_availabl: prompt cache update took 204.97 ms |
| 0.30.299.702 I slot launch_slot_: id 0 | task 52 | processing task, is_child = 0 |
| 0.30.299.714 W slot update_slots: id 0 | task 52 | erased invalidated context checkpoint (pos_min = 78, pos_max = 78, n_tokens = 79, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.30.675.549 I slot create_check: id 0 | task 52 | created context checkpoint 1 of 32 (pos_min = 60, pos_max = 60, n_tokens = 61, size = 149.626 MiB) |
| 0.31.141.961 I slot print_timing: id 0 | task 52 | prompt eval time = 427.26 ms / 65 tokens ( 6.57 ms per token, 152.13 tokens per second) |
| 0.31.141.967 I slot print_timing: id 0 | task 52 | eval time = 414.98 ms / 12 tokens ( 34.58 ms per token, 28.92 tokens per second) |
| 0.31.141.968 I slot print_timing: id 0 | task 52 | total time = 842.24 ms / 77 tokens |
| 0.31.141.969 I slot print_timing: id 0 | task 52 | graphs reused = 47 |
| 0.31.141.994 I slot release: id 0 | task 52 | stop processing: n_tokens = 76, truncated = 0 |
| 0.31.142.004 I srv update_slots: all slots are idle |
| 0.31.175.490 I srv params_from_: Chat format: peg-native |
| 0.31.175.743 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 18028308266 |
| 0.31.175.746 I srv get_availabl: updating prompt cache |
| 0.31.175.777 W srv prompt_save: - saving prompt with length 76, total state size = 152.151 MiB (draft: 0.000 MiB) |
| 0.31.543.999 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.039 |
| 0.31.544.006 I srv load: - found better prompt with f_keep = 0.340, sim = 0.442 |
| 0.31.591.495 I srv update: - cache state: 4 prompts, 1206.811 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.31.591.500 I srv update: - prompt 0x5dccfc719990: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.31.591.501 I srv update: - prompt 0x5dccfc266bc0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.31.591.501 I srv update: - prompt 0x5dccfb9e1b90: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.31.591.502 I srv update: - prompt 0x5dccfc86f6e0: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.31.591.504 I srv get_availabl: prompt cache update took 415.76 ms |
| 0.31.591.608 I slot launch_slot_: id 0 | task 66 | processing task, is_child = 0 |
| 0.31.591.623 W slot update_slots: id 0 | task 66 | erased invalidated context checkpoint (pos_min = 78, pos_max = 78, n_tokens = 79, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.31.726.565 I slot create_check: id 0 | task 66 | created context checkpoint 1 of 32 (pos_min = 72, pos_max = 72, n_tokens = 73, size = 149.626 MiB) |
| 0.32.800.786 I slot print_timing: id 0 | task 66 | prompt eval time = 186.35 ms / 77 tokens ( 2.42 ms per token, 413.19 tokens per second) |
| 0.32.800.793 I slot print_timing: id 0 | task 66 | eval time = 1022.80 ms / 24 tokens ( 42.62 ms per token, 23.46 tokens per second) |
| 0.32.800.794 I slot print_timing: id 0 | task 66 | total time = 1209.16 ms / 101 tokens |
| 0.32.800.795 I slot print_timing: id 0 | task 66 | graphs reused = 69 |
| 0.32.800.820 I slot release: id 0 | task 66 | stop processing: n_tokens = 100, truncated = 0 |
| 0.32.800.830 I srv update_slots: all slots are idle |
| 0.32.837.351 I srv params_from_: Chat format: peg-native |
| 0.32.838.745 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.393 (> 0.100 thold), f_keep = 0.350 |
| 0.32.838.748 I srv get_availabl: updating prompt cache |
| 0.32.838.774 W srv prompt_save: - saving prompt with length 100, total state size = 152.949 MiB (draft: 0.000 MiB) |
| 0.33.053.153 I srv load: - looking for better prompt, base f_keep = 0.350, sim = 0.393 |
| 0.33.053.167 I srv update: - cache state: 5 prompts, 1509.386 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.33.053.168 I srv update: - prompt 0x5dccfc719990: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.33.053.169 I srv update: - prompt 0x5dccfc266bc0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.33.053.170 I srv update: - prompt 0x5dccfb9e1b90: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.33.053.170 I srv update: - prompt 0x5dccfc86f6e0: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.33.053.171 I srv update: - prompt 0x5dccfbca49d0: 100 tokens, checkpoints: 1, 302.575 MiB |
| 0.33.053.172 I srv get_availabl: prompt cache update took 214.42 ms |
| 0.33.053.276 I slot launch_slot_: id 0 | task 92 | processing task, is_child = 0 |
| 0.33.053.288 W slot update_slots: id 0 | task 92 | erased invalidated context checkpoint (pos_min = 72, pos_max = 72, n_tokens = 73, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.33.245.791 I slot create_check: id 0 | task 92 | created context checkpoint 1 of 32 (pos_min = 84, pos_max = 84, n_tokens = 85, size = 149.626 MiB) |
| 0.34.208.386 I slot print_timing: id 0 | task 92 | prompt eval time = 245.07 ms / 89 tokens ( 2.75 ms per token, 363.16 tokens per second) |
| 0.34.208.395 I slot print_timing: id 0 | task 92 | eval time = 910.02 ms / 19 tokens ( 47.90 ms per token, 20.88 tokens per second) |
| 0.34.208.396 I slot print_timing: id 0 | task 92 | total time = 1155.09 ms / 108 tokens |
| 0.34.208.397 I slot print_timing: id 0 | task 92 | graphs reused = 86 |
| 0.34.208.424 I slot release: id 0 | task 92 | stop processing: n_tokens = 107, truncated = 0 |
| 0.34.208.434 I srv update_slots: all slots are idle |
| 0.34.240.580 I srv params_from_: Chat format: peg-native |
| 0.34.240.835 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 18031374696 |
| 0.34.240.837 I srv get_availabl: updating prompt cache |
| 0.34.240.864 W srv prompt_save: - saving prompt with length 107, total state size = 153.181 MiB (draft: 0.000 MiB) |
| 0.34.613.564 I srv load: - looking for better prompt, base f_keep = 0.028, sim = 0.050 |
| 0.34.613.578 I srv update: - cache state: 6 prompts, 1812.193 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.34.613.579 I srv update: - prompt 0x5dccfc719990: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.34.613.580 I srv update: - prompt 0x5dccfc266bc0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.34.613.580 I srv update: - prompt 0x5dccfb9e1b90: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.34.613.581 I srv update: - prompt 0x5dccfc86f6e0: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.34.613.582 I srv update: - prompt 0x5dccfbca49d0: 100 tokens, checkpoints: 1, 302.575 MiB |
| 0.34.613.582 I srv update: - prompt 0x5dccfbc14560: 107 tokens, checkpoints: 1, 302.807 MiB |
| 0.34.613.584 I srv get_availabl: prompt cache update took 372.75 ms |
| 0.34.613.685 I slot launch_slot_: id 0 | task 113 | processing task, is_child = 0 |
| 0.34.613.699 W slot update_slots: id 0 | task 113 | erased invalidated context checkpoint (pos_min = 84, pos_max = 84, n_tokens = 85, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.34.740.228 I slot create_check: id 0 | task 113 | created context checkpoint 1 of 32 (pos_min = 55, pos_max = 55, n_tokens = 56, size = 149.626 MiB) |
| 0.35.094.998 I slot print_timing: id 0 | task 113 | prompt eval time = 177.99 ms / 60 tokens ( 2.97 ms per token, 337.10 tokens per second) |
| 0.35.095.005 I slot print_timing: id 0 | task 113 | eval time = 303.30 ms / 9 tokens ( 33.70 ms per token, 29.67 tokens per second) |
| 0.35.095.006 I slot print_timing: id 0 | task 113 | total time = 481.29 ms / 69 tokens |
| 0.35.095.007 I slot print_timing: id 0 | task 113 | graphs reused = 93 |
| 0.35.095.031 I slot release: id 0 | task 113 | stop processing: n_tokens = 68, truncated = 0 |
| 0.35.095.042 I srv update_slots: all slots are idle |
| 0.35.131.493 I srv params_from_: Chat format: peg-native |
| 0.35.131.758 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.183 (> 0.100 thold), f_keep = 0.221 |
| 0.35.131.762 I srv get_availabl: updating prompt cache |
| 0.35.131.792 W srv prompt_save: - saving prompt with length 68, total state size = 151.886 MiB (draft: 0.000 MiB) |
| 0.35.528.979 I srv load: - looking for better prompt, base f_keep = 0.221, sim = 0.183 |
| 0.35.528.992 I srv update: - cache state: 7 prompts, 2113.705 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.35.528.993 I srv update: - prompt 0x5dccfc719990: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.35.528.994 I srv update: - prompt 0x5dccfc266bc0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.35.528.994 I srv update: - prompt 0x5dccfb9e1b90: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.35.528.995 I srv update: - prompt 0x5dccfc86f6e0: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.35.528.996 I srv update: - prompt 0x5dccfbca49d0: 100 tokens, checkpoints: 1, 302.575 MiB |
| 0.35.528.996 I srv update: - prompt 0x5dccfbc14560: 107 tokens, checkpoints: 1, 302.807 MiB |
| 0.35.528.997 I srv update: - prompt 0x5dccff995dd0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.35.528.998 I srv get_availabl: prompt cache update took 397.24 ms |
| 0.35.529.098 I slot launch_slot_: id 0 | task 124 | processing task, is_child = 0 |
| 0.35.529.110 W slot update_slots: id 0 | task 124 | erased invalidated context checkpoint (pos_min = 55, pos_max = 55, n_tokens = 56, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.35.668.410 I slot create_check: id 0 | task 124 | created context checkpoint 1 of 32 (pos_min = 77, pos_max = 77, n_tokens = 78, size = 149.626 MiB) |
| 0.36.302.010 I slot print_timing: id 0 | task 124 | prompt eval time = 190.86 ms / 82 tokens ( 2.33 ms per token, 429.64 tokens per second) |
| 0.36.302.016 I slot print_timing: id 0 | task 124 | eval time = 582.03 ms / 16 tokens ( 36.38 ms per token, 27.49 tokens per second) |
| 0.36.302.017 I slot print_timing: id 0 | task 124 | total time = 772.89 ms / 98 tokens |
| 0.36.302.018 I slot print_timing: id 0 | task 124 | graphs reused = 107 |
| 0.36.302.047 I slot release: id 0 | task 124 | stop processing: n_tokens = 97, truncated = 0 |
| 0.36.302.058 I srv update_slots: all slots are idle |
| 0.36.509.774 I srv params_from_: Chat format: peg-native |
| 0.36.509.954 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 18033468319 |
| 0.36.509.958 I srv get_availabl: updating prompt cache |
| 0.36.510.002 W srv prompt_save: - saving prompt with length 97, total state size = 152.849 MiB (draft: 0.000 MiB) |
| 0.36.703.084 I srv load: - looking for better prompt, base f_keep = 0.031, sim = 0.033 |
| 0.36.703.096 I srv load: - found better prompt with f_keep = 0.340, sim = 0.374 |
| 0.36.754.101 I srv update: - cache state: 7 prompts, 2113.605 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.36.754.107 I srv update: - prompt 0x5dccfc719990: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.36.754.108 I srv update: - prompt 0x5dccfc266bc0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.36.754.109 I srv update: - prompt 0x5dccfb9e1b90: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.36.754.109 I srv update: - prompt 0x5dccfc86f6e0: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.36.754.110 I srv update: - prompt 0x5dccfbc14560: 107 tokens, checkpoints: 1, 302.807 MiB |
| 0.36.754.111 I srv update: - prompt 0x5dccff995dd0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.36.754.111 I srv update: - prompt 0x5dccfc682b30: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.36.754.113 I srv get_availabl: prompt cache update took 244.15 ms |
| 0.36.754.227 I slot launch_slot_: id 0 | task 142 | processing task, is_child = 0 |
| 0.36.754.241 W slot update_slots: id 0 | task 142 | erased invalidated context checkpoint (pos_min = 72, pos_max = 72, n_tokens = 73, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.36.903.852 I slot create_check: id 0 | task 142 | created context checkpoint 1 of 32 (pos_min = 86, pos_max = 86, n_tokens = 87, size = 149.626 MiB) |
| 0.38.096.335 I slot print_timing: id 0 | task 142 | prompt eval time = 201.07 ms / 91 tokens ( 2.21 ms per token, 452.59 tokens per second) |
| 0.38.096.341 I slot print_timing: id 0 | task 142 | eval time = 1141.02 ms / 27 tokens ( 42.26 ms per token, 23.66 tokens per second) |
| 0.38.096.342 I slot print_timing: id 0 | task 142 | total time = 1342.09 ms / 118 tokens |
| 0.38.096.343 I slot print_timing: id 0 | task 142 | graphs reused = 132 |
| 0.38.096.371 I slot release: id 0 | task 142 | stop processing: n_tokens = 117, truncated = 0 |
| 0.38.096.382 I srv update_slots: all slots are idle |
| 0.38.128.958 I srv params_from_: Chat format: peg-native |
| 0.38.129.191 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 18035262643 |
| 0.38.129.194 I srv get_availabl: updating prompt cache |
| 0.38.129.220 W srv prompt_save: - saving prompt with length 117, total state size = 153.514 MiB (draft: 0.000 MiB) |
| 0.38.514.157 I srv load: - looking for better prompt, base f_keep = 0.026, sim = 0.024 |
| 0.38.514.173 I srv update: - cache state: 8 prompts, 2416.745 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.38.514.174 I srv update: - prompt 0x5dccfc719990: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.38.514.176 I srv update: - prompt 0x5dccfc266bc0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.38.514.183 I srv update: - prompt 0x5dccfb9e1b90: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.38.514.183 I srv update: - prompt 0x5dccfc86f6e0: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.38.514.184 I srv update: - prompt 0x5dccfbc14560: 107 tokens, checkpoints: 1, 302.807 MiB |
| 0.38.514.185 I srv update: - prompt 0x5dccff995dd0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.38.514.186 I srv update: - prompt 0x5dccfc682b30: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.38.514.187 I srv update: - prompt 0x5dccfc26ace0: 117 tokens, checkpoints: 1, 303.140 MiB |
| 0.38.514.188 I srv get_availabl: prompt cache update took 384.99 ms |
| 0.38.514.308 I slot launch_slot_: id 0 | task 171 | processing task, is_child = 0 |
| 0.38.514.321 W slot update_slots: id 0 | task 171 | erased invalidated context checkpoint (pos_min = 86, pos_max = 86, n_tokens = 87, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.38.723.167 I slot create_check: id 0 | task 171 | created context checkpoint 1 of 32 (pos_min = 119, pos_max = 119, n_tokens = 120, size = 149.626 MiB) |
| 0.43.068.946 I slot print_timing: id 0 | task 171 | n_decoded = 100, tg = 23.29 t/s |
| 0.44.251.553 I slot print_timing: id 0 | task 171 | prompt eval time = 260.26 ms / 124 tokens ( 2.10 ms per token, 476.45 tokens per second) |
| 0.44.251.559 I slot print_timing: id 0 | task 171 | eval time = 5476.97 ms / 128 tokens ( 42.79 ms per token, 23.37 tokens per second) |
| 0.44.251.560 I slot print_timing: id 0 | task 171 | total time = 5737.23 ms / 252 tokens |
| 0.44.251.561 I slot print_timing: id 0 | task 171 | graphs reused = 258 |
| 0.44.251.591 I slot release: id 0 | task 171 | stop processing: n_tokens = 251, truncated = 0 |
| 0.44.251.602 I srv update_slots: all slots are idle |
| 0.44.292.982 I srv params_from_: Chat format: peg-native |
| 0.44.293.166 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.129 (> 0.100 thold), f_keep = 0.048 |
| 0.44.293.169 I srv get_availabl: updating prompt cache |
| 0.44.293.210 W srv prompt_save: - saving prompt with length 251, total state size = 157.965 MiB (draft: 0.000 MiB) |
| 0.44.679.691 I srv load: - looking for better prompt, base f_keep = 0.048, sim = 0.129 |
| 0.44.679.705 I srv update: - cache state: 9 prompts, 2724.336 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.44.679.706 I srv update: - prompt 0x5dccfc719990: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.44.679.707 I srv update: - prompt 0x5dccfc266bc0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.44.679.708 I srv update: - prompt 0x5dccfb9e1b90: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.44.679.709 I srv update: - prompt 0x5dccfc86f6e0: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.44.679.709 I srv update: - prompt 0x5dccfbc14560: 107 tokens, checkpoints: 1, 302.807 MiB |
| 0.44.679.710 I srv update: - prompt 0x5dccff995dd0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.44.679.710 I srv update: - prompt 0x5dccfc682b30: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.44.679.711 I srv update: - prompt 0x5dccfc26ace0: 117 tokens, checkpoints: 1, 303.140 MiB |
| 0.44.679.712 I srv update: - prompt 0x5dccfbca2080: 251 tokens, checkpoints: 1, 307.591 MiB |
| 0.44.679.713 I srv get_availabl: prompt cache update took 386.54 ms |
| 0.44.679.817 I slot launch_slot_: id 0 | task 301 | processing task, is_child = 0 |
| 0.44.679.829 W slot update_slots: id 0 | task 301 | erased invalidated context checkpoint (pos_min = 119, pos_max = 119, n_tokens = 120, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.44.827.424 I slot create_check: id 0 | task 301 | created context checkpoint 1 of 32 (pos_min = 88, pos_max = 88, n_tokens = 89, size = 149.626 MiB) |
| 0.49.114.879 I slot print_timing: id 0 | task 301 | prompt eval time = 199.09 ms / 93 tokens ( 2.14 ms per token, 467.12 tokens per second) |
| 0.49.114.885 I slot print_timing: id 0 | task 301 | eval time = 4235.95 ms / 97 tokens ( 43.67 ms per token, 22.90 tokens per second) |
| 0.49.114.885 I slot print_timing: id 0 | task 301 | total time = 4435.04 ms / 190 tokens |
| 0.49.114.886 I slot print_timing: id 0 | task 301 | graphs reused = 353 |
| 0.49.114.919 I slot release: id 0 | task 301 | stop processing: n_tokens = 189, truncated = 0 |
| 0.49.114.932 I srv update_slots: all slots are idle |
| 0.49.168.237 I srv params_from_: Chat format: peg-native |
| 0.49.168.478 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.138 (> 0.100 thold), f_keep = 0.063 |
| 0.49.168.482 I srv get_availabl: updating prompt cache |
| 0.49.168.535 W srv prompt_save: - saving prompt with length 189, total state size = 155.906 MiB (draft: 0.000 MiB) |
| 0.49.595.846 I srv load: - looking for better prompt, base f_keep = 0.063, sim = 0.138 |
| 0.49.595.860 I srv update: - cache state: 10 prompts, 3029.868 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.49.595.861 I srv update: - prompt 0x5dccfc719990: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.49.595.862 I srv update: - prompt 0x5dccfc266bc0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.49.595.862 I srv update: - prompt 0x5dccfb9e1b90: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.49.595.863 I srv update: - prompt 0x5dccfc86f6e0: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.49.595.863 I srv update: - prompt 0x5dccfbc14560: 107 tokens, checkpoints: 1, 302.807 MiB |
| 0.49.595.864 I srv update: - prompt 0x5dccff995dd0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.49.595.865 I srv update: - prompt 0x5dccfc682b30: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.49.595.865 I srv update: - prompt 0x5dccfc26ace0: 117 tokens, checkpoints: 1, 303.140 MiB |
| 0.49.595.866 I srv update: - prompt 0x5dccfbca2080: 251 tokens, checkpoints: 1, 307.591 MiB |
| 0.49.595.866 I srv update: - prompt 0x5dccffae7560: 189 tokens, checkpoints: 1, 305.532 MiB |
| 0.49.595.867 I srv get_availabl: prompt cache update took 427.38 ms |
| 0.49.595.969 I slot launch_slot_: id 0 | task 400 | processing task, is_child = 0 |
| 0.49.595.983 W slot update_slots: id 0 | task 400 | erased invalidated context checkpoint (pos_min = 88, pos_max = 88, n_tokens = 89, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 0.49.744.965 I slot create_check: id 0 | task 400 | created context checkpoint 1 of 32 (pos_min = 82, pos_max = 82, n_tokens = 83, size = 149.626 MiB) |
| 0.54.100.411 I slot print_timing: id 0 | task 400 | n_decoded = 100, tg = 23.23 t/s |
| 0.57.131.991 I slot print_timing: id 0 | task 400 | n_decoded = 169, tg = 23.04 t/s |
| 0.59.694.188 I slot print_timing: id 0 | task 400 | prompt eval time = 200.56 ms / 87 tokens ( 2.31 ms per token, 433.79 tokens per second) |
| 0.59.694.193 I slot print_timing: id 0 | task 400 | eval time = 9897.64 ms / 225 tokens ( 43.99 ms per token, 22.73 tokens per second) |
| 0.59.694.194 I slot print_timing: id 0 | task 400 | total time = 10098.20 ms / 312 tokens |
| 0.59.694.194 I slot print_timing: id 0 | task 400 | graphs reused = 575 |
| 0.59.694.221 I slot release: id 0 | task 400 | stop processing: n_tokens = 311, truncated = 0 |
| 0.59.694.232 I srv update_slots: all slots are idle |
| 0.59.744.334 I srv params_from_: Chat format: peg-native |
| 0.59.744.560 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.039 |
| 0.59.744.563 I srv get_availabl: updating prompt cache |
| 0.59.744.598 W srv prompt_save: - saving prompt with length 311, total state size = 159.959 MiB (draft: 0.000 MiB) |
| 0.59.981.691 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.107 |
| 0.59.981.708 I srv update: - cache state: 11 prompts, 3339.453 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 0.59.981.709 I srv update: - prompt 0x5dccfc719990: 73 tokens, checkpoints: 1, 301.678 MiB |
| 0.59.981.710 I srv update: - prompt 0x5dccfc266bc0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 0.59.981.711 I srv update: - prompt 0x5dccfb9e1b90: 75 tokens, checkpoints: 1, 301.744 MiB |
| 0.59.981.711 I srv update: - prompt 0x5dccfc86f6e0: 76 tokens, checkpoints: 1, 301.778 MiB |
| 0.59.981.712 I srv update: - prompt 0x5dccfbc14560: 107 tokens, checkpoints: 1, 302.807 MiB |
| 0.59.981.713 I srv update: - prompt 0x5dccff995dd0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 0.59.981.713 I srv update: - prompt 0x5dccfc682b30: 97 tokens, checkpoints: 1, 302.475 MiB |
| 0.59.981.714 I srv update: - prompt 0x5dccfc26ace0: 117 tokens, checkpoints: 1, 303.140 MiB |
| 0.59.981.714 I srv update: - prompt 0x5dccfbca2080: 251 tokens, checkpoints: 1, 307.591 MiB |
| 0.59.981.715 I srv update: - prompt 0x5dccffae7560: 189 tokens, checkpoints: 1, 305.532 MiB |
| 0.59.981.715 I srv update: - prompt 0x5dccfc223650: 311 tokens, checkpoints: 1, 309.585 MiB |
| 0.59.981.717 I srv get_availabl: prompt cache update took 237.15 ms |
| 0.59.981.832 I slot launch_slot_: id 0 | task 627 | processing task, is_child = 0 |
| 0.59.981.846 W slot update_slots: id 0 | task 627 | erased invalidated context checkpoint (pos_min = 82, pos_max = 82, n_tokens = 83, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.00.189.265 I slot create_check: id 0 | task 627 | created context checkpoint 1 of 32 (pos_min = 107, pos_max = 107, n_tokens = 108, size = 149.626 MiB) |
| 1.04.687.084 I slot print_timing: id 0 | task 627 | n_decoded = 100, tg = 22.49 t/s |
| 1.07.694.480 I slot print_timing: id 0 | task 627 | n_decoded = 166, tg = 22.27 t/s |
| 1.08.728.630 I slot print_timing: id 0 | task 627 | prompt eval time = 259.02 ms / 112 tokens ( 2.31 ms per token, 432.40 tokens per second) |
| 1.08.728.636 I slot print_timing: id 0 | task 627 | eval time = 8487.76 ms / 190 tokens ( 44.67 ms per token, 22.39 tokens per second) |
| 1.08.728.636 I slot print_timing: id 0 | task 627 | total time = 8746.78 ms / 302 tokens |
| 1.08.728.637 I slot print_timing: id 0 | task 627 | graphs reused = 762 |
| 1.08.728.671 I slot release: id 0 | task 627 | stop processing: n_tokens = 301, truncated = 0 |
| 1.08.728.684 I srv update_slots: all slots are idle |
| 1.08.780.272 I srv params_from_: Chat format: peg-native |
| 1.08.780.522 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.136 (> 0.100 thold), f_keep = 0.040 |
| 1.08.780.527 I srv get_availabl: updating prompt cache |
| 1.08.780.590 W srv prompt_save: - saving prompt with length 301, total state size = 159.626 MiB (draft: 0.000 MiB) |
| 1.08.984.387 I srv load: - looking for better prompt, base f_keep = 0.040, sim = 0.136 |
| 1.08.984.404 I srv update: - cache state: 12 prompts, 3648.705 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.08.984.405 I srv update: - prompt 0x5dccfc719990: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.08.984.405 I srv update: - prompt 0x5dccfc266bc0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.08.984.406 I srv update: - prompt 0x5dccfb9e1b90: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.08.984.407 I srv update: - prompt 0x5dccfc86f6e0: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.08.984.407 I srv update: - prompt 0x5dccfbc14560: 107 tokens, checkpoints: 1, 302.807 MiB |
| 1.08.984.408 I srv update: - prompt 0x5dccff995dd0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.08.984.408 I srv update: - prompt 0x5dccfc682b30: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.08.984.409 I srv update: - prompt 0x5dccfc26ace0: 117 tokens, checkpoints: 1, 303.140 MiB |
| 1.08.984.409 I srv update: - prompt 0x5dccfbca2080: 251 tokens, checkpoints: 1, 307.591 MiB |
| 1.08.984.410 I srv update: - prompt 0x5dccffae7560: 189 tokens, checkpoints: 1, 305.532 MiB |
| 1.08.984.410 I srv update: - prompt 0x5dccfc223650: 311 tokens, checkpoints: 1, 309.585 MiB |
| 1.08.984.411 I srv update: - prompt 0x5dccfcb2e8e0: 301 tokens, checkpoints: 1, 309.253 MiB |
| 1.08.984.412 I srv get_availabl: prompt cache update took 203.88 ms |
| 1.08.984.519 I slot launch_slot_: id 0 | task 819 | processing task, is_child = 0 |
| 1.08.984.530 W slot update_slots: id 0 | task 819 | erased invalidated context checkpoint (pos_min = 107, pos_max = 107, n_tokens = 108, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.09.131.173 I slot create_check: id 0 | task 819 | created context checkpoint 1 of 32 (pos_min = 83, pos_max = 83, n_tokens = 84, size = 149.626 MiB) |
| 1.12.949.024 I slot print_timing: id 0 | task 819 | prompt eval time = 198.16 ms / 88 tokens ( 2.25 ms per token, 444.08 tokens per second) |
| 1.12.949.030 I slot print_timing: id 0 | task 819 | eval time = 3766.33 ms / 85 tokens ( 44.31 ms per token, 22.57 tokens per second) |
| 1.12.949.030 I slot print_timing: id 0 | task 819 | total time = 3964.49 ms / 173 tokens |
| 1.12.949.031 I slot print_timing: id 0 | task 819 | graphs reused = 845 |
| 1.12.949.061 I slot release: id 0 | task 819 | stop processing: n_tokens = 172, truncated = 0 |
| 1.12.949.072 I srv update_slots: all slots are idle |
| 1.12.999.486 I srv params_from_: Chat format: peg-native |
| 1.12.999.794 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.114 (> 0.100 thold), f_keep = 0.070 |
| 1.12.999.799 I srv get_availabl: updating prompt cache |
| 1.12.999.861 W srv prompt_save: - saving prompt with length 172, total state size = 155.341 MiB (draft: 0.000 MiB) |
| 1.13.197.824 I srv load: - looking for better prompt, base f_keep = 0.070, sim = 0.114 |
| 1.13.197.839 I srv update: - cache state: 13 prompts, 3953.672 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.13.197.840 I srv update: - prompt 0x5dccfc719990: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.13.197.841 I srv update: - prompt 0x5dccfc266bc0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.13.197.842 I srv update: - prompt 0x5dccfb9e1b90: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.13.197.843 I srv update: - prompt 0x5dccfc86f6e0: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.13.197.843 I srv update: - prompt 0x5dccfbc14560: 107 tokens, checkpoints: 1, 302.807 MiB |
| 1.13.197.844 I srv update: - prompt 0x5dccff995dd0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.13.197.844 I srv update: - prompt 0x5dccfc682b30: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.13.197.845 I srv update: - prompt 0x5dccfc26ace0: 117 tokens, checkpoints: 1, 303.140 MiB |
| 1.13.197.845 I srv update: - prompt 0x5dccfbca2080: 251 tokens, checkpoints: 1, 307.591 MiB |
| 1.13.197.846 I srv update: - prompt 0x5dccffae7560: 189 tokens, checkpoints: 1, 305.532 MiB |
| 1.13.197.846 I srv update: - prompt 0x5dccfc223650: 311 tokens, checkpoints: 1, 309.585 MiB |
| 1.13.197.847 I srv update: - prompt 0x5dccfcb2e8e0: 301 tokens, checkpoints: 1, 309.253 MiB |
| 1.13.197.847 I srv update: - prompt 0x5dccfc70f990: 172 tokens, checkpoints: 1, 304.967 MiB |
| 1.13.197.849 I srv get_availabl: prompt cache update took 198.05 ms |
| 1.13.197.953 I slot launch_slot_: id 0 | task 906 | processing task, is_child = 0 |
| 1.13.197.965 W slot update_slots: id 0 | task 906 | erased invalidated context checkpoint (pos_min = 83, pos_max = 83, n_tokens = 84, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.13.524.274 I slot create_check: id 0 | task 906 | created context checkpoint 1 of 32 (pos_min = 100, pos_max = 100, n_tokens = 101, size = 149.626 MiB) |
| 1.17.874.086 I slot print_timing: id 0 | task 906 | n_decoded = 100, tg = 23.27 t/s |
| 1.20.909.215 I slot print_timing: id 0 | task 906 | n_decoded = 169, tg = 23.05 t/s |
| 1.22.024.431 I slot print_timing: id 0 | task 906 | prompt eval time = 377.89 ms / 105 tokens ( 3.60 ms per token, 277.85 tokens per second) |
| 1.22.024.436 I slot print_timing: id 0 | task 906 | eval time = 8448.57 ms / 195 tokens ( 43.33 ms per token, 23.08 tokens per second) |
| 1.22.024.437 I slot print_timing: id 0 | task 906 | total time = 8826.46 ms / 300 tokens |
| 1.22.024.438 I slot print_timing: id 0 | task 906 | graphs reused = 1037 |
| 1.22.024.466 I slot release: id 0 | task 906 | stop processing: n_tokens = 299, truncated = 0 |
| 1.22.024.477 I srv update_slots: all slots are idle |
| 1.22.074.416 I srv params_from_: Chat format: peg-native |
| 1.22.074.652 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.133 (> 0.100 thold), f_keep = 0.040 |
| 1.22.074.655 I srv get_availabl: updating prompt cache |
| 1.22.074.693 W srv prompt_save: - saving prompt with length 299, total state size = 159.560 MiB (draft: 0.000 MiB) |
| 1.22.279.831 I srv load: - looking for better prompt, base f_keep = 0.040, sim = 0.133 |
| 1.22.279.849 I srv update: - cache state: 14 prompts, 4262.858 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.22.279.851 I srv update: - prompt 0x5dccfc719990: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.22.279.851 I srv update: - prompt 0x5dccfc266bc0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.22.279.852 I srv update: - prompt 0x5dccfb9e1b90: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.22.279.853 I srv update: - prompt 0x5dccfc86f6e0: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.22.279.854 I srv update: - prompt 0x5dccfbc14560: 107 tokens, checkpoints: 1, 302.807 MiB |
| 1.22.279.854 I srv update: - prompt 0x5dccff995dd0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.22.279.855 I srv update: - prompt 0x5dccfc682b30: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.22.279.856 I srv update: - prompt 0x5dccfc26ace0: 117 tokens, checkpoints: 1, 303.140 MiB |
| 1.22.279.856 I srv update: - prompt 0x5dccfbca2080: 251 tokens, checkpoints: 1, 307.591 MiB |
| 1.22.279.857 I srv update: - prompt 0x5dccffae7560: 189 tokens, checkpoints: 1, 305.532 MiB |
| 1.22.279.859 I srv update: - prompt 0x5dccfc223650: 311 tokens, checkpoints: 1, 309.585 MiB |
| 1.22.279.859 I srv update: - prompt 0x5dccfcb2e8e0: 301 tokens, checkpoints: 1, 309.253 MiB |
| 1.22.279.860 I srv update: - prompt 0x5dccfc70f990: 172 tokens, checkpoints: 1, 304.967 MiB |
| 1.22.279.861 I srv update: - prompt 0x5dccfc55bb90: 299 tokens, checkpoints: 1, 309.186 MiB |
| 1.22.279.862 I srv get_availabl: prompt cache update took 205.21 ms |
| 1.22.279.984 I slot launch_slot_: id 0 | task 1103 | processing task, is_child = 0 |
| 1.22.279.997 W slot update_slots: id 0 | task 1103 | erased invalidated context checkpoint (pos_min = 100, pos_max = 100, n_tokens = 101, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.22.571.661 I slot create_check: id 0 | task 1103 | created context checkpoint 1 of 32 (pos_min = 85, pos_max = 85, n_tokens = 86, size = 149.626 MiB) |
| 1.26.911.871 I slot print_timing: id 0 | task 1103 | n_decoded = 100, tg = 23.32 t/s |
| 1.29.801.280 I slot print_timing: id 0 | task 1103 | prompt eval time = 343.21 ms / 90 tokens ( 3.81 ms per token, 262.23 tokens per second) |
| 1.29.801.285 I slot print_timing: id 0 | task 1103 | eval time = 7178.06 ms / 166 tokens ( 43.24 ms per token, 23.13 tokens per second) |
| 1.29.801.286 I slot print_timing: id 0 | task 1103 | total time = 7521.28 ms / 256 tokens |
| 1.29.801.287 I slot print_timing: id 0 | task 1103 | graphs reused = 1201 |
| 1.29.801.314 I slot release: id 0 | task 1103 | stop processing: n_tokens = 255, truncated = 0 |
| 1.29.801.326 I srv update_slots: all slots are idle |
| 1.29.848.352 I srv params_from_: Chat format: peg-native |
| 1.29.848.788 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 18086967586 |
| 1.29.848.791 I srv get_availabl: updating prompt cache |
| 1.29.848.827 W srv prompt_save: - saving prompt with length 255, total state size = 158.098 MiB (draft: 0.000 MiB) |
| 1.30.049.275 I srv load: - looking for better prompt, base f_keep = 0.031, sim = 0.060 |
| 1.30.049.292 I srv update: - cache state: 15 prompts, 4570.583 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.30.049.293 I srv update: - prompt 0x5dccfc719990: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.30.049.294 I srv update: - prompt 0x5dccfc266bc0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.30.049.295 I srv update: - prompt 0x5dccfb9e1b90: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.30.049.295 I srv update: - prompt 0x5dccfc86f6e0: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.30.049.296 I srv update: - prompt 0x5dccfbc14560: 107 tokens, checkpoints: 1, 302.807 MiB |
| 1.30.049.296 I srv update: - prompt 0x5dccff995dd0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.30.049.297 I srv update: - prompt 0x5dccfc682b30: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.30.049.298 I srv update: - prompt 0x5dccfc26ace0: 117 tokens, checkpoints: 1, 303.140 MiB |
| 1.30.049.298 I srv update: - prompt 0x5dccfbca2080: 251 tokens, checkpoints: 1, 307.591 MiB |
| 1.30.049.299 I srv update: - prompt 0x5dccffae7560: 189 tokens, checkpoints: 1, 305.532 MiB |
| 1.30.049.299 I srv update: - prompt 0x5dccfc223650: 311 tokens, checkpoints: 1, 309.585 MiB |
| 1.30.049.300 I srv update: - prompt 0x5dccfcb2e8e0: 301 tokens, checkpoints: 1, 309.253 MiB |
| 1.30.049.300 I srv update: - prompt 0x5dccfc70f990: 172 tokens, checkpoints: 1, 304.967 MiB |
| 1.30.049.301 I srv update: - prompt 0x5dccfc55bb90: 299 tokens, checkpoints: 1, 309.186 MiB |
| 1.30.049.301 I srv update: - prompt 0x5dccfc70cb90: 255 tokens, checkpoints: 1, 307.724 MiB |
| 1.30.049.303 I srv get_availabl: prompt cache update took 200.51 ms |
| 1.30.049.410 I slot launch_slot_: id 0 | task 1271 | processing task, is_child = 0 |
| 1.30.049.422 W slot update_slots: id 0 | task 1271 | erased invalidated context checkpoint (pos_min = 85, pos_max = 85, n_tokens = 86, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.30.260.511 I slot create_check: id 0 | task 1271 | created context checkpoint 1 of 32 (pos_min = 128, pos_max = 128, n_tokens = 129, size = 149.626 MiB) |
| 1.34.851.128 I slot print_timing: id 0 | task 1271 | n_decoded = 100, tg = 22.03 t/s |
| 1.37.854.610 I slot print_timing: id 0 | task 1271 | n_decoded = 174, tg = 23.07 t/s |
| 1.38.115.046 I slot print_timing: id 0 | task 1271 | prompt eval time = 262.64 ms / 133 tokens ( 1.97 ms per token, 506.40 tokens per second) |
| 1.38.115.050 I slot print_timing: id 0 | task 1271 | eval time = 7802.98 ms / 181 tokens ( 43.11 ms per token, 23.20 tokens per second) |
| 1.38.115.051 I slot print_timing: id 0 | task 1271 | total time = 8065.62 ms / 314 tokens |
| 1.38.115.052 I slot print_timing: id 0 | task 1271 | graphs reused = 1379 |
| 1.38.115.092 I slot release: id 0 | task 1271 | stop processing: n_tokens = 313, truncated = 0 |
| 1.38.115.106 I srv update_slots: all slots are idle |
| 1.38.166.627 I srv params_from_: Chat format: peg-native |
| 1.38.166.864 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 18095281365 |
| 1.38.166.867 I srv get_availabl: updating prompt cache |
| 1.38.166.908 W srv prompt_save: - saving prompt with length 313, total state size = 160.025 MiB (draft: 0.000 MiB) |
| 1.38.632.771 I srv load: - looking for better prompt, base f_keep = 0.026, sim = 0.075 |
| 1.38.632.786 I srv update: - cache state: 16 prompts, 4880.234 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.38.632.787 I srv update: - prompt 0x5dccfc719990: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.38.632.788 I srv update: - prompt 0x5dccfc266bc0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.38.632.789 I srv update: - prompt 0x5dccfb9e1b90: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.38.632.790 I srv update: - prompt 0x5dccfc86f6e0: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.38.632.791 I srv update: - prompt 0x5dccfbc14560: 107 tokens, checkpoints: 1, 302.807 MiB |
| 1.38.632.797 I srv update: - prompt 0x5dccff995dd0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.38.632.798 I srv update: - prompt 0x5dccfc682b30: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.38.632.798 I srv update: - prompt 0x5dccfc26ace0: 117 tokens, checkpoints: 1, 303.140 MiB |
| 1.38.632.799 I srv update: - prompt 0x5dccfbca2080: 251 tokens, checkpoints: 1, 307.591 MiB |
| 1.38.632.800 I srv update: - prompt 0x5dccffae7560: 189 tokens, checkpoints: 1, 305.532 MiB |
| 1.38.632.801 I srv update: - prompt 0x5dccfc223650: 311 tokens, checkpoints: 1, 309.585 MiB |
| 1.38.632.801 I srv update: - prompt 0x5dccfcb2e8e0: 301 tokens, checkpoints: 1, 309.253 MiB |
| 1.38.632.802 I srv update: - prompt 0x5dccfc70f990: 172 tokens, checkpoints: 1, 304.967 MiB |
| 1.38.632.806 I srv update: - prompt 0x5dccfc55bb90: 299 tokens, checkpoints: 1, 309.186 MiB |
| 1.38.632.807 I srv update: - prompt 0x5dccfc70cb90: 255 tokens, checkpoints: 1, 307.724 MiB |
| 1.38.632.808 I srv update: - prompt 0x5dccfb953b60: 313 tokens, checkpoints: 1, 309.651 MiB |
| 1.38.632.809 I srv get_availabl: prompt cache update took 465.94 ms |
| 1.38.632.919 I slot launch_slot_: id 0 | task 1454 | processing task, is_child = 0 |
| 1.38.632.935 W slot update_slots: id 0 | task 1454 | erased invalidated context checkpoint (pos_min = 128, pos_max = 128, n_tokens = 129, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.38.792.299 I slot create_check: id 0 | task 1454 | created context checkpoint 1 of 32 (pos_min = 101, pos_max = 101, n_tokens = 102, size = 149.626 MiB) |
| 1.41.127.237 I slot print_timing: id 0 | task 1454 | prompt eval time = 210.94 ms / 106 tokens ( 1.99 ms per token, 502.50 tokens per second) |
| 1.41.127.243 I slot print_timing: id 0 | task 1454 | eval time = 2283.35 ms / 56 tokens ( 40.77 ms per token, 24.53 tokens per second) |
| 1.41.127.244 I slot print_timing: id 0 | task 1454 | total time = 2494.29 ms / 162 tokens |
| 1.41.127.245 I slot print_timing: id 0 | task 1454 | graphs reused = 1433 |
| 1.41.127.283 I slot release: id 0 | task 1454 | stop processing: n_tokens = 161, truncated = 0 |
| 1.41.127.296 I srv update_slots: all slots are idle |
| 1.41.177.409 I srv params_from_: Chat format: peg-native |
| 1.41.177.642 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 18098293555 |
| 1.41.177.644 I srv get_availabl: updating prompt cache |
| 1.41.177.680 W srv prompt_save: - saving prompt with length 161, total state size = 154.975 MiB (draft: 0.000 MiB) |
| 1.41.372.130 I srv load: - looking for better prompt, base f_keep = 0.075, sim = 0.094 |
| 1.41.372.150 I srv update: - cache state: 17 prompts, 5184.835 MiB (limits: 8192.000 MiB, 16384 tokens, 16384 est) |
| 1.41.372.152 I srv update: - prompt 0x5dccfc719990: 73 tokens, checkpoints: 1, 301.678 MiB |
| 1.41.372.153 I srv update: - prompt 0x5dccfc266bc0: 71 tokens, checkpoints: 1, 301.611 MiB |
| 1.41.372.153 I srv update: - prompt 0x5dccfb9e1b90: 75 tokens, checkpoints: 1, 301.744 MiB |
| 1.41.372.154 I srv update: - prompt 0x5dccfc86f6e0: 76 tokens, checkpoints: 1, 301.778 MiB |
| 1.41.372.155 I srv update: - prompt 0x5dccfbc14560: 107 tokens, checkpoints: 1, 302.807 MiB |
| 1.41.372.155 I srv update: - prompt 0x5dccff995dd0: 68 tokens, checkpoints: 1, 301.512 MiB |
| 1.41.372.156 I srv update: - prompt 0x5dccfc682b30: 97 tokens, checkpoints: 1, 302.475 MiB |
| 1.41.372.157 I srv update: - prompt 0x5dccfc26ace0: 117 tokens, checkpoints: 1, 303.140 MiB |
| 1.41.372.157 I srv update: - prompt 0x5dccfbca2080: 251 tokens, checkpoints: 1, 307.591 MiB |
| 1.41.372.158 I srv update: - prompt 0x5dccffae7560: 189 tokens, checkpoints: 1, 305.532 MiB |
| 1.41.372.160 I srv update: - prompt 0x5dccfc223650: 311 tokens, checkpoints: 1, 309.585 MiB |
| 1.41.372.161 I srv update: - prompt 0x5dccfcb2e8e0: 301 tokens, checkpoints: 1, 309.253 MiB |
| 1.41.372.161 I srv update: - prompt 0x5dccfc70f990: 172 tokens, checkpoints: 1, 304.967 MiB |
| 1.41.372.162 I srv update: - prompt 0x5dccfc55bb90: 299 tokens, checkpoints: 1, 309.186 MiB |
| 1.41.372.163 I srv update: - prompt 0x5dccfc70cb90: 255 tokens, checkpoints: 1, 307.724 MiB |
| 1.41.372.164 I srv update: - prompt 0x5dccfb953b60: 313 tokens, checkpoints: 1, 309.651 MiB |
| 1.41.372.164 I srv update: - prompt 0x5dccfc198530: 161 tokens, checkpoints: 1, 304.601 MiB |
| 1.41.372.166 I srv get_availabl: prompt cache update took 194.52 ms |
| 1.41.372.277 I slot launch_slot_: id 0 | task 1512 | processing task, is_child = 0 |
| 1.41.372.290 W slot update_slots: id 0 | task 1512 | erased invalidated context checkpoint (pos_min = 101, pos_max = 101, n_tokens = 102, n_swa = 0, pos_next = 0, size = 149.626 MiB) |
| 1.41.549.010 I slot create_check: id 0 | task 1512 | created context checkpoint 1 of 32 (pos_min = 122, pos_max = 122, n_tokens = 123, size = 149.626 MiB) |
| 1.45.993.451 I slot print_timing: id 0 | task 1512 | n_decoded = 100, tg = 22.76 t/s |
| 1.48.115.147 I slot print_timing: id 0 | task 1512 | prompt eval time = 228.25 ms / 127 tokens ( 1.80 ms per token, 556.40 tokens per second) |
| 1.48.115.158 I slot print_timing: id 0 | task 1512 | eval time = 6514.60 ms / 148 tokens ( 44.02 ms per token, 22.72 tokens per second) |
| 1.48.115.159 I slot print_timing: id 0 | task 1512 | total time = 6742.85 ms / 275 tokens |
| 1.48.115.161 I slot print_timing: id 0 | task 1512 | graphs reused = 1578 |
| 1.48.115.214 I slot release: id 0 | task 1512 | stop processing: n_tokens = 274, truncated = 0 |
| 1.48.115.229 I srv update_slots: all slots are idle |
| 1.48.200.079 I srv operator(): operator(): cleaning up before exit... |
|
|
