---
title: Optimize Qwen3.6-35B-A3B tok/s on a10g-small
created: '2026-06-15T00:08:20.710Z'
updated: '2026-06-15T00:16:35.715Z'
artifacts:
  - type: link
    title: Static HF Space experiment log
    href: 'https://abidlabs-qwen36-a10g-experiment-log.static.hf.space/index.html'
    note: >-
      Published static HTML export of /Users/abubakar/dev/experiment-log-data to
      Hugging Face Space abidlabs/qwen36-a10g-experiment-log.
---

## Goal
Maximize inference decode throughput (tokens/sec) for `Qwen/Qwen3.6-35B-A3B` on Hugging Face Jobs `a10g-small` (single A10G 24GB).

## Hypothesis
Because the dense BF16 model cannot fit in 24GB, fastest practical paths will be quantized runtimes: llama.cpp CUDA with GGUF quants and/or Transformers/vLLM 4-bit/FP8 variants. Start with cheap single-job benchmarks over candidate runtimes/settings, then record the best median decode tok/s.

## Method
Use fixed prompt length and generation length, warmup once, then time several decode runs. Record model variant, runtime, quantization, GPU offload settings, memory, and tok/s.