Kernels:

---

rootonchair

/

nunchaku-lite-kernels

like 2

sure!

I have updated the build to cover torch 2.13 in v2 branch now. You could try testing it

thanks! working nicely & integrated into sdnext:
(note that i'm overriding bnb based text-encoder with our sdnq quants)

12:59:55-778162 INFO     Load model: select="Diffusers/lite-infer/z-image-turbo-nunchaku-lite-int4_r32-bnb4-text-encoder [8fdcb1d089]"
12:59:59-020398 DEBUG    Download: type=xet mode="XET" fn="build/torch213-cxx11-cu132-x86_64-linux/_nunchaku_lite_kernels_cuda_c1b1dfd.abi3.so" size=202624464
13:00:12-839261 DEBUG    Module: name=transformer cls=ZImageTransformer2DModel size=3.385 params=3321321536 logical=3321321536 quant=QuantizationMethod.NUNCHAKU_LITE
13:00:12-843160 DEBUG    Module: name=text_encoder cls=Qwen3ForCausalLM size=2.840 params=2321880576 logical=4411424256 quant=QuantizationMethod.SDNQ
13:00:12-845608 DEBUG    Module: name=vae cls=AutoencoderKL size=0.156 params=83819683 logical=83819683 quant=None
13:00:12-846296 INFO     Model class=ZImagePipeline modules=3 size=6.382

It's great to see sdnext adopts the work!

it works really well!
would be great to see most popular models covered
from the new ones, you already have zimage, but missing krea2, klein-9b and anima
and those are by far more popular than flux.1, qwen-image or ernie-image

Thanks! These feedbacks are valuable. I will cover those really quickly.

Hi @vladmandic
I have updated a list of new models (krea2, flux klein 9b, 4b) in https://huggingface.co/lite-infer
LMK if you need more models

thanks!
took a quick look at klein-4b and klein-9b are working, but i'm having issues with krea2:

/home/vlado/dev/sdnext/venv/lib/python3.12/site-packages/diffusers/models/modeling_utils.py:1375 in from_pretrained

  1374         if hf_quantizer is not None:
❱ 1375             hf_quantizer.preprocess_model(
  1376                 model=model,

/home/vlado/dev/sdnext/venv/lib/python3.12/site-packages/diffusers/quantizers/base.py:204 in preprocess_model

  203         model.quantization_method = self.quantization_config.quant_method
❱ 204         return self._process_model_before_weight_loading(model, **kwargs)
  205

/home/vlado/dev/sdnext/venv/lib/python3.12/site-packages/diffusers/quantizers/nunchaku/nunchaku_quantizer.py:70 in _process_model_before_weight_loading

  69         quantization_config = self.quantization_config.to_dict()
❱ 70         num_replaced = replace_with_nunchaku_linear(model, quantization_config, self.compute_dtype)
  71

/home/vlado/dev/sdnext/venv/lib/python3.12/site-packages/diffusers/quantizers/nunchaku/utils.py:97 in replace_with_nunchaku_linear

   96     if svdq_config is not None:
❱  97         num_replaced += _replace_quantize_targets(model, "svdq_w4a4", svdq_config, compute_dtype)
   98     if awq_config is not None:

/home/vlado/dev/sdnext/venv/lib/python3.12/site-packages/diffusers/quantizers/nunchaku/utils.py:297 in _replace_quantize_targets

  296         except AttributeError as exc:
❱ 297             raise ValueError(f"Nunchaku target {target!r} does not exist in the model.") from exc
  298
ValueError: Nunchaku target 'text_fusion.layerwise_blocks.0.attn.to_q' does not exist in the model.

Thanks for testing! I think it's because SD.Next loads Krea 2 with its own transformer port, whose module names differ from Diffusers', so the quantizer can't find the targets.

yup, all good.