nphSi

/

Z-Image-Lora

like 127

Unlikely. Maybe for a test when OT/RunPod is back on track.

I also did some extensive A/B testing today using both my custom trained LoRAs for the same character based on the same dataset.

Krea 2 has definitely taken a noticeable step up compared to Z-Image. Surfaces, textures, fabrics, hair, skin – everything looks way more realistic, and some parts, like the hair, are insanely detailed.
The prompt adherence is also a lot better, even compared to Z-Image Base (which I use with my own distilled finetune, since Base sticks to prompts way better than Turbo). Some poses aren't even possible with Z-Image.

Anyway, for a local base model without any realism LoRAs or other fine tunes yet, this is really impressive, and I don't think I'll bother training for Z-Image anymore. The training times are pretty similar too, at least with the settings I've been using for Z-Image so far (multi-resolution, 1024, 512 minimum). It usually takes about 1 to 2 hours per LoRA on a 5090.
Too bad there’s no support in Onetrainer yet, but I'm sure that's coming. In the meantime, I'll keep training a few of my datasets via AI Toolkit.
By the way, I wrote myself a GPU sniper via the Runpod API—it spins up a 5090 pod with either the AI Toolkit or Onetrainer template within 15 to 20 minutes max. If you're interested in that, nphSI, hit me up. ;)

I like prompt 17. Dont know why, cant think with 30C in room...

Need to wait for proper GGUF support in Comfy for both model and TE before i can test. I doubt i will be happy with Q4 quality...

Need to wait for proper GGUF support in Comfy for both model and TE before i can test. I doubt i will be happy with Q4 quality...

Both fp8 models are working fine on my 16 GB 4070 Ti with ComfyUI's smart memory management.

Hi there, thanks for sharing all these LoRAs. Surprisingly, your vrtlRihanna LoRA that you trained on ZiT is also working with Krea 2 Turbo... but only the Rihanna LoRA is working. Do you remember how you trained it compared to the other ones?

I've also trained about 10 ZiT LoRAs with AI Toolkit, but none of them are working with Krea 2. Only your vrtlRihanna is working... pretty impressive!

I am pretty sure its not working, its just that krea2´s knowledge about Rihanna is very good in the base model.
All loras here are trained on ZI Base, ZI DeTurbo version have been removed/replaced long ago.

OneTrainer now supports Krea2 as well.
And just like with Z-Image, training Krea2 models is way faster here than with AI-Toolkit. My settings for the 5090: Standard OneTrainer Krea preset, but regular AdamW (not AdamW-8Bit), batch size 1 instead of 2, cosine scheduler, and masked training turned on.

For a quick comparison, I ran one of my 30-image datasets for 100 epochs (= 3000 steps)—once at 512px resolution, and once mixed 1024px and 512px (1024,512). The 512px training took just under 17 minutes, while the 1024,512 run took a little over 40 minutes (without sample generation). AI-Toolkit took twice as long. Good and usable results started popping up around 2200-2400 steps.
The quality with the 1024 LoRA is on another level. It’s really noticeable starting right from cowboy shots / half-body portraits—and obviously, the closer you get to the character, the bigger the difference. With the exact same prompt, you get more and finer details, way better skin textures and skin tones, as well as better color resolution and rendering overall. You can see this for example by comparing gradients in blue skies. But yeah, that was already the case with Z-Image anyway.

Current OT RunPod docker images misses some CUDA 13 libs but you can easily install them using the web terminal with "apt update" and "apt install libnvjitlink-13-1". After this bitsandbytes is working again and so optimisers like RMSPROP. ADAMW is basically RMSPROP with added Momentum and some large scale dataset optimisations for finetuning. RMSPROP has gotten Momentum too (use 0.9) and other ADAMW stuff does not matter much for small scale datasets we use on simple character Loras so you can get another big speed up by using RMSPROP.
I currently do not have any funds in my RunPod to test but i would like to see some K2 1024 vs ZI 512 comparisons of Audrey Hepburn. Dataset is here: https://huggingface.co/nphSi/Z-Image-Lora/tree/main/temp
Its a very average and heavy edited dataset. No masks needed as backgrounds are clean. Quality improvements should be best seen here...

My RunPods always starts with cuda 12.8. Tried installing libnvjitlink-12-8 and using RMSPROP with Momentum, but doesn't work for me. Onetrainer is freezing before the training.

You use the "dxqbyd/runpod-onetrainer-cli:1.2" template and the GPU filter for Cuda 13? And enable "Update" in OT Cloud Tab and before you start training you have to enter the two apt commands.
Be aware that it can take some time until training starts because it has to download the models first. Raw, TE and VAE, about 40gb...

@nphSi , please can you share your ostris Ai Toolkit config to training z-image Loras in Runpod ? Thanks for all.

I never used AI Toolkit and never will because it want to install stuff system wide.
Hiding post soon for Off-Topic.

You use the "dxqbyd/runpod-onetrainer-cli:1.2" template and the GPU filter for Cuda 13? And enable "Update" in OT Cloud Tab and before you start training you have to enter the two apt commands.
Be aware that it can take some time until training starts because it has to download the models first. Raw, TE and VAE, about 40gb...

Yes, I did all that, I have a script to search und run Pods with this template automatically.
tried it again with cuda 13.1, same errors. I even updated Pytorch, but didn't work either.

Gemini says:

"The fact that the error persists is not due to you or OneTrainer, but rather a known infrastructure issue with RunPod.
In the RunPod developer forums and community Discords, these exact reports are currently piling up for newly rented RTX 5090 instances.
The error RuntimeError: CUDA unknown error at this exact point means that RunPod's virtual Docker container cannot access the physical graphics card on the server.
When the pod is being created, the provider's so-called GPU passthrough is failing in the background. While the Nvidia driver is running on the host server (which is why nvidia-smi often still displays values), the OneTrainer container is completely blocked from accessing the GPU. Unfortunately, updating packages won't help here, as the problem sits deeper within the system.

The Quick Test (The Confirmation): If you want to see it in black and white, log in via SSH and run this short Python command through your regular system terminal (not inside the venv):

python3 -c "import ctypes; lib=ctypes.CDLL('libcuda.so.1'); print(lib.cuInit(0))"

If the output returns 999 (CUDA_ERROR_UNKNOWN), that's your proof: the system is blocking GPU initialization at the hardware level."

And I get 999.

As i said before current OT + RunPod duo is an absolute frustrating lottery game where you lose real money. Took me 5 tries to get a working Pod. Its training now on 1024 with 1.1s/it. Alone the speed makes it uninteresting for more than some tests. IMHO no quality improvement to ZI can justify this cost/time loss.
Will post the Lora DL link when finished here... Will try do to a 512 run also.

Edit: Will also do a run with Jessy Wellmer because krea2´s knowledge on Audrey H is too good and will sway results.

If someone feels the urgent need to lend a hand (or finger) on my RunPod costs - my Ko-Fi is still up 😁

Edit: I stopped Audrey because it did not improve since step 300.

I'm going to try another training run with my AdamW settings and CUDA 13, since that worked well with CUDA 12.8 over the last few days. However, it's currently difficult to snag any pods.

IMHO no quality improvement to ZI can justify this cost/time loss.

Personally, I see things a bit differently. I've already trained my Loras for Z-Image at 512,1024 and Adamw (did a lot of testing), and that actually runs faster with Krea2 in Onetrainer. Plus, the leap in quality is significant.
Sure, the training takes much longer compared to only 512 with Rmsprop, but for me personally, I'd rather do fewer Loras but at a higher quality.

This statement was made with the thread question and this site in the background. Do i train my Loras on krea2 in the future and ditch ZI or do i train both? No to both. I wont and cant handle that.
In a personal view its clear to aim for the best quality possible.

• k2 Jessy Wellmer (vrtlJessyWellmer).safetensors (1024) (1.1s/it)

• k2l Jessy Wellmer (vrtlJessyWellmer).safetensors (512) (2.7it/s)

• vrtlJessyWellmer.7z dataset if you want to test yourself

https://huggingface.co/nphSi/Z-Image-Lora/tree/main/temp

Samples: (Not for technical comparison, real world output. Only identical is pre LLM prompt) Left k2 1024 - right ZI 512

• k2 512 vs k2 1024

btw k2 training settings are adopted from my ZI settings without any testing for optimisation. I am pretty sure they could be improved...

See https://huggingface.co/nphSi/Z-Image-Lora/discussions/55

• k2 Jessy Wellmer (vrtlJessyWellmer).safetensors (1024) (1.1s/it)

• k2l Jessy Wellmer (vrtlJessyWellmer).safetensors (512) (2.7it/s)

• vrtlJessyWellmer.7z dataset if you want to test yourself

Great stuff, thanks for sharing. I found it interesting that you don't worry about too many buckets being created due to the different aspect ratios. Also, the detail quality of many images in this dataset isn't even that great, yet the LoRAs turn out pretty usable.
I always went through the trouble of cropping and outpainting everything to a 1:1 format and upscaling it to 4K, just to generate 1024 or 1216 resolutions for the dataset again. I’ll keep doing the latter because you can actually see it in the 1024 LoRAs, but I'm going to skip the former next time. Curious to see how it goes!

The rules about exact AR and res is from the days of SD/SDXL because they are limited to max 512 resp. 1024 and every pixel above that could break it. Newer models dont care because of much higher limits. My AR handling is done automatically with Irfanview and output res of 1.06MP. It does the correct AR and always 1MP. OT does the rest in buckets.
Newer models LOVE AI upscaled/refined images and train very well on it and in combination with adjusted noising strength, offset and bias it captures every detail and even add it where its not present in the dataset.

Very interesting and helpful, that saves a lot of time in preparing the datasets.

By the way, here are a few examples from one of my 1024 Loras based on a dataset I described above. I scaled down the images so as not to overwhelm things here.
However, you can find them in full resolution here, where I have also uploaded the LoRA for you to try out yourself: https://huggingface.co/Jadawin84/Kaley/tree/main

One more thing, maybe 2 tips or tools in case you aren't familiar with them yet:

Krea 2 follows prompts very well, but struggles when it comes to different emotions. The Realism Engine LoRA, available on CivitAI, helps with this. Although v3 is out now, I recommend v2: https://civitai.red/models/2688234/realism-engine-ideogram-4-krea-2?modelVersionId=3070702

And for testing LoRAs or making comparisons, character sheets can be prompted directly with Krea 2. LoRAs for this are popping up everywhere, but it's completely unnecessary.
Below are some examples.

I've uploaded the corresponding prompts to my Hugging Face (works with Z-Image too, Base at least). It works best with an aspect ratio as specified in the filenames and a resolution of 1.5+ MP.
https://huggingface.co/Jadawin84/Character-Sheet_Prompts

hello from france, check this link, this a list of all celebrities that Krea 2 had in his database !
https://photos.google.com/share/AF1QipNIONmNur4qtfMg7ar2MD5z-1opZQBBzoefJfVEAKLyjwmU-wOphoVyyUuKK6gcWA?key=VC0zX0ZUd0diQUJpTWRxYThBelA5QWNQc3EzT3p3

Good idea but this does not look very well...

I spent all morning trying out Krea 2 Turbo, and I have to say it's actually superior to Z-Image Turbo, by a lot. Z-Image Base is already closer, but still, it's no contest. Z-Image Base is too much of a wild horse, hard to tame even with deep negative prompts. It's too prone to generating deformities and color palettes that too often tend towards sepia, making it really hard to control. Krea 2, on the other hand, is incredible. Its consistency with the text is spectacular. It NEVER generates deformations or color palettes tending towards sepia, which I hate to death. The quality is extremely photographic and realistic, and the thing I appreciate most is that you no longer have to struggle with overly in-depth descriptions to get the character to pose, which I hated about Z-Image Turbo, which often insisted on not wanting to do simple poses like crossed legs or the like.

I'm really impressed with Krea 2, it will probably become my favorite image model, images that I will then animate with LTX-2.3 distilled GGUF which also became my favorite video model, managing to make 121 frames and 24 fps videos in less than a minute, something that before with Wan 2.1 and 2.2 I could only dream of.

Very interesting and helpful, that saves a lot of time in preparing the datasets.

By the way, here are a few examples from one of my 1024 Loras based on a dataset I described above. I scaled down the images so as not to overwhelm things here.
However, you can find them in full resolution here, where I have also uploaded the LoRA for you to try out yourself: https://huggingface.co/Jadawin84/Kaley/tree/main

Wow, the Loras look incredible. Especially at this resolution. Can you tell us anything about the training process, or—in the case of the AI Toolkit—maybe even share your configuration? I'm still trying
to convince nphSi to use krea2 Loras 🙃
EDIT: I see that you shared some settings above. Will try them. One question though: What is better 1024 resolution or mixed resolution (1024,512)? Thank you.

Thank you.
Regarding Multiresolution-Training: Unfortunately: if you ask five different people, you get five different answers. Even the AIs can't agree on this one. ;-)

In any case, my experience - even with modern DiT models like Krea2 and Z-Image - is that training multiple resolutions is extremely helpful when a character isn't just shown in close-ups or medium close-ups (where the 1024px really shines), but also in wide shots or distant shots. If you only train at 1024px, both models start to melt faces - especially around the eyes—or just hallucinate weird details in those distant shots. Not so much as Flux for example, but still....
If you've also trained at least at 512px, the models have 'learned' much better how to handle smaller variations of the character. At least, that’s been my experience.

Onetrainer is currently unusable via Runpod; my pods kept freezing up, at least as of last weekend. Which is a shame, because Onetrainer's 'Concepts' feature allows you to set up and use different image sets for 512px and 1024px. Combined with masking, this pushes the quality of the Lora up another notch.
But that’s complaining at a very high level, because even with AI-Toolkit, the results are highly usable—even though its resolution bucketing always pulls from the same shared (1024px) dataset, and it doesn't support masks. And AI-Toolkit is slower, of course.
But ultimately, these are just nuances for character Loras. The most important thing is to train at sizes above 512px in the first place, even if it takes nearly twice as long. However, you can save yourself the trouble if you don't have a truly high-quality dataset. In the end, preparing and curating your dataset is more than 80% of the battle.

Hope this helps a bit. Here is an example of a Lora (Eliza Taylor, she is also in nphSi's Z-Image pool) trained just the day before yesterday using AI-Toolkit ("massaged" and upscaled dataset of 28 images) with her positioned at some distance. Generated in 1.5 MP, no face detailer used, just a simple general 4-step second pass, and downscaled for convenience here ...

Thank you for your detailed answer. I just trained a Lora with OneTrainer and with the settings you mentioned above (also masking). I set the resolution to 1024 only. I will test it tomorrow and compare it with my Ai Toolkit Lora which was trained in 512 resolution. The training was much faster (I have a 4090) than the AI toolkit training. I'll write about it tomorrow. Thanks again.