circlestone-labs

/

Anima

like 1.74k

Follow

CircleStone Labs 819

You can use the following parameters in supported trainers to exclude the following modules and layers:
exclude_patterns=['.self_attn.','.mlp.','blocks.(?:19|2[0-7])..*']
This has some effect, but the improvements from parameter tuning are quite limited. It is best to use Nano Banana or other models to convert your training set into a 2D anime style.

You can use the following parameters in supported trainers to exclude the following modules and layers:
exclude_patterns=['.self_attn.','.mlp.','blocks.(?:19|2[0-7])..*']
This has some effect, but the improvements from parameter tuning are quite limited. It is best to use Nano Banana or other models to convert your training set into a 2D anime style.

I don't think excluding MLP layers would work out that well for character training. They do have the strongest influence on style in Loras (which you can see easily by lowering the MLP layer weights of any style Lora) but they also have a pretty strong influence on characters too. Excluding it will probably make training a character really hard; I think you'd be better off either lowering the LR for MLP layers or lowering the LR for other blocks instead of excluding them entirely

This may be a bit of a silly question, but do you have all of your images tagged with "3d"? That's what I'd try first.

You can use the following parameters in supported trainers to exclude the following modules and layers:
exclude_patterns=['.self_attn.','.mlp.','blocks.(?:19|2[0-7])..*']
This has some effect, but the improvements from parameter tuning are quite limited. It is best to use Nano Banana or other models to convert your training set into a 2D anime style.

I don't think excluding MLP layers would work out that well for character training. They do have the strongest influence on style in Loras (which you can see easily by lowering the MLP layer weights of any style Lora) but they also have a pretty strong influence on characters too. Excluding it will probably make training a character really hard; I think you'd be better off either lowering the LR for MLP layers or lowering the LR for other blocks instead of excluding them entirely

I know. This parameter is specifically intended to mitigate style overfitting and reduce the LoRA's impact on the art style, since I have no way to add more styles to the training set. This is a suboptimal workaround. As I mentioned above, when it is possible to optimize the training set, doing so is far more effective than adjusting parameters.

Hey, if I’m training a Concept LoRA, should I disable that setting to avoid style drift?

You can use the following parameters in supported trainers to exclude the following modules and layers:
exclude_patterns=['.self_attn.','.mlp.','blocks.(?:19|2[0-7])..*']
This has some effect, but the improvements from parameter tuning are quite limited. It is best to use Nano Banana or other models to convert your training set into a 2D anime style.

Thanks for the advice. I think the Nano Banana will be better.

This may be a bit of a silly question, but do you have all of your images tagged with "3d"? That's what I'd try first.

Hmm, no... I'm using auto-tagger and he doesn't mark them with that tag.

This may be a bit of a silly question, but do you have all of your images tagged with "3d"? That's what I'd try first.

Hmm, no... I'm using auto-tagger and he doesn't mark them with that tag.

Then adding "3d" to the tags of all your dataset images would be the first thing I'd try. That should allow the training to at least try to separate the essence of the character from the 3D bias of the dataset.

This may be a bit of a silly question, but do you have all of your images tagged with "3d"? That's what I'd try first.

Hmm, no... I'm using auto-tagger and he doesn't mark them with that tag.

Then adding "3d" to the tags of all your dataset images would be the first thing I'd try. That should allow the training to at least try to separate the essence of the character from the 3D bias of the dataset.

Thanks for the advice, but I think I’ve already solved the problem. I just change the style of images from 3D to 2D and everything works well :)

Was there any custom comfyui node to visualize which layer a lora has affected the model?

Was there any custom comfyui node to visualize which layer a lora has affected the model?

For looking at layers there was a Comfy extension posted here a couple weeks ago that could do it (https://huggingface.co/circlestone-labs/Anima/discussions/170). Though you should only really focus on MLP, cross attention, and self attention (AdaLN modulation layers are excluded by default in sd-scripts unless you're using diffusion-pipe, which you probably aren't)

Was there any custom comfyui node to visualize which layer a lora has affected the model?

For looking at layers there was a Comfy extension posted here a couple weeks ago that could do it (https://huggingface.co/circlestone-labs/Anima/discussions/170). Though you should only really focus on MLP, cross attention, and self attention (AdaLN modulation layers are excluded by default in sd-scripts unless you're using diffusion-pipe, which you probably aren't)

Thanks a lot