| # ComfyUI-layerdiffuse |
| ComfyUI implementation of https://github.com/layerdiffusion/LayerDiffuse. |
|
|
| ## Installation |
| Download the repository and unpack into the custom_nodes folder in the ComfyUI installation directory. |
| |
| Or clone via GIT, starting from ComfyUI installation directory: |
| ```bash |
| cd custom_nodes |
| git clone git@github.com:huchenlei/ComfyUI-layerdiffuse.git |
| ``` |
| |
| Run `pip install -r requirements.txt` to install python dependencies. You might experience version conflict on diffusers if you have other extensions |
| that depends on other versions of diffusers. In this case, it is recommended to setup separate Python venvs. |
| |
| ## Workflows |
| ### [Generate foreground](https://github.com/huchenlei/ComfyUI-layerdiffuse/blob/main/examples/layer_diffusion_fg_example_rgba.json) |
|  |
| |
| ### [Generate foreground (RGB + alpha)](https://github.com/huchenlei/ComfyUI-layerdiffuse/blob/main/examples/layer_diffusion_fg_example.json) |
| If you want more control of getting RGB image and alpha channel mask separately, you can use this workflow. |
|  |
| |
| ### [Blending (FG/BG)](https://github.com/huchenlei/ComfyUI-layerdiffuse/blob/main/examples/layer_diffusion_cond_example.json) |
| Blending given FG |
|  |
| |
| Blending given BG |
|  |
| |
| ### [Extract FG from Blended + BG](https://github.com/huchenlei/ComfyUI-layerdiffuse/blob/main/examples/layer_diffusion_diff_fg.json) |
|  |
| |
| ### [Extract BG from Blended + FG](https://github.com/huchenlei/ComfyUI-layerdiffuse/blob/main/examples/layer_diffusion_diff_bg.json) |
| [Forge impl's sanity check](https://github.com/layerdiffuse/sd-forge-layerdiffuse#sanity-check) sets `Stop at` to 0.5 to get better quality BG. |
| This workflow might be inferior comparing to other object removal workflows. |
|  |
| |
| ### [Extract BG from Blended + FG (Stop at 0.5)](https://github.com/huchenlei/ComfyUI-layerdiffuse/blob/main/examples/layer_diffusion_diff_bg_stop_at.json) |
| In [SD Forge impl](https://github.com/layerdiffuse/sd-forge-layerdiffuse), there is a `stop at` param that determines when |
| layer diffuse should stop in the denosing process. In the background, what this param does is unapply the LoRA and c_concat cond after a certain step |
| threshold. This is hard/risky to implement directly in ComfyUI as it requires manually load a model that has every changes except the layer diffusion |
| change applied. A workaround in ComfyUI is to have another img2img pass on the layer diffuse result to simulate the effect of `stop at` param. |
|  |
| |
| |
| ### [Generate FG from BG combined](https://github.com/huchenlei/ComfyUI-layerdiffuse/blob/main/examples/layer_diffusion_cond_fg_all.json) |
| Combines previous workflows to generate blended and FG given BG. We found that there are some color variations in the extracted FG. Need to confirm |
| with layer diffusion authors on whether this is expected. |
|  |
| |
| ### [2024-3-9] [Generate FG + Blended given BG](https://github.com/huchenlei/ComfyUI-layerdiffuse/blob/main/examples/layer_diffusion_cond_joint_bg.json) |
| Need batch size = 2N. Currently only for SD15. |
|  |
| |
| ### [2024-3-9] [Generate BG + Blended given FG](https://github.com/huchenlei/ComfyUI-layerdiffuse/blob/main/examples/layer_diffusion_cond_joint_fg.json) |
| Need batch size = 2N. Currently only for SD15. |
|  |
| |
| ### [2024-3-9] [Generate BG + FG + Blended together](https://github.com/huchenlei/ComfyUI-layerdiffuse/blob/main/examples/layer_diffusion_joint.json) |
| Need batch size = 3N. Currently only for SD15. |
|  |
| |
| ## Note |
| - Currently only SDXL/SD15 are supported. See https://github.com/layerdiffuse/sd-forge-layerdiffuse#model-notes for more details. |
| - To decode RGBA result, the generation dimension must be multiple of 64. Otherwise, you will get decode error:  |
| |
