LiconStudio commited on
Commit
28b9b7c
·
verified ·
1 Parent(s): 6ca733a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +60 -29
README.md CHANGED
@@ -10,6 +10,33 @@ github: https://github.com/liconstudio/ComfyUI-Licon-MSR
10
  library_name: diffusers
11
  ---
12
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
13
  ## Overview
14
 
15
  This model implements a novel approach to multi-reference video generation using **Multiple Subject Reference (MSR)**. Instead of introducing additional encoder branches or fusion modules, we transform multiple static reference images into a pseudo-video sequence that shares the same representation space as the target video.
@@ -22,59 +49,63 @@ This LoRA requires the **[ComfyUI-Licon-MSR](https://github.com/liconstudio/Comf
22
 
23
  ### Multi-Reference Visual Memory
24
 
25
- - **Token-level reference preservation**: Multiple reference images are encoded as video latents, preserving fine-grained visual information at token level rather than compressing into a single embedding
26
- - **Native self-attention retrieval**: The target video tokens directly access reference tokens through the model's existing self-attention mechanismno new architectural components needed
27
- - **In-context conditioning**: References serve as "visual memory" within the main token sequence, not as external conditioning inputs
28
 
29
  ### Flexible Reference Composition
30
 
31
- - **2 to 5 reference images**: Supports varying numbers of reference inputs with increasing complexity
32
- - **Complementary semantic roles**: Each reference image can carry different information:
33
  - Subject identity
34
- - Object/prop details
35
- - Scene/background
36
  - Local textures
37
  - Multiple viewpoints
38
 
39
-
40
  ## What It Can Do
41
 
42
  ### Identity Preservation Across References
43
 
44
- Generate videos where multiple reference identities are simultaneously preserved:
 
45
  - Multiple characters from different reference images
46
- - Character + object combinations
47
- - Object + scene compositions
48
 
49
  ### Relation-Based Composition
50
 
51
- Beyond mere identity preservation, the model can compose references based on textual relation descriptions:
52
- - Action interactions (handing, picking up, pushing)
53
- - Spatial relationships (left-right, foreground-background)
54
- - Temporal event structures (start process result)
 
55
 
56
  ### Cross-Reference Attribute Selection
57
 
58
  The model learns to selectively retrieve attributes from different references:
59
- - Face from reference A, clothing from reference B
60
- - Object identity from one reference, pose/position from another
61
- - Background elements from scene references
62
 
63
- ## Usage Tips (V1 Version)
 
 
64
 
65
- - **Prompt description**: Requires concise but accurate description of reference images. Over-description or under-description both lead to consistency degradation
66
- - **High-motion scenes**: 50fps recommended to ensure smooth motion coherence
67
- - **Generation reliability**: Typically requires 2-3 sampling runs to achieve accurate results
68
 
 
 
 
 
69
 
70
- ## Results Showcase
71
 
72
- ### V1 Version
73
 
74
- | Reference Images | Generated Video |
75
  |:---:|:---:|
76
- | <img src="validition_v1/01/1.jpg" width="80"> <img src="validition_v1/01/2.jpg" width="80"> <img src="validition_v1/01/bg.png" width="80"> | [▶ Play](validition_v1/01/video.mp4) |
77
- | <img src="validition_v1/07/1.jpg" width="80"> <img src="validition_v1/07/2.jpg" width="80"> <img src="validition_v1/07/bg.png" width="80"> | [▶ Play](validition_v1/07/video.mp4) |
78
- | <img src="validition_v1/05/1.png" width="70"> <img src="validition_v1/05/2.png" width="70"> <img src="validition_v1/05/5.png" width="70"> <img src="validition_v1/05/bg.png" width="70"> | [▶ Play](validition_v1/05/video.mp4) |
79
 
80
- ---
 
 
 
 
 
10
  library_name: diffusers
11
  ---
12
 
13
+ # Licon MSR V2 for LTX-2.3
14
+
15
+ ## What's New in V2
16
+
17
+ Compared with V1, **Licon MSR V2** introduces significant improvements in three key areas:
18
+
19
+ ### 1. Improved Consistency
20
+
21
+ - Better preservation of character identity, clothing, objects, and scene details
22
+ - More consistent appearance across frames
23
+ - Improved alignment between multiple reference images and the generated video
24
+ - Reduced identity drift and reference attribute loss
25
+
26
+ ### 2. Improved Stability
27
+
28
+ - More reliable results across repeated sampling runs
29
+ - Reduced visual artifacts, flickering, and temporal inconsistencies
30
+ - More stable generation in complex multi-subject compositions
31
+ - Improved handling of motion and interactions between subjects
32
+
33
+ ### 3. Improved Scene Logic
34
+
35
+ - Better understanding of spatial and action relationships described in prompts
36
+ - More natural subject positioning and interaction
37
+ - Improved temporal progression from the beginning to the end of a video
38
+ - More coherent composition of characters, objects, and backgrounds
39
+
40
  ## Overview
41
 
42
  This model implements a novel approach to multi-reference video generation using **Multiple Subject Reference (MSR)**. Instead of introducing additional encoder branches or fusion modules, we transform multiple static reference images into a pseudo-video sequence that shares the same representation space as the target video.
 
49
 
50
  ### Multi-Reference Visual Memory
51
 
52
+ - **Token-level reference preservation**: Multiple reference images are encoded as video latents, preserving fine-grained visual information at the token level instead of compressing them into a single embedding
53
+ - **Native self-attention retrieval**: Target video tokens directly access reference tokens through the model's existing self-attention mechanism, with no additional architectural components required
54
+ - **In-context conditioning**: References serve as visual memory within the main token sequence rather than as external conditioning inputs
55
 
56
  ### Flexible Reference Composition
57
 
58
+ - **2 to 5 reference images**: Supports varying numbers of reference inputs with increasing composition complexity
59
+ - **Complementary semantic roles**: Each reference image can provide different information:
60
  - Subject identity
61
+ - Object or prop details
62
+ - Scene or background
63
  - Local textures
64
  - Multiple viewpoints
65
 
 
66
  ## What It Can Do
67
 
68
  ### Identity Preservation Across References
69
 
70
+ Generate videos in which multiple reference identities are simultaneously preserved:
71
+
72
  - Multiple characters from different reference images
73
+ - Character and object combinations
74
+ - Object and scene compositions
75
 
76
  ### Relation-Based Composition
77
 
78
+ Beyond identity preservation, the model can compose references according to textual relationship descriptions:
79
+
80
+ - Action interactions, such as handing, picking up, or pushing
81
+ - Spatial relationships, such as left and right or foreground and background
82
+ - Temporal event structures, such as start → process → result
83
 
84
  ### Cross-Reference Attribute Selection
85
 
86
  The model learns to selectively retrieve attributes from different references:
 
 
 
87
 
88
+ - Face from reference A and clothing from reference B
89
+ - Object identity from one reference and pose or position from another
90
+ - Background elements from scene references
91
 
92
+ ## Usage Tips
 
 
93
 
94
+ - **Prompt description**: Use concise but accurate descriptions of the reference images. Both excessive and insufficient descriptions may reduce consistency.
95
+ - **Reference roles**: Clearly describe the role of each referenced subject, object, or scene in the target video.
96
+ - **High-motion scenes**: 50 fps is recommended for smoother motion coherence.
97
+ - **Sampling**: Complex multi-subject interactions may still benefit from multiple sampling runs.
98
 
99
+ ## V1 vs. V2 Comparison
100
 
101
+ ### Comparison 1
102
 
103
+ | V1 | V2 |
104
  |:---:|:---:|
105
+ | [▶ Play V1](Validition_V2/01/V1_1.mp4) | [▶ Play V2](Validition_V2/01/V2.mp4) |
 
 
106
 
107
+ ### Comparison 2
108
+
109
+ | V1 | V2 |
110
+ |:---:|:---:|
111
+ | [▶ Play V1](Validition_V2/02/V1.mp4) | [▶ Play V2](Validition_V2/02/V2.mp4)