88hours commited on
Commit
976b772
·
1 Parent(s): 04d7dde

Update ReadMe: Understand Vector DB

Browse files
Files changed (1) hide show
  1. README.md +26 -0
README.md CHANGED
@@ -77,3 +77,29 @@ Developed in collaboration with Intel, this model maps image-caption pairs into
77
  - Encodes each frame as a **Base64 image**.
78
  - Extracts context and captions from video frames.
79
  - Uses **Case 1** processing.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
77
  - Encodes each frame as a **Base64 image**.
78
  - Extracts context and captions from video frames.
79
  - Uses **Case 1** processing.
80
+
81
+ # Step 4 - What is LLaVA?
82
+ LLaVA (Large Language-and-Vision Assistant), a large multimodal model that connects a vision encoder that doesn't just see images but understands them, reads the text embedded in them, and reasons about their context—all.
83
+
84
+ # Step 5 - what is a vector Store?
85
+ A vector store is a specialized database designed to:
86
+
87
+ - Store and manage high-dimensional vector data efficiently
88
+ - Perform similarity-based searches where K=1 returns the most similar result
89
+
90
+ - In LanceDB specifically, store multiple data types:
91
+ . Text content (captions)
92
+ . Image file paths
93
+ . Metadata
94
+ . Vector embeddings
95
+
96
+ ```python
97
+ _ = MultimodalLanceDB.from_text_image_pairs(
98
+ texts=updated_vid1_trans+vid2_trans,
99
+ image_paths=vid1_img_path+vid2_img_path,
100
+ embedding=BridgeTowerEmbeddings(),
101
+ metadatas=vid1_metadata+vid2_metadata,
102
+ connection=db,
103
+ table_name=TBL_NAME,
104
+ mode="overwrite",
105
+ )```