Update ReadMe: Understand Vector DB
Browse files
README.md
CHANGED
|
@@ -77,3 +77,29 @@ Developed in collaboration with Intel, this model maps image-caption pairs into
|
|
| 77 |
- Encodes each frame as a **Base64 image**.
|
| 78 |
- Extracts context and captions from video frames.
|
| 79 |
- Uses **Case 1** processing.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 77 |
- Encodes each frame as a **Base64 image**.
|
| 78 |
- Extracts context and captions from video frames.
|
| 79 |
- Uses **Case 1** processing.
|
| 80 |
+
|
| 81 |
+
# Step 4 - What is LLaVA?
|
| 82 |
+
LLaVA (Large Language-and-Vision Assistant), a large multimodal model that connects a vision encoder that doesn't just see images but understands them, reads the text embedded in them, and reasons about their context—all.
|
| 83 |
+
|
| 84 |
+
# Step 5 - what is a vector Store?
|
| 85 |
+
A vector store is a specialized database designed to:
|
| 86 |
+
|
| 87 |
+
- Store and manage high-dimensional vector data efficiently
|
| 88 |
+
- Perform similarity-based searches where K=1 returns the most similar result
|
| 89 |
+
|
| 90 |
+
- In LanceDB specifically, store multiple data types:
|
| 91 |
+
. Text content (captions)
|
| 92 |
+
. Image file paths
|
| 93 |
+
. Metadata
|
| 94 |
+
. Vector embeddings
|
| 95 |
+
|
| 96 |
+
```python
|
| 97 |
+
_ = MultimodalLanceDB.from_text_image_pairs(
|
| 98 |
+
texts=updated_vid1_trans+vid2_trans,
|
| 99 |
+
image_paths=vid1_img_path+vid2_img_path,
|
| 100 |
+
embedding=BridgeTowerEmbeddings(),
|
| 101 |
+
metadatas=vid1_metadata+vid2_metadata,
|
| 102 |
+
connection=db,
|
| 103 |
+
table_name=TBL_NAME,
|
| 104 |
+
mode="overwrite",
|
| 105 |
+
)```
|