---
language: en
license: other
license_name: polyform-noncommercial-1.0.0
license_link: https://polyformproject.org/licenses/noncommercial/1.0.0
library_name: sentence-transformers
tags:
  - sentence-transformers
  - embedding
  - emotion
  - memory
  - neuroscience
  - vividness
  - companion-ai
  - pad-model
  - reconsolidation
  - fine-tuned
base_model: sentence-transformers/all-MiniLM-L6-v2
pipeline_tag: sentence-similarity
datasets: []
metrics:
  - f1
  - accuracy
model-index:
  - name: all-MiniLLM-VividTuned
    results:
      - task:
          type: memory-retrieval
          name: MemGPT/Letta EmbedBench
        metrics:
          - name: Tool Accuracy
            type: accuracy
            value: 0.4400
          - name: F1 Score
            type: f1
            value: 0.5151
          - name: BLEU-1
            type: bleu
            value: 0.6660
---

# all-MiniLLM-VividTuned

A purpose-built fine-tuned embedding model for **emotion-aware memory retrieval** in AI companions. Based on [all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2), fine-tuned with 10 specialised objectives to natively encode emotion, importance, narrative arcs, and emotional transitions into the embedding space.

**GitHub:** [Kronic90/VividEmbed](https://github.com/Kronic90/VividEmbed)
**PyPI:** [pip install vividembed](https://pypi.org/project/vividembed/)

## What This Model Does

Standard sentence embedding models produce vectors that capture *what* was said but nothing about *how it felt*. This model learns to differentiate memories by emotional context, not just semantic content.

It does this through **58 special tokens** added during fine-tuning:

```
[EMO:happy] [IMP:8] [ARC:climax] [FROM:anxious] I finally got the promotion!
```

| Token Type | Examples | Purpose |
|-----------|---------|---------|
| `[EMO:x]` | `[EMO:happy]`, `[EMO:anxious]` | Encode the emotion of the memory |
| `[IMP:x]` | `[IMP:1]` through `[IMP:10]` | Encode importance (1-10 scale) |
| `[ARC:x]` | `[ARC:setup]`, `[ARC:climax]` | Encode narrative arc position |
| `[FROM:x]` | `[FROM:calm]`, `[FROM:sad]` | Encode emotional transition (previous state) |
| `[MOOD:x]` | `[MOOD:happy]` | Encode query-time mood for retrieval |
| `[QUERY]` | `[QUERY]` | Query-mode flag |

This means a memory about "I got the promotion" encoded with `[EMO:happy]` produces a **different vector** than the same text encoded with `[EMO:anxious]` — and the model has learned what that emotional difference means geometrically.

## How It Compares to Standard Embeddings

On the MemGPT/Letta EmbedBench benchmark (500 evaluations, 5 seeds):

| Metric | Standard all-MiniLM-L6-v2 | **all-MiniLLM-VividTuned** | Leading Memory System |
|--------|---------------------------|----------------------------|----------------------|
| **Tool Accuracy** | 0.4320 | **0.4400** (+1.9%) | 0.4300 |
| **F1 Score** | 0.5148 | **0.5151** (+0.1%) | 0.4945 |
| **BLEU-1** | 0.6338 | **0.6660** (+5.1%) | 0.6310 |

The fine-tuned model outperforms both the base model it was built on *and* leading memory systems — with the same 22M parameters and no cloud APIs.

### What the Numbers Mean

- **Tool Accuracy:** How often the system uses the right memory tool for the task
- **F1 Score:** Precision-recall balance on memory retrieval relevance
- **BLEU-1:** How well retrieved memories match expected content (unigram overlap)

## Model Details

| Property | Value |
|----------|-------|
| Base model | [all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) |
| Parameters | **22M** |
| Output dimension | **384** |
| Max sequence length | 256 tokens |
| Special tokens added | **58** |
| Training objectives | **10** |
| Training examples | **~35,000** |
| Final training loss | **0.0208** |
| Format | SafeTensors |

### Training Objectives

The model was trained on 10 simultaneous objectives:

1. **Emotion clustering** — same-emotion memories cluster together
2. **Cross-emotion separation** — different-emotion memories separate
3. **Semantic similarity** — semantically similar content stays close
4. **Importance ordering** — higher importance → larger vector magnitude
5. **Mood-congruent retrieval** — queries match mood-aligned memories
6. **Emotional transition encoding** — `[FROM:x]` tokens differentiate transition context
7. **Narrative arc structure** — arc positions produce distinct vectors
8. **Pattern separation** — near-duplicate memories are de-correlated
9. **Contradiction detection** — contradicting memories have low similarity
10. **Entity grounding** — entity-specific facts remain retrievable

### Key Design Choice: Vector Magnitude Encodes Importance

Unlike the base model which L2-normalises everything, **this model preserves vector magnitude**. High-importance memories (`[IMP:9]`) produce longer vectors than low-importance ones (`[IMP:2]`). The VividEmbed scoring function uses this:

```python
viv_signal = vector_norm / 5.0  # normalised to baseline MiniLM magnitude
score = cos_sim × (0.7 + 0.3 × viv_signal × decay) + 0.1 × recency
```

## Usage

### With VividEmbed (Recommended)

```python
pip install vividembed
```

```python
from vividembed import VividEmbed

# Automatically uses VividTuned if found, falls back to base model
ve = VividEmbed(model_name="Kronic90/all-MiniLLM-VividTuned")

ve.add("I finally got the job offer!", emotion="excited", importance=9)
ve.add("The rejection letter was devastating", emotion="sad", importance=8)

# Mood-congruent retrieval
results = ve.query("career milestones", mood="happy", top_k=3)
```

### With sentence-transformers Directly

```python
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("Kronic90/all-MiniLLM-VividTuned")

# Encode with emotion tokens
vec1 = model.encode("[EMO:happy] [IMP:8] I got the promotion!")
vec2 = model.encode("[EMO:sad] [IMP:8] I got the promotion!")

# These produce DIFFERENT vectors — the model understands emotional context
import numpy as np
cos_sim = np.dot(vec1, vec2) / (np.linalg.norm(vec1) * np.linalg.norm(vec2))
print(f"Same text, different emotion: similarity = {cos_sim:.3f}")
# Expect < 1.0 — emotion shifts the embedding
```

### Checking If Tokens Are Active

```python
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("Kronic90/all-MiniLLM-VividTuned")
tokenizer = model.tokenizer

# Verify emotion tokens are in vocabulary
has_vivid = "[EMO:happy]" in tokenizer.get_vocab()
print(f"VividTuned tokens active: {has_vivid}")  # True
```

## Neuroscience Foundations

This model implements concepts from cognitive neuroscience:

- **PAD Emotion Model** (Mehrabian, 1996) — 76 emotions mapped to Pleasure-Arousal-Dominance space
- **Memory Reconsolidation** (Nader et al., 2000) — recalled memories subtly shift toward retrieval context
- **Hippocampal Pattern Separation** — near-duplicate memories are actively de-correlated
- **Narrative Arc Theory** — episodic memories organised along 5-act story structures
- **Affect-as-Information** — emotional state during encoding influences retrieval

See the [VividEmbed GitHub repo](https://github.com/Kronic90/VividEmbed) for full architectural documentation, 190/190 passing tests, and 17 visual reports proving each mechanism works.

## Checkpoints

This repository contains two checkpoints:

| Checkpoint | Description |
|-----------|-------------|
| `best/` | Best validation loss during training (recommended) |
| `final/` | End-of-training checkpoint |

Use `best/` for production. Use `final/` if you want to continue fine-tuning.

## License

**PolyForm Noncommercial 1.0.0** — free for personal, research, educational, and non-profit use. Commercial use requires a separate agreement.

Contact: [@Kronic90](https://github.com/Kronic90)

## Citation

```bibtex
@software{vividembed2026,
  title   = {VividEmbed: Neuroscience-Inspired Memory Embeddings for AI Companions},
  author  = {Kronic90},
  year    = {2026},
  url     = {https://github.com/Kronic90/VividEmbed}
}
```