README.md

metadata

language:
  - en
license: apache-2.0
tags:
  - glove
  - lora
  - distillation
  - bpe
  - cl100k_base
base_model: jsanzolac/bpe_glove_512
datasets:
  - jsanzolac/qwen3_emb_512
  - jsanzolac/qwen3_emb_512_packed

bpe_glove_512_lora_v1

LoRA drifts on top of frozen jsanzolac/bpe_glove_512 BPE-GloVe-512 embeddings, distilled from Qwen/Qwen3-Embedding-8B (MRL-truncated to 512 dims).

Variant 1 loss: λ_c·InfoNCE + λ_D·||ρ_T - ρ_S||_F² with λ_c=1.0, λ_D=0.1.

Each rank_<r>/ folder contains:

• checkpoint_final.pt
• config.json
• vectors_drifted.txt
• train_log.jsonl

Ranks shipped: [512, 256, 128, 64, 32, 16, 8, 4, 2]