---
language: en
license: apache-2.0
base_model: BAAI/bge-large-en-v1.5
tags:
  - sentence-transformers
  - feature-extraction
  - sentence-similarity
  - information-retrieval
  - sutd
pipeline_tag: sentence-similarity
---

# sutd-bge-large-aug98

Finetuned [`BAAI/bge-large-en-v1.5`](https://huggingface.co/BAAI/bge-large-en-v1.5) for job-description-to-module retrieval in the **SUTD Course Recommendation Chatbot** (MLOps Group 9). This is the production embedding model used in the final Hybrid pipeline.

Given a job description, this model retrieves relevant SUTD elective modules. It is the dense retrieval backbone in the RAG and Hybrid pipelines.

## Model Details

| Property | Value |
|---|---|
| Base model | `BAAI/bge-large-en-v1.5` |
| Embedding dimension | 1024 |
| Max sequence length | 512 |
| Similarity function | Cosine |
| Loss | `MultipleNegativesRankingLoss` |

## Training Data

Trained on **98 (job description, relevant SUTD module) pairs** spanning four pillars: ASD, EPD, ESD, ISTD/CSD:

- **67 hand-annotated pairs** — original dataset
- **31 augmented pairs** — sourced from MyCareersFuture postings, labelled by Claude

After train/validation splitting and hard-negative expansion by the Sentence Transformers trainer, this produces **831 training samples** and **146 validation samples**.

A version trained on the smaller 67-pair dataset is available at [`henreads/sutd-bge-large-ft67`](https://huggingface.co/henreads/sutd-bge-large-ft67).

## Training Setup

- **Hardware:** Modal A10G (24 GB VRAM)
- **Epochs:** up to 10 with early stopping (patience 4); converged at epoch 3
- **Effective batch size:** 16 (per-device batch 4, gradient accumulation 4)
- **Learning rate:** 2e-5
- **Tracking:** Weights & Biases (`sutd-mlops-bge-finetune`)

## Evaluation

Evaluated on a 10-job held-out retrieval set (completely separate from training). This model achieves the best retrieval NDCG@10 among all embedding configurations.

| Model | NDCG@10 |
|---|---|
| BGE-large-en-v1.5 (base) | 0.679 |
| sutd-bge-large-ft67 | 0.747 |
| **sutd-bge-large-aug98 (this model)** | **0.770** |

When paired with the matched [`henreads/sutd-reranker-aug98`](https://huggingface.co/henreads/sutd-reranker-aug98), the Hybrid pipeline achieves a Gemini LLM-as-judge chat quality score of **4.06 / 5.0**, exceeding the project success criterion of 4.0.

## Usage

```python
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("henreads/sutd-bge-large-aug98")

job_description = "Data Scientist at GovTech. Build ML models with Python..."
module_passage = "50.007 Machine Learning — Topics: supervised learning, neural networks..."

embeddings = model.encode([job_description, module_passage], normalize_embeddings=True)
similarity = embeddings[0] @ embeddings[1]
print(similarity)
```

## Project

Part of the SUTD Course Recommendation Chatbot — MLOps Group 9.  
Code: [github.com/henreads/sutd-mlops-group9](https://github.com/henreads/sutd-mlops-group9)