README.md

---
title: MTEB-BR Leaderboard
emoji: 🏆
colorFrom: green
colorTo: yellow
sdk: static
pinned: true
license: apache-2.0
short_description: Massive Text Embedding Benchmark for Brazilian Portuguese
thumbnail: https://mteb-br.org/img/og.png
tags:
  - leaderboard
  - benchmark
  - embeddings
  - portuguese
  - brazilian-portuguese
  - mteb
  - retrieval
  - sentence-transformers
datasets:
  - MTEB-BR/mteb-pt-results
models:
  - Qwen/Qwen3-Embedding-8B
  - tencent/KaLM-Embedding-Gemma3-12B-2511
  - Octen/Octen-Embedding-8B
  - Qwen/Qwen3-Embedding-4B
  - Salesforce/SFR-Embedding-Mistral
  - BidirLM/BidirLM-1.7B-Embedding
  - ICT-TIME-and-Querit/BOOM_4B_v1
  - google/embeddinggemma-300m
  - Linq-AI-Research/Linq-Embed-Mistral
  - jinaai/jina-embeddings-v5-text-small
  - intfloat/multilingual-e5-large-instruct
  - Salesforce/SFR-Embedding-2_R
  - Alibaba-NLP/gte-Qwen2-7B-instruct
  - microsoft/harrier-oss-v1-27b
  - BidirLM/BidirLM-1B-Embedding
  - codefuse-ai/F2LLM-v2-8B
  - microsoft/harrier-oss-v1-0.6b
  - codefuse-ai/F2LLM-v2-14B
  - codefuse-ai/F2LLM-v2-4B
  - SamilPwC-AXNode-GenAI/PwC-Embedding_expr
  - Octen/Octen-Embedding-0.6B
  - telepix/PIXIE-Rune-v1.0
  - Qwen/Qwen3-Embedding-0.6B
  - Alibaba-NLP/gte-Qwen2-1.5B-instruct
  - Snowflake/snowflake-arctic-embed-l-v2.0
  - BAAI/bge-m3
  - codefuse-ai/F2LLM-v2-1.7B
  - BidirLM/BidirLM-0.6B-Embedding
  - ufca-llms/jua-4B-mixed
  - codefuse-ai/F2LLM-0.6B
  - microsoft/harrier-oss-v1-270m
  - codefuse-ai/F2LLM-v2-0.6B
  - intfloat/multilingual-e5-large
  - ibm-granite/granite-embedding-311m-multilingual-r2
  - ufca-llms/jua-4B-legal-only
  - codefuse-ai/F2LLM-v2-330M
  - intfloat/multilingual-e5-base
  - PORTULAN/serafim-100m-portuguese-pt-sentence-encoder-ir
  - PORTULAN/serafim-335m-portuguese-pt-sentence-encoder-ir
  - intfloat/multilingual-e5-small
  - ibm-granite/granite-embedding-107m-multilingual
  - ibm-granite/granite-embedding-97m-multilingual-r2
  - PORTULAN/serafim-900m-portuguese-pt-sentence-encoder-ir
  - PORTULAN/serafim-335m-portuguese-pt-sentence-encoder
  - PORTULAN/serafim-900m-portuguese-pt-sentence-encoder
  - codefuse-ai/F2LLM-v2-160M
  - sentence-transformers/paraphrase-multilingual-mpnet-base-v2
  - PORTULAN/serafim-100m-portuguese-pt-sentence-encoder
  - ufca-llms/Qwen3-Embedding-0.6B-jua-V2
  - codefuse-ai/F2LLM-v2-80M
  - sentence-transformers/LaBSE
  - lfcc/medlink-bi-encoder
  - intfloat/e5-small-v2
  - mixedbread-ai/mxbai-embed-large-v1
  - BAAI/bge-small-en-v1.5
  - neuralmind/bert-large-portuguese-cased
  - thenlper/gte-small
  - neuralmind/bert-base-portuguese-cased
  - stjiris/bert-large-portuguese-cased-legal-mlm-sts-v1.0
  - intfloat/e5-mistral-7b-instruct
  - nvidia/llama-embed-nemotron-8b
  - rufimelo/Legal-BERTimbau-sts-large
  - stjiris/bert-large-portuguese-cased-legal-mlm-mkd-nli-sts-v1
  - Mihaiii/Ivysaur
  - PORTULAN/albertina-900m-portuguese-ptbr-encoder
  - sentence-transformers/all-MiniLM-L12-v2
  - avsolatorio/GIST-all-MiniLM-L6-v2
  - sentence-transformers/all-mpnet-base-v2
  - ulysses-camara/legal-bert-pt-br
  - sentence-transformers/all-MiniLM-L6-v2
  - rufimelo/Legal-BERTimbau-sts-large-ma-v3
  - stjiris/bert-large-portuguese-cased-legal-tsdae-gpl-nli-sts-MetaKD-v0
  - sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
---

# MTEB-BR — Brazilian Portuguese Massive Text Embedding Benchmark

A Massive Text Embedding Benchmark for **Brazilian Portuguese** — native, not translated.

- **93 models** (73 open-weight + 20 closed commercial APIs) evaluated on **22 native Brazilian-Portuguese tasks** across **7 categories**
- Admits only data created or found in Portuguese; machine-translated benchmarks (e.g. mMARCO, mkqa) are excluded by construction
- Headline metric: **mean_22** (average across all 22 tasks), reported with per-task bootstrap confidence intervals, paired-bootstrap significance, and IRT task discrimination
- Website: [mteb-br.org](https://mteb-br.org) · Paper: [arXiv:2607.04581](https://arxiv.org/abs/2607.04581)
- All raw results: [MTEB-BR/mteb-pt-results](https://huggingface.co/datasets/MTEB-BR/mteb-pt-results) · Source code: [github.com/tardellirs/mteb-br](https://github.com/tardellirs/mteb-br)

## Tasks (22)

| Category | Tasks |
|---|---|
| Classification | HateBR, ToxSyn-PT, FactckBR, PortuLexRRIP |
| Multi-label classification | BrighterEmotion |
| Pair classification | ASSIN-RTE, InferBR |
| Semantic textual similarity (STS) | ASSIN-STS, ASSIN2-STS |
| Clustering | WikipediaPT-Categories, MedPT, JurisTCU, SciELO, StackOverflow-PT |
| Retrieval | Quati, JurisTCU, BRTaxQA, FaQuAD, MedPT, FaQ-Bacen |
| Reranking | Quati, JurisTCU |

## How to add a model

Submit via GitHub Issues at [tardellirs/mteb-br/issues](https://github.com/tardellirs/mteb-br/issues)
with the model ID, evaluation JSONs, and a reproduction script.

## Citation

```bibtex
@article{stekel2026mtebbr,
  title   = {MTEB-BR: A Text Embedding Benchmark for Brazilian Portuguese},
  author  = {Stekel, Tardelli Ronan Coelho},
  journal = {arXiv preprint arXiv:2607.04581},
  year    = {2026}
}
```