BM-K commited on
Commit
f75d2c6
·
verified ·
1 Parent(s): 2dc8cb4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -7
README.md CHANGED
@@ -71,7 +71,7 @@ Descriptions of the benchmark datasets used for evaluation are as follows:
71
  > **Note:**
72
  > While many benchmark datasets are available for evaluation, in this project we chose to use only those that contain clean positive documents for each query. Keep in mind that a benchmark dataset is just that a benchmark. For real-world applications, it is best to construct an evaluation dataset tailored to your specific domain and evaluate embedding models, such as PIXIE, in that environment to determine the most suitable one.
73
 
74
- #### 7 Datasets of BEIR (English)
75
  Our model, **telepix/PIXIE-Spell-Reranker-Preview-0.6B**, achieves strong performance on a wide range of tasks, including fact verification, multi-hop question answering, financial QA, and scientific document retrieval, demonstrating competitive generalization across diverse domains.
76
 
77
  | Model Name | # params | Avg. NDCG | NDCG@1 | NDCG@3 | NDCG@5 | NDCG@10 |
@@ -87,16 +87,10 @@ Our model, **telepix/PIXIE-Spell-Reranker-Preview-0.6B**, achieves strong perfor
87
  Descriptions of the benchmark datasets used for evaluation are as follows:
88
  - **ArguAna**
89
  A dataset for argument retrieval based on claim-counterclaim pairs from online debate forums.
90
- - **FEVER**
91
- A fact verification dataset using Wikipedia for evidence-based claim validation.
92
  - **FiQA-2018**
93
  A retrieval benchmark tailored to the finance domain with real-world questions and answers.
94
- - **HotpotQA**
95
- A multi-hop open-domain QA dataset requiring reasoning across multiple documents.
96
  - **MSMARCO**
97
  A large-scale benchmark using real Bing search queries and corresponding web documents.
98
- - **NQ**
99
- A Google QA dataset where user questions are answered using Wikipedia articles.
100
  - **SCIDOCS**
101
  A citation-based document retrieval dataset focused on scientific papers.
102
 
 
71
  > **Note:**
72
  > While many benchmark datasets are available for evaluation, in this project we chose to use only those that contain clean positive documents for each query. Keep in mind that a benchmark dataset is just that a benchmark. For real-world applications, it is best to construct an evaluation dataset tailored to your specific domain and evaluate embedding models, such as PIXIE, in that environment to determine the most suitable one.
73
 
74
+ #### 4 Datasets of BEIR (English)
75
  Our model, **telepix/PIXIE-Spell-Reranker-Preview-0.6B**, achieves strong performance on a wide range of tasks, including fact verification, multi-hop question answering, financial QA, and scientific document retrieval, demonstrating competitive generalization across diverse domains.
76
 
77
  | Model Name | # params | Avg. NDCG | NDCG@1 | NDCG@3 | NDCG@5 | NDCG@10 |
 
87
  Descriptions of the benchmark datasets used for evaluation are as follows:
88
  - **ArguAna**
89
  A dataset for argument retrieval based on claim-counterclaim pairs from online debate forums.
 
 
90
  - **FiQA-2018**
91
  A retrieval benchmark tailored to the finance domain with real-world questions and answers.
 
 
92
  - **MSMARCO**
93
  A large-scale benchmark using real Bing search queries and corresponding web documents.
 
 
94
  - **SCIDOCS**
95
  A citation-based document retrieval dataset focused on scientific papers.
96