Upload hybrid model for hotpotqa (trained on 2000 queries)

README.md

@@ -28,25 +28,25 @@ model-index:
       type: validation
     metrics:
     - type: accuracy
-      value: 0.99
+      value: 0.9933333333333333
       name: Accuracy
     - type: accuracy_threshold
-      value: 0.0021620974875986576
+      value: 0.16204833984375
       name: Accuracy Threshold
     - type: f1
-      value: 0.9900332225913622
+      value: 0.9932885906040269
       name: F1
     - type: f1_threshold
-      value: 0.0011168117634952068
+      value: 0.16204833984375
       name: F1 Threshold
     - type: precision
-      value: 0.9867549668874173
+      value: 1.0
       name: Precision
     - type: recall
-      value: 0.9933333333333333
+      value: 0.9866666666666667
       name: Recall
     - type: average_precision
-      value: 0.9995697171348688
+      value: 0.9992144440924928
       name: Average Precision
   - task:
       type: cross-encoder-binary-classification
@@ -56,25 +56,25 @@ model-index:
       type: train_subset
     metrics:
     - type: accuracy
-      value: 0.926
+      value: 0.976
       name: Accuracy
     - type: accuracy_threshold
-      value: 0.6817218065261841
+      value: 0.599609375
       name: Accuracy Threshold
     - type: f1
-      value: 0.9270216962524654
+      value: 0.9770992366412214
       name: F1
     - type: f1_threshold
-      value: 0.007857910357415676
+      value: 0.599609375
       name: F1 Threshold
     - type: precision
-      value: 0.9288537549407114
+      value: 0.973384030418251
       name: Precision
     - type: recall
-      value: 0.9251968503937008
+      value: 0.9808429118773946
       name: Recall
     - type: average_precision
-      value: 0.9650983248932256
+      value: 0.9928588723194676
       name: Average Precision
 ---
 
@@ -118,11 +118,11 @@ from sentence_transformers import CrossEncoder
 model = CrossEncoder("cross_encoder_model_id")
 # Get scores for pairs of texts
 pairs = [
-    ['What nationality was the person whose music inspired the Folias Flute and Guitar Duo ?', 'Kenneth Ascher. Kenneth Lee "Kenny" Ascher (born October 26, 1944 in Washington, D.C.) is an American jazz pianist, composer, and arranger who is active in jazz, rock, classical, and musical theater genres — in live venues, recording studios, and cinema production.  He is widely known for co-writing, with Paul Williams, "Rainbow Connection" from "The Muppet Movie".  Both Williams and Ascher received Oscar nominations for the 1979 Academy Awards for Best Original Song ("Rainbow Connection") and Best Original Score ("The Muppet Movie" Soundtrack).  The song was also nominated for the Golden Globes for "Best Original Song" that same year.'],
-    ['Orff Schulwerk was developed by the composer who was best known for what cantata?', 'Isabel McNeill Carley. Isabel McNeill Carley (4 December 1918 – 14 July 2011) was a published writer, editor, composer and music teacher.  She’s considered one of the leaders of the Orff Schulwerk when it began to take hold in the United States in the 1960s.  As a co-founder of the American Orff-Schulwerk Association (AOSA), Carley contributed greatly to the organization’s beginnings, serving as a board member and magazine editor.  Carley devoted much of her life to musical instruction, publishing a series of books titled Recorder Improvisation and Technique.'],
-    ['What country does Pioneer Canal and Orleans Canal have in common?', 'Same-sex marriage in the Sixth Circuit. On April 28, 2015, the Supreme Court of the United States heard oral arguments for "Obergefell v. Hodges" (Ohio), which was consolidated with three other same-sex marriage cases from the other states in the Sixth Circuit: "Tanco v. Haslam" (Tennessee), "DeBoer v. Snyder" (Michigan), "Bourke v. Beshear" (Kentucky).  On June 26, 2015 the Supreme Court reversed the Sixth Circuit\'s decision, paving the way for same-sex marriage to become legal in those states, and setting a precedent for the entire nation.  All four states complied with the ruling the same day it was issued before the mandate was actually issued.  Every state in the circuit had a district court ruling against their states\' ban, but they were eventually stayed pending appeal.  The Sixth Circuit consists of Kentucky, Michigan, Ohio, and Tennessee.  On August 6, 2014, the Sixth Circuit heard oral arguments for same-sex marriage cases from each state within the circuit.  On November 6, 2014, the Sixth Circuit in a split 2-1 decision, upheld the states\' same-sex marriage bans, reversing the district courts\' rulings that struck them down.  The Sixth Circuit was the first and only circuit court since the landmark ruling "United States v. Windsor" to uphold the constitutionality of states\' same-sex marriage bans which caused a circuit split.'],
-    ['In what year was the actor who plays an anxious novelist in the movie Ruby Sparks born?', 'Rufus Sewell. Rufus Frederik Sewell ( ; born 29 October 1967) is an English actor.  In film, he has appeared in Kenneth Branagh\'s rendition of "Hamlet" (1996) playing Fortinbras, "The Woodlanders", "Dangerous Beauty", "Dark City", "A Knight\'s Tale", "The Illusionist", "Tristan and Isolde", and "Martha, Meet Frank, Daniel and Laurence".  On television, he starred as "Obergruppenführer" John Smith in "The Man in the High Castle", an Italian detective in the BBC\'s television series "Zen" (2011) and also appeared in the mini-series "The Pillars of the Earth" (2010).  In 1993 he played the hero, Will Ladislaw, in the BBC\'s adaptation of George Eliot\'s "Middlemarch".  In 2003, he appeared in the lead role in "".  He starred in the CBS drama "Eleventh Hour", which was cancelled in April 2009.  On stage, he originated the role of Septimus Hodge in Tom Stoppard\'s "Arcadia" and the role of Jan in Stoppard\'s "Rock \'n\' Roll"; the latter earned him an Olivier Award and a Tony Award nomination.'],
-    ["Brent Ferrin Ashworth appeared on Beck's special show on collecting, along with a Las Vegas-based businessman and reality television personality who made $2,000 a week selling what?", 'Marlo Stanfield. Marlo Stanfield is a fictional character on the HBO television drama "The Wire", played by actor Jamie Hector.  Stanfield is a young, ambitious, intelligent and extremely ruthless player and is head of the eponymous Stanfield Organization in the Baltimore drug trade.  Marlo\'s organization starts out small-time, operating in the vacuum left by the Barksdale Organization, and rises to the top of the Baltimore drug trade fairly quickly.  A repeated theme in Marlo\'s characterization is his demand for unconditional respect, which trumps all other concerns.  He frequently orders the deaths of those who disrespect him or undermine his name on the streets, however unwittingly.  He is arguably the most violent and ruthless of the drug kingpins portrayed in "The Wire"; In 2016, "Rolling Stone" ranked him #2 of their "40 Greatest TV Villains of All Time".'],
+    ['Paloma and Monkey Gland, are what type of item?', "Monkey Gland. The Monkey Gland is a cocktail of gin, orange juice, grenadine and absinthe created in the 1920s by Harry MacElhone, owner of Harry's New York Bar in Paris, France."],
+    ['Tess Asplund fought against the Neo-Nazi movement that existed in which countries?', 'Tess Asplund. Tess Asplund, born 1974, is a Swedish activist who gained attention following her protest against neo-Nazis in Borlänge, Sweden.  David Lagerlof is the photographer of the viral image of Asplund, which shows her facing uniformed members of the Swedish Nordic Resistance Movement with her fist in the air.  She is originally from Colombia and describes herself as Afro-Swedish.  About the incident, Asplund is quoted as having said “If this picture of me can get more people to dare to show resistance, then it’s all good...the people must unite and show that it is not okay that racism is becoming normalised and that fascists are running around on our streets.”'],
+    ['Which magazine, Arizona Highways or Adventist World, is published by Herald Publishing Association?', 'My Worlds: The Collection. My Worlds: The Collection is the first compilation album released by Canadian recording artist Justin Bieber.  As the international alternative to the Walmart and Sam\'s Club exclusive "My Worlds Acoustic" (2010), "My Worlds: The Collection" was released in numerous European countries on November 19, 2010.  The album consists of two discs; the first is a slightly altered version of "My Worlds Acoustic", and the second is "My Worlds", a compilation itself made up of "My World" (2009) and "My World 2.0" (2010).  In addition, the album also features a new song, an inspirational ballad entitled "Pray", a Jaden Smith collaboration, "Never Say Never", and remixes of "Somebody to Love".  The new versions of the songs were produced by Bieber\'s music director, Dan Kanter, his vocal producer Kuk Harrell, and also producer Rob Wells.  While most reviewers complimented the set , several thought that its release was unneeded.  The album charted moderately in Europe, reaching the top half of several album charts.'],
+    ['Who made the sculpture of the an American professional baseball player and manager regarded as one of the greatest players in baseball history?', 'Ted Williams. Theodore Samuel Williams (August 30, 1918 – July 5, 2002) was an American professional baseball player and manager.  He played his entire 19-year Major League Baseball (MLB) career as a left fielder for the Boston Red Sox from 1939 to 1960, only interrupted by service time during World War II and the Korean War.  Nicknamed "The Kid", "The Splendid Splinter", "Teddy Ballgame", "The Thumper", and "The Greatest Hitter Who Ever Lived", Williams is regarded as one of the greatest players in baseball history.  Williams was also an outstanding fielder, especially in the difficult left field of Fenway Park in Boston, where he played his entire Major League career at that position.'],
+    ['Paul-Werner Krapke is notable for his management of a main battle tank developed by who?', 'Paul-Werner Krapke. Paul-Werner Krapke (born 1915) is a German armored fighting vehicle engineer, notable for his management of the Leopard 2 project.'],
 ]
 scores = model.predict(pairs)
 print(scores.shape)
@@ -130,13 +130,13 @@ print(scores.shape)
 
 # Or rank different texts based on similarity to a single text
 ranks = model.rank(
-    'What nationality was the person whose music inspired the Folias Flute and Guitar Duo ?',
+    'Paloma and Monkey Gland, are what type of item?',
     [
-        'Kenneth Ascher. Kenneth Lee "Kenny" Ascher (born October 26, 1944 in Washington, D.C.) is an American jazz pianist, composer, and arranger who is active in jazz, rock, classical, and musical theater genres — in live venues, recording studios, and cinema production.  He is widely known for co-writing, with Paul Williams, "Rainbow Connection" from "The Muppet Movie".  Both Williams and Ascher received Oscar nominations for the 1979 Academy Awards for Best Original Song ("Rainbow Connection") and Best Original Score ("The Muppet Movie" Soundtrack).  The song was also nominated for the Golden Globes for "Best Original Song" that same year.',
-        'Isabel McNeill Carley. Isabel McNeill Carley (4 December 1918 – 14 July 2011) was a published writer, editor, composer and music teacher.  She’s considered one of the leaders of the Orff Schulwerk when it began to take hold in the United States in the 1960s.  As a co-founder of the American Orff-Schulwerk Association (AOSA), Carley contributed greatly to the organization’s beginnings, serving as a board member and magazine editor.  Carley devoted much of her life to musical instruction, publishing a series of books titled Recorder Improvisation and Technique.',
-        'Same-sex marriage in the Sixth Circuit. On April 28, 2015, the Supreme Court of the United States heard oral arguments for "Obergefell v. Hodges" (Ohio), which was consolidated with three other same-sex marriage cases from the other states in the Sixth Circuit: "Tanco v. Haslam" (Tennessee), "DeBoer v. Snyder" (Michigan), "Bourke v. Beshear" (Kentucky).  On June 26, 2015 the Supreme Court reversed the Sixth Circuit\'s decision, paving the way for same-sex marriage to become legal in those states, and setting a precedent for the entire nation.  All four states complied with the ruling the same day it was issued before the mandate was actually issued.  Every state in the circuit had a district court ruling against their states\' ban, but they were eventually stayed pending appeal.  The Sixth Circuit consists of Kentucky, Michigan, Ohio, and Tennessee.  On August 6, 2014, the Sixth Circuit heard oral arguments for same-sex marriage cases from each state within the circuit.  On November 6, 2014, the Sixth Circuit in a split 2-1 decision, upheld the states\' same-sex marriage bans, reversing the district courts\' rulings that struck them down.  The Sixth Circuit was the first and only circuit court since the landmark ruling "United States v. Windsor" to uphold the constitutionality of states\' same-sex marriage bans which caused a circuit split.',
-        'Rufus Sewell. Rufus Frederik Sewell ( ; born 29 October 1967) is an English actor.  In film, he has appeared in Kenneth Branagh\'s rendition of "Hamlet" (1996) playing Fortinbras, "The Woodlanders", "Dangerous Beauty", "Dark City", "A Knight\'s Tale", "The Illusionist", "Tristan and Isolde", and "Martha, Meet Frank, Daniel and Laurence".  On television, he starred as "Obergruppenführer" John Smith in "The Man in the High Castle", an Italian detective in the BBC\'s television series "Zen" (2011) and also appeared in the mini-series "The Pillars of the Earth" (2010).  In 1993 he played the hero, Will Ladislaw, in the BBC\'s adaptation of George Eliot\'s "Middlemarch".  In 2003, he appeared in the lead role in "".  He starred in the CBS drama "Eleventh Hour", which was cancelled in April 2009.  On stage, he originated the role of Septimus Hodge in Tom Stoppard\'s "Arcadia" and the role of Jan in Stoppard\'s "Rock \'n\' Roll"; the latter earned him an Olivier Award and a Tony Award nomination.',
-        'Marlo Stanfield. Marlo Stanfield is a fictional character on the HBO television drama "The Wire", played by actor Jamie Hector.  Stanfield is a young, ambitious, intelligent and extremely ruthless player and is head of the eponymous Stanfield Organization in the Baltimore drug trade.  Marlo\'s organization starts out small-time, operating in the vacuum left by the Barksdale Organization, and rises to the top of the Baltimore drug trade fairly quickly.  A repeated theme in Marlo\'s characterization is his demand for unconditional respect, which trumps all other concerns.  He frequently orders the deaths of those who disrespect him or undermine his name on the streets, however unwittingly.  He is arguably the most violent and ruthless of the drug kingpins portrayed in "The Wire"; In 2016, "Rolling Stone" ranked him #2 of their "40 Greatest TV Villains of All Time".',
+        "Monkey Gland. The Monkey Gland is a cocktail of gin, orange juice, grenadine and absinthe created in the 1920s by Harry MacElhone, owner of Harry's New York Bar in Paris, France.",
+        'Tess Asplund. Tess Asplund, born 1974, is a Swedish activist who gained attention following her protest against neo-Nazis in Borlänge, Sweden.  David Lagerlof is the photographer of the viral image of Asplund, which shows her facing uniformed members of the Swedish Nordic Resistance Movement with her fist in the air.  She is originally from Colombia and describes herself as Afro-Swedish.  About the incident, Asplund is quoted as having said “If this picture of me can get more people to dare to show resistance, then it’s all good...the people must unite and show that it is not okay that racism is becoming normalised and that fascists are running around on our streets.”',
+        'My Worlds: The Collection. My Worlds: The Collection is the first compilation album released by Canadian recording artist Justin Bieber.  As the international alternative to the Walmart and Sam\'s Club exclusive "My Worlds Acoustic" (2010), "My Worlds: The Collection" was released in numerous European countries on November 19, 2010.  The album consists of two discs; the first is a slightly altered version of "My Worlds Acoustic", and the second is "My Worlds", a compilation itself made up of "My World" (2009) and "My World 2.0" (2010).  In addition, the album also features a new song, an inspirational ballad entitled "Pray", a Jaden Smith collaboration, "Never Say Never", and remixes of "Somebody to Love".  The new versions of the songs were produced by Bieber\'s music director, Dan Kanter, his vocal producer Kuk Harrell, and also producer Rob Wells.  While most reviewers complimented the set , several thought that its release was unneeded.  The album charted moderately in Europe, reaching the top half of several album charts.',
+        'Ted Williams. Theodore Samuel Williams (August 30, 1918 – July 5, 2002) was an American professional baseball player and manager.  He played his entire 19-year Major League Baseball (MLB) career as a left fielder for the Boston Red Sox from 1939 to 1960, only interrupted by service time during World War II and the Korean War.  Nicknamed "The Kid", "The Splendid Splinter", "Teddy Ballgame", "The Thumper", and "The Greatest Hitter Who Ever Lived", Williams is regarded as one of the greatest players in baseball history.  Williams was also an outstanding fielder, especially in the difficult left field of Fenway Park in Boston, where he played his entire Major League career at that position.',
+        'Paul-Werner Krapke. Paul-Werner Krapke (born 1915) is a German armored fighting vehicle engineer, notable for his management of the Leopard 2 project.',
     ]
 )
 # [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]
@@ -177,13 +177,13 @@ You can finetune this model on your own dataset.
 
 | Metric                | validation | train_subset |
 |:----------------------|:-----------|:-------------|
-| accuracy              | 0.99       | 0.926        |
-| accuracy_threshold    | 0.0022     | 0.6817       |
-| f1                    | 0.99       | 0.927        |
-| f1_threshold          | 0.0011     | 0.0079       |
-| precision             | 0.9868     | 0.9289       |
-| recall                | 0.9933     | 0.9252       |
-| **average_precision** | **0.9996** | **0.9651**   |
+| accuracy              | 0.9933     | 0.976        |
+| accuracy_threshold    | 0.162      | 0.5996       |
+| f1                    | 0.9933     | 0.9771       |
+| f1_threshold          | 0.162      | 0.5996       |
+| precision             | 1.0        | 0.9734       |
+| recall                | 0.9867     | 0.9808       |
+| **average_precision** | **0.9992** | **0.9929**   |
 
 <!--
 ## Bias, Risks and Limitations
@@ -209,13 +209,13 @@ You can finetune this model on your own dataset.
   |         | sentence_0                                                                                       | sentence_1                                                                                        | label                                                          |
   |:--------|:-------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------|:---------------------------------------------------------------|
   | type    | string                                                                                           | string                                                                                            | float                                                          |
-  | details | <ul><li>min: 35 characters</li><li>mean: 104.32 characters</li><li>max: 474 characters</li></ul> | <ul><li>min: 92 characters</li><li>mean: 537.42 characters</li><li>max: 2340 characters</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.51</li><li>max: 1.0</li></ul> |
+  | details | <ul><li>min: 35 characters</li><li>mean: 103.16 characters</li><li>max: 498 characters</li></ul> | <ul><li>min: 74 characters</li><li>mean: 550.07 characters</li><li>max: 2501 characters</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.51</li><li>max: 1.0</li></ul> |
 * Samples:
-  | sentence_0                                                                                          | sentence_1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | label            |
-  |:----------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------|
-  | <code>What nationality was the person whose music inspired the Folias Flute and Guitar Duo ?</code> | <code>Kenneth Ascher. Kenneth Lee "Kenny" Ascher (born October 26, 1944 in Washington, D.C.) is an American jazz pianist, composer, and arranger who is active in jazz, rock, classical, and musical theater genres — in live venues, recording studios, and cinema production.  He is widely known for co-writing, with Paul Williams, "Rainbow Connection" from "The Muppet Movie".  Both Williams and Ascher received Oscar nominations for the 1979 Academy Awards for Best Original Song ("Rainbow Connection") and Best Original Score ("The Muppet Movie" Soundtrack).  The song was also nominated for the Golden Globes for "Best Original Song" that same year.</code>                                                                                                                                                                                                                                                                                                                                                                         | <code>0.0</code> |
-  | <code>Orff Schulwerk was developed by the composer who was best known for what cantata?</code>      | <code>Isabel McNeill Carley. Isabel McNeill Carley (4 December 1918 – 14 July 2011) was a published writer, editor, composer and music teacher.  She’s considered one of the leaders of the Orff Schulwerk when it began to take hold in the United States in the 1960s.  As a co-founder of the American Orff-Schulwerk Association (AOSA), Carley contributed greatly to the organization’s beginnings, serving as a board member and magazine editor.  Carley devoted much of her life to musical instruction, publishing a series of books titled Recorder Improvisation and Technique.</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                       | <code>0.0</code> |
-  | <code>What country does Pioneer Canal and Orleans Canal have in common?</code>                      | <code>Same-sex marriage in the Sixth Circuit. On April 28, 2015, the Supreme Court of the United States heard oral arguments for "Obergefell v. Hodges" (Ohio), which was consolidated with three other same-sex marriage cases from the other states in the Sixth Circuit: "Tanco v. Haslam" (Tennessee), "DeBoer v. Snyder" (Michigan), "Bourke v. Beshear" (Kentucky).  On June 26, 2015 the Supreme Court reversed the Sixth Circuit's decision, paving the way for same-sex marriage to become legal in those states, and setting a precedent for the entire nation.  All four states complied with the ruling the same day it was issued before the mandate was actually issued.  Every state in the circuit had a district court ruling against their states' ban, but they were eventually stayed pending appeal.  The Sixth Circuit consists of Kentucky, Michigan, Ohio, and Tennessee.  On August 6, 2014, the Sixth Circuit heard oral arguments for same-sex marriage cases from each state within the circuit.  On November 6, 2...</code> | <code>0.0</code> |
+  | sentence_0                                                                                                       | sentence_1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | label            |
+  |:-----------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------|
+  | <code>Paloma and Monkey Gland, are what type of item?</code>                                                     | <code>Monkey Gland. The Monkey Gland is a cocktail of gin, orange juice, grenadine and absinthe created in the 1920s by Harry MacElhone, owner of Harry's New York Bar in Paris, France.</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          | <code>1.0</code> |
+  | <code>Tess Asplund fought against the Neo-Nazi movement that existed in which countries?</code>                  | <code>Tess Asplund. Tess Asplund, born 1974, is a Swedish activist who gained attention following her protest against neo-Nazis in Borlänge, Sweden.  David Lagerlof is the photographer of the viral image of Asplund, which shows her facing uniformed members of the Swedish Nordic Resistance Movement with her fist in the air.  She is originally from Colombia and describes herself as Afro-Swedish.  About the incident, Asplund is quoted as having said “If this picture of me can get more people to dare to show resistance, then it’s all good...the people must unite and show that it is not okay that racism is becoming normalised and that fascists are running around on our streets.”</code>                                                                                                                                                                                                                                                                                                                                        | <code>1.0</code> |
+  | <code>Which magazine, Arizona Highways or Adventist World, is published by Herald Publishing Association?</code> | <code>My Worlds: The Collection. My Worlds: The Collection is the first compilation album released by Canadian recording artist Justin Bieber.  As the international alternative to the Walmart and Sam's Club exclusive "My Worlds Acoustic" (2010), "My Worlds: The Collection" was released in numerous European countries on November 19, 2010.  The album consists of two discs; the first is a slightly altered version of "My Worlds Acoustic", and the second is "My Worlds", a compilation itself made up of "My World" (2009) and "My World 2.0" (2010).  In addition, the album also features a new song, an inspirational ballad entitled "Pray", a Jaden Smith collaboration, "Never Say Never", and remixes of "Somebody to Love".  The new versions of the songs were produced by Bieber's music director, Dan Kanter, his vocal producer Kuk Harrell, and also producer Rob Wells.  While most reviewers complimented the set , several thought that its release was unneeded.  The album charted moderately in Europe, reachi...</code> | <code>0.0</code> |
 * Loss: [<code>BinaryCrossEntropyLoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#binarycrossentropyloss) with these parameters:
   ```json
   {
@@ -354,13 +354,10 @@ You can finetune this model on your own dataset.
 ### Training Logs
 | Epoch | Step | Training Loss | validation_average_precision | train_subset_average_precision |
 |:-----:|:----:|:-------------:|:----------------------------:|:------------------------------:|
-| 0.025 | 100  | -             | 0.9988                       | 0.9757                         |
-| 0.05  | 200  | -             | 0.9992                       | 0.9766                         |
-| 0.075 | 300  | -             | 0.9997                       | 0.9806                         |
-| 0.1   | 400  | -             | 0.9997                       | 0.8754                         |
-| 0.125 | 500  | 0.4041        | 0.9997                       | 0.9600                         |
-| 0.15  | 600  | -             | 0.9995                       | 0.9732                         |
-| 0.175 | 700  | -             | 0.9996                       | 0.9651                         |
+| 0.125 | 250  | -             | 0.9988                       | 0.9896                         |
+| 0.25  | 500  | 0.2247        | 0.9989                       | 0.9896                         |
+| 0.375 | 750  | -             | 0.9988                       | 0.9918                         |
+| 0.5   | 1000 | 0.2255        | 0.9992                       | 0.9929                         |
 
 
 ### Framework Versions

config.json

@@ -30,7 +30,7 @@
     "activation_fn": "torch.nn.modules.activation.Sigmoid",
     "version": "5.2.2"
   },
-  "torch_dtype": "float32",
+  "torch_dtype": "bfloat16",
   "transformers_version": "4.44.2",
   "type_vocab_size": 1,
   "use_cache": true,

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:32c6c440cedb9d2e88aa61031dd79dd7a0c56e29cd72b3c1c1bde9f225a0e292
-size 2271071852
+oid sha256:43baf54c79f0f0cbd34a6f24f8179ec5397b7c698c356ab5dfc7ef80ef4e0308
+size 1135560090