| --- |
| language: |
| - en |
| license: mit |
| library_name: span-marker |
| tags: |
| - span-marker |
| - token-classification |
| - ner |
| - named-entity-recognition |
| - generated_from_span_marker_trainer |
| datasets: |
| - DFKI-SLT/few-nerd |
| metrics: |
| - precision |
| - recall |
| - f1 |
| widget: |
| - text: The Hebrew Union College libraries in Cincinnati and Los Angeles, the Library |
| of Congress in Washington, D.C ., the Jewish Theological Seminary in New York |
| City, and the Harvard University Library (which received donations of Deinard's |
| texts from Lucius Nathan Littauer, housed in Widener and Houghton libraries) also |
| have large collections of Deinard works. |
| - text: Abu Abd Allah Muhammad al-Idrisi (1099–1165 or 1166), the Moroccan Muslim |
| geographer, cartographer, Egyptologist and traveller who lived in Sicily at the |
| court of King Roger II, mentioned this island, naming it جزيرة مليطمة ("jazīrat |
| Malīṭma", "the island of Malitma ") on page 583 of his book "Nuzhat al-mushtaq |
| fi ihtiraq ghal afaq", otherwise known as The Book of Roger, considered a geographic |
| encyclopaedia of the medieval world. |
| - text: The font is also used in the logo of the American rock band Greta Van Fleet, |
| in the logo for Netflix show "Stranger Things ", and in the album art for rapper |
| Logic's album "Supermarket ". |
| - text: Caretaker manager George Goss led them on a run in the FA Cup, defeating Liverpool |
| in round 4, to reach the semi-final at Stamford Bridge, where they were defeated |
| 2–0 by Sheffield United on 28 March 1925. |
| - text: In 1991, the National Science Foundation (NSF), which manages the U.S . Antarctic |
| Program (US AP), honoured his memory by dedicating a state-of-the-art laboratory |
| complex in his name, the Albert P. Crary Science and Engineering Center (CSEC) |
| located in McMurdo Station. |
| pipeline_tag: token-classification |
| base_model: numind/generic-entity_recognition_NER-v1 |
| model-index: |
| - name: SpanMarker with numind/generic-entity_recognition_NER-v1 on DFKI-SLT/few-nerd |
| results: |
| - task: |
| type: token-classification |
| name: Named Entity Recognition |
| dataset: |
| name: Unknown |
| type: DFKI-SLT/few-nerd |
| split: test |
| metrics: |
| - type: f1 |
| value: 0.7665505226480835 |
| name: F1 |
| - type: precision |
| value: 0.7581967213114754 |
| name: Precision |
| - type: recall |
| value: 0.775090458960198 |
| name: Recall |
| --- |
| |
| # SpanMarker with numind/generic-entity_recognition_NER-v1 on DFKI-SLT/few-nerd |
|
|
| This is a [SpanMarker](https://github.com/tomaarsen/SpanMarkerNER) model trained on the [DFKI-SLT/few-nerd](https://huggingface.co/datasets/DFKI-SLT/few-nerd) dataset that can be used for Named Entity Recognition. This SpanMarker model uses [numind/generic-entity_recognition_NER-v1](https://huggingface.co/numind/generic-entity_recognition_NER-v1) as the underlying encoder. |
|
|
| ## Model Details |
|
|
| ### Model Description |
| - **Model Type:** SpanMarker |
| - **Encoder:** [numind/generic-entity_recognition_NER-v1](https://huggingface.co/numind/generic-entity_recognition_NER-v1) |
| - **Maximum Sequence Length:** 256 tokens |
| - **Maximum Entity Length:** 19 words |
| - **Training Dataset:** [DFKI-SLT/few-nerd](https://huggingface.co/datasets/DFKI-SLT/few-nerd) |
| - **Language:** en |
| - **License:** mit |
|
|
| ### Model Sources |
|
|
| - **Repository:** [SpanMarker on GitHub](https://github.com/tomaarsen/SpanMarkerNER) |
| - **Thesis:** [SpanMarker For Named Entity Recognition](https://raw.githubusercontent.com/tomaarsen/SpanMarkerNER/main/thesis.pdf) |
|
|
| ### Model Labels |
| | Label | Examples | |
| |:-------------|:-------------------------------------------------------------------------------| |
| | art | "Time", "The Seven Year Itch", "Imelda de ' Lambertazzi" | |
| | building | "Boston Garden", "Sheremetyevo International Airport", "Henry Ford Museum" | |
| | event | "Iranian Constitutional Revolution", "Russian Revolution", "French Revolution" | |
| | location | "the Republic of Croatia", "Croatian", "Mediterranean Basin" | |
| | organization | "IAEA", "Texas Chicken", "Church 's Chicken" | |
| | other | "BAR", "Amphiphysin", "N-terminal lipid" | |
| | person | "Edmund Payne", "Hicks", "Ellaline Terriss" | |
| | product | "Phantom", "100EX", "Corvettes - GT1 C6R" | |
|
|
| ## Evaluation |
|
|
| ### Metrics |
| | Label | Precision | Recall | F1 | |
| |:-------------|:----------|:-------|:-------| |
| | **all** | 0.7582 | 0.7751 | 0.7666 | |
| | art | 0.7713 | 0.7783 | 0.7748 | |
| | building | 0.6034 | 0.7085 | 0.6518 | |
| | event | 0.5512 | 0.5207 | 0.5355 | |
| | location | 0.8163 | 0.8321 | 0.8242 | |
| | organization | 0.7083 | 0.6894 | 0.6987 | |
| | other | 0.6748 | 0.7253 | 0.6991 | |
| | person | 0.8987 | 0.9053 | 0.9020 | |
| | product | 0.5685 | 0.6431 | 0.6035 | |
|
|
| ## Uses |
|
|
| ### Direct Use for Inference |
|
|
| ```python |
| from span_marker import SpanMarkerModel |
| |
| # Download from the 🤗 Hub |
| model = SpanMarkerModel.from_pretrained("span_marker_model_id") |
| # Run inference |
| entities = model.predict("Caretaker manager George Goss led them on a run in the FA Cup, defeating Liverpool in round 4, to reach the semi-final at Stamford Bridge, where they were defeated 2–0 by Sheffield United on 28 March 1925.") |
| ``` |
|
|
| ### Downstream Use |
| You can finetune this model on your own dataset. |
|
|
| <details><summary>Click to expand</summary> |
|
|
| ```python |
| from span_marker import SpanMarkerModel, Trainer |
| |
| # Download from the 🤗 Hub |
| model = SpanMarkerModel.from_pretrained("span_marker_model_id") |
| |
| # Specify a Dataset with "tokens" and "ner_tag" columns |
| dataset = load_dataset("conll2003") # For example CoNLL2003 |
| |
| # Initialize a Trainer using the pretrained model & dataset |
| trainer = Trainer( |
| model=model, |
| train_dataset=dataset["train"], |
| eval_dataset=dataset["validation"], |
| ) |
| trainer.train() |
| trainer.save_model("span_marker_model_id-finetuned") |
| ``` |
| </details> |
|
|
| <!-- |
| ### Out-of-Scope Use |
|
|
| *List how the model may foreseeably be misused and address what users ought not to do with the model.* |
| --> |
|
|
| <!-- |
| ## Bias, Risks and Limitations |
|
|
| *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* |
| --> |
|
|
| <!-- |
| ### Recommendations |
|
|
| *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* |
| --> |
|
|
| ## Training Details |
|
|
| ### Training Set Metrics |
| | Training set | Min | Median | Max | |
| |:----------------------|:----|:--------|:----| |
| | Sentence length | 1 | 24.4956 | 163 | |
| | Entities per sentence | 0 | 2.5439 | 35 | |
|
|
| ### Training Hyperparameters |
| - learning_rate: 5e-05 |
| - train_batch_size: 64 |
| - eval_batch_size: 128 |
| - seed: 42 |
| - gradient_accumulation_steps: 2 |
| - total_train_batch_size: 128 |
| - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
| - lr_scheduler_type: linear |
| - lr_scheduler_warmup_ratio: 0.1 |
| - num_epochs: 10 |
| - mixed_precision_training: Native AMP |
|
|
| ### Training Results |
| | Epoch | Step | Validation Loss | Validation Precision | Validation Recall | Validation F1 | Validation Accuracy | |
| |:------:|:----:|:---------------:|:--------------------:|:-----------------:|:-------------:|:-------------------:| |
| | 1.7467 | 200 | 0.0120 | 0.7533 | 0.7473 | 0.7503 | 0.9286 | |
| | 3.4934 | 400 | 0.0110 | 0.7659 | 0.7761 | 0.7710 | 0.9385 | |
| | 5.2402 | 600 | 0.0114 | 0.7772 | 0.7899 | 0.7835 | 0.9424 | |
| | 6.9869 | 800 | 0.0120 | 0.7724 | 0.7953 | 0.7837 | 0.9421 | |
| | 8.7336 | 1000 | 0.0124 | 0.7680 | 0.7942 | 0.7809 | 0.9413 | |
|
|
| ### Framework Versions |
| - Python: 3.10.12 |
| - SpanMarker: 1.5.0 |
| - Transformers: 4.35.2 |
| - PyTorch: 2.1.0+cu118 |
| - Datasets: 2.14.7 |
| - Tokenizers: 0.15.0 |
|
|
| ## Citation |
|
|
| ### BibTeX |
| ``` |
| @software{Aarsen_SpanMarker, |
| author = {Aarsen, Tom}, |
| license = {Apache-2.0}, |
| title = {{SpanMarker for Named Entity Recognition}}, |
| url = {https://github.com/tomaarsen/SpanMarkerNER} |
| } |
| ``` |
|
|
| <!-- |
| ## Glossary |
|
|
| *Clearly define terms in order to be accessible across audiences.* |
| --> |
|
|
| <!-- |
| ## Model Card Authors |
|
|
| *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* |
| --> |
|
|
| <!-- |
| ## Model Card Contact |
|
|
| *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* |
| --> |
