README.md

---
language:
- az
license: apache-2.0
tags:
- token-classification
- ner
- xlm-roberta
- azerbaijani
- multilingual
library_name: transformers
pipeline_tag: token-classification
datasets:
- LocalDoc/azerbaijani-ner-dataset
metrics:
- precision
- recall
- f1
base_model: xlm-roberta-large
model-index:
- name: XLM-RoBERTa Large Azerbaijani NER
  results:
  - task:
      type: token-classification
      name: Named Entity Recognition
    dataset:
      type: LocalDoc/azerbaijani-ner-dataset
      name: Azerbaijani NER Dataset
    metrics:
    - type: precision
      name: Precision
      value: 0.7831
    - type: recall
      name: Recall
      value: 0.7284
    - type: f1
      name: F1
      value: 0.7548
---

# Azerbaijani Named Entity Recognition with XLM-RoBERTa Large

Fine-tuned version of `xlm-roberta-large` for Named Entity Recognition (NER) on Azerbaijani text. Recognizes 12 entity types including persons, locations, organizations, dates, and more.

**Hugging Face**: [IsmatS/xlm_roberta_large_az_ner](https://huggingface.co/IsmatS/xlm_roberta_large_az_ner)

## Quick Start

```python
from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline

model_name = "IsmatS/xlm_roberta_large_az_ner"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForTokenClassification.from_pretrained(model_name)

ner = pipeline("ner", model=model, tokenizer=tokenizer, aggregation_strategy="simple")

text = "Shahla Khuduyeva və Pasha Sığorta şirkəti haqqında məlumat."
entities = ner(text)
for e in entities:
    print(f"{e['entity_group']:15} {e['word']:25} ({e['score']:.2f})")
```

**Output:**
```
PERSON          Shahla Khuduyeva          (0.97)
ORGANISATION    Pasha Sığorta             (0.95)
```

## Model Details

| Property | Value |
|----------|-------|
| **Base Model** | `xlm-roberta-large` (355M parameters) |
| **Task** | Named Entity Recognition (NER) |
| **Language** | Azerbaijani (`az`) |
| **Dataset** | [LocalDoc/azerbaijani-ner-dataset](https://huggingface.co/datasets/LocalDoc/azerbaijani-ner-dataset) |
| **License** | Apache 2.0 |

## Supported Entity Types

| Entity | Description | Example |
|--------|-------------|---------|
| PERSON | Person names | İlham Əliyev |
| LOCATION | Geographic locations | Bakı, Azərbaycan |
| ORGANISATION | Companies, institutions | SOCAR, Bakı Dövlət Universiteti |
| DATE | Dates and periods | 2024-cü il, sentyabr |
| TIME | Time expressions | səhər saat 9:00 |
| MONEY | Monetary values | 150 manat |
| PERCENTAGE | Percentage values | 18% |
| FACILITY | Buildings, landmarks | Heydər Əliyev Mərkəzi |
| PRODUCT | Products and items | - |
| EVENT | Events | - |
| LAW | Legal documents | - |
| ART | Artworks | - |

## Performance

### Best Checkpoint (Epoch 6)

| Metric | Value |
|--------|-------|
| Precision | 0.7831 |
| Recall | 0.7284 |
| **F1** | **0.7548** |

### Training History

| Epoch | Train Loss | Val Loss | Precision | Recall | F1 |
|-------|-----------|----------|-----------|--------|-----|
| 1 | 0.4075 | 0.2538 | 0.7689 | 0.7214 | 0.7444 |
| 2 | 0.2556 | 0.2497 | 0.7835 | 0.7245 | 0.7528 |
| 3 | 0.2144 | 0.2488 | 0.7509 | 0.7489 | 0.7499 |
| 4 | 0.1934 | 0.2571 | 0.7686 | 0.7404 | 0.7542 |
| 5 | 0.1698 | 0.2757 | 0.7458 | 0.7537 | 0.7497 |
| 6 | 0.1526 | 0.2881 | **0.7831** | **0.7284** | **0.7548** |
| 7 | 0.1443 | 0.3034 | 0.7585 | 0.7381 | 0.7481 |

### Entity-Level Metrics (Epoch 7)

| Entity | Precision | Recall | F1 | Support |
|--------|-----------|--------|----|---------|
| ART | 0.41 | 0.19 | 0.26 | 1828 |
| DATE | 0.53 | 0.49 | 0.51 | 834 |
| EVENT | 0.67 | 0.51 | 0.58 | 63 |
| FACILITY | 0.74 | 0.68 | 0.71 | 1134 |
| LAW | 0.62 | 0.58 | 0.60 | 1066 |
| LOCATION | 0.81 | 0.79 | 0.80 | 8795 |
| MONEY | 0.59 | 0.56 | 0.58 | 555 |
| ORGANISATION | 0.70 | 0.69 | 0.70 | 554 |
| PERCENTAGE | 0.80 | 0.82 | 0.81 | 3502 |
| PERSON | 0.90 | 0.82 | 0.86 | 7007 |
| PRODUCT | 0.83 | 0.84 | 0.84 | 2624 |
| TIME | 0.60 | 0.53 | 0.57 | 1584 |

**Overall**: Micro Avg F1 = 0.75 | Weighted Avg F1 = 0.74

## Training Configuration

```python
TrainingArguments(
    learning_rate=2e-5,
    per_device_train_batch_size=128,
    per_device_eval_batch_size=128,
    num_train_epochs=12,
    weight_decay=0.005,
    fp16=True,
    metric_for_best_model="f1",
    load_best_model_at_end=True,
)
```

- **Optimizer**: AdamW
- **Early stopping**: patience=5 on F1
- **Infrastructure**: Google Colab A100 GPU

## Usage

### Installation

```bash
pip install transformers torch
```

### Inference

```python
from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline
import torch

model_name = "IsmatS/xlm_roberta_large_az_ner"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForTokenClassification.from_pretrained(model_name)

device = 0 if torch.cuda.is_available() else -1
ner = pipeline("ner", model=model, tokenizer=tokenizer,
               aggregation_strategy="simple", device=device)

# Single text
text = "Bakı şəhərində Azərbaycan Respublikasının prezidenti İlham Əliyev."
results = ner(text)
for e in results:
    print(f"[{e['entity_group']}] {e['word']} (score: {e['score']:.3f})")
```

### Batch Inference

```python
texts = [
    "Bakı şəhərində İlham Əliyev çıxış etdi.",
    "SOCAR şirkəti 2024-cü ildə rekord gəlir əldə etdi.",
    "Heydər Əliyev Beynəlxalq Hava Limanı yeni terminalı açıldı.",
]

results = ner(texts)
for text, entities in zip(texts, results):
    print(f"\nText: {text}")
    for e in entities:
        print(f"  [{e['entity_group']}] {e['word']}")
```

## Dataset

Trained on [LocalDoc/azerbaijani-ner-dataset](https://huggingface.co/datasets/LocalDoc/azerbaijani-ner-dataset) with 25 entity categories annotated in IOB2 format.

```python
from datasets import load_dataset
dataset = load_dataset("LocalDoc/azerbaijani-ner-dataset")
```

## Model Comparison

| Model | F1 | Parameters |
|-------|-----|-----------|
| [mBERT Azerbaijani NER](https://huggingface.co/IsmatS/mbert-az-ner) | 0.677 | 180M |
| [XLM-RoBERTa Base Azerbaijani NER](https://huggingface.co/IsmatS/xlm-roberta-az-ner) | 0.752 | 125M |
| **XLM-RoBERTa Large Azerbaijani NER** (this model) | **0.755** | **355M** |
| [Azeri-Turkish BERT NER](https://huggingface.co/IsmatS/azeri-turkish-bert-ner) | 0.736 | 110M |

## Files

```
xlm_roberta_large_az_ner/
├── README.md                 # This file
├── config.json               # Model configuration
├── model-001.safetensors     # Model weights
├── sentencepiece.bpe.model   # SentencePiece tokenizer
├── special_tokens_map.json   # Special token mappings
├── tokenizer.json            # Tokenizer vocabulary
├── tokenizer_config.json     # Tokenizer configuration
├── xlm_roberta_large.ipynb   # Training notebook
└── xlm_roberta_large.py      # Training script
```

## Citation

```bibtex
@model{samadov2024xlm_large_az_ner,
  author = {Ismat Samadov},
  title = {XLM-RoBERTa Large Azerbaijani NER},
  year = {2024},
  publisher = {Hugging Face},
  url = {https://huggingface.co/IsmatS/xlm_roberta_large_az_ner}
}
```

## License

Apache 2.0 — see [LICENSE](LICENSE) for details.