---
language: vi
library_name: transformers
tags:
  - ner
  - token-classification
  - phobert
  - roberta
  - vietnamese
  - food-order
  - cs221
metrics:
  - f1
  - precision
  - recall
  - accuracy
pipeline_tag: token-classification
---
# PhoBERT-FoodNER: Vietnamese Food Order Named Entity Recognition

#### Table of contents
1. [Introduction](#introduction)
2. [Dataset Overview](#dataset)
3. [Entity Label Space](#labels)
4. [Empirical Evaluation](#evaluation)
5. [Training Hyperparameters](#hyperparameters)
6. [Using PhoBERT-FoodNER with `transformers`](#transformers)
7. [Notes on Word Segmentation](#vncorenlp)
8. [Authors & Citation](#citation)

---

## <a name="introduction"></a> 1. Introduction

**`CS221DoAn/vietnamese_food_order_extraction`** is a fine-tuned Named Entity Recognition (NER) model based on the pre-trained monolingual language model [PhoBERT-base](https://github.com/VinAIResearch/PhoBERT) (`vinai/phobert-base`)[cite: 1]. It is specifically trained and optimized for **Token Classification** on domain-specific Vietnamese unstructured text: **Online Food Delivery Orders and Messages**.

The model acts as an automated information extraction engine capable of accurately resolving complex entities within conversational speech, abbreviations, and student slang commonly used in food delivery platforms. This project was developed as part of the CS221.Q21 course (Natural Language Processing).

## <a name="dataset"></a> 2. Dataset Overview

The model was fine-tuned on a custom, manually annotated dataset consisting of **2,325 real-world food ordering messages** extracted from Facebook comments (specifically from Sơn Nguyễn vegetarian restaurant). The dataset exhibits strong social media characteristics including teencode, abbreviations, and missing diacritics. 
* **Train Set:** 1,860 samples (80%)
* **Validation Set:** 232 samples (10%)
* **Test Set:** 233 samples (10%)

## <a name="labels"></a> 3. Entity Label Space

The model utilizes the standard **BIO (Begin - Inside - Outside)** tagging scheme across **15 distinct entity labels**, mapping to **7 core information classes**:

| Tag | Entity Class | Description | Examples |
| :--- | :--- | :--- | :--- |
| `B-FOOD`, `I-FOOD` | **FOOD** | Names of dishes, drinks, toppings | *cơm sườn, trà sữa, trân châu* |
| `B-QUANTITY`, `I-QUANTITY` | **QUANTITY** | Portions, servings, item counts | *1p, 2 ly, một hộp, 3 suất* |
| `B-NOTE`, `I-NOTE` | **NOTE** | Special requests, flavor modifications | *không ..., ít ngọt, nhiều ...* |
| `B-PLACE`, `I-PLACE` | **PLACE** | Delivery location, addresses | *Ký túc xá khu A, tòa D6, rào b4* |
| `B-PHONE`, `I-PHONE` | **PHONE** | Receiver's contact number | *0794987xxx, 0903123xxx* |
| `B-TIME`, `I-TIME` | **TIME** | Expected/requested delivery time | *lúc 11h30, trưa nay, 18h* |
| `B-PRICE`, `I-PRICE` | **PRICE** | Monetary cost, item prices | *35k, 40000, 25 ngàn* |
| `O` | **OUTSIDE** | Non-entity words, syntax connecting words | *cho em, giao qua, với, ạ, nhé* |

## <a name="evaluation"></a> 4. Empirical Evaluation

The model's performance was evaluated on an unseen Test Set using strict **Entity-level F1-Scores** via the `seqeval` framework:

```text
              precision    recall  f1-score   support

        FOOD     0.9921    0.9947    0.9934      1135
        NOTE     0.9938    0.9815    0.9876       162
       PHONE     1.0000    1.0000    1.0000       225
       PLACE     0.9926    0.9963    0.9945       270
       PRICE     1.0000    1.0000    1.0000        34
    QUANTITY     0.9965    0.9965    0.9965       282
        TIME     0.9593    0.9752    0.9672       121

   micro avg     0.9919    0.9937    0.9928      2229
   macro avg     0.9906    0.9920    0.9913      2229
weighted avg     0.9920    0.9937    0.9928      2229

```

##  5. Training Hyperparameters

The model was fine-tuned using the `Trainer` API from Hugging Face with the following configuration:

* **Learning Rate:** `2e-5`
* **Batch Size (Train/Eval):** `16`
* **Epochs:** `10`
* **Weight Decay:** `0.01`
* **Optimizer:** AdamW
* **Evaluation Strategy:** Epoch-based (loading the best model based on F1-score at the end)

##  6. Using PhoBERT-FoodNER with `transformers`

### Installation

Install the required libraries using pip:

```bash
pip install -q transformers py_vncorenlp torch

```

### Example usage (Full Inference Pipeline)

```py3
import os
import re
import torch
import urllib.request
import py_vncorenlp
from transformers import AutoTokenizer, AutoModelForTokenClassification

# 1. Setup Device and Load Model from Hugging Face Hub
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
MODEL_ID = "CS221DoAn/vietnamese_food_order_extraction"

print("Loading Model and Tokenizer...")
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForTokenClassification.from_pretrained(MODEL_ID).to(device)
model.eval()

# 2. Initialize VnCoreNLP (Robust setup for Windows/Linux/Colab)
vncorenlp_dir = os.path.abspath('./vncorenlp')
os.makedirs(os.path.join(vncorenlp_dir, "models", "wordsegmenter"), exist_ok=True)

if not os.path.exists(os.path.join(vncorenlp_dir, "VnCoreNLP-1.2.jar")):
    print("Downloading VnCoreNLP models...")
    base_url = "https://raw.githubusercontent.com/vncorenlp/VnCoreNLP/master/"
    files = ["VnCoreNLP-1.2.jar", "models/wordsegmenter/vi-vocab", "models/wordsegmenter/wordsegmenter.rdr"]
    for f in files:
        urllib.request.urlretrieve(base_url + f, os.path.join(vncorenlp_dir, f))

rdrsegmenter = py_vncorenlp.VnCoreNLP(annotators=["wseg"], save_dir=vncorenlp_dir)
user_dict = {"a", "giao"}  # Custom lexicon to prevent incorrect compounding

def predict_food_order(raw_text):
    print(f"\nInput: {raw_text}")
    
    # Preprocessing & Text Cleaning
    clean_text = re.sub(r'([.,()!?:+])', r' \1 ', raw_text.replace('_', ' '))
    clean_text = re.sub(r'\s+', ' ', clean_text).strip()
    
    # Word Segmentation
    tokens = []
    for token in rdrsegmenter.word_segment(clean_text)[0].split():
        parts = token.split('_')
        if any(p.lower() in user_dict for p in parts):
            tokens.extend(parts)  
        else:
            tokens.append(token)              
    
    # Tokenization & Subword Alignment
    input_ids = [tokenizer.cls_token_id]
    word_ids = [None]
    
    for i, word in enumerate(tokens):
        sub_ids = tokenizer.encode(word, add_special_tokens=False)
        input_ids.extend(sub_ids)
        word_ids.extend([i] * len(sub_ids))
        
    input_ids.append(tokenizer.sep_token_id)
    word_ids.append(None)
    
    # Forward Pass / Inference
    inputs = torch.tensor([input_ids]).to(device)
    with torch.no_grad():
        logits = model(inputs).logits
        preds = logits.argmax(dim=-1)[0].tolist()
        
    # Display Extracted Entities
    prev_idx = None
    for i, idx in enumerate(word_ids):
        if idx is not None and idx != prev_idx:
            tag_id = preds[i]
            label = model.config.id2label.get(tag_id, "O")
            print(f"{tokens[idx]:<20} {label}")
            prev_idx = idx

# --- EXECUTE TEST CASES ---
test_cases = [
    "1p cải xào, gà lát chiên giòn, chả giò, cơm thêm, Start Cf, 0384293xxx, giao lúc 12h15 ạ",
    "em 1p sườn ngào, bầu xào, chả giò, rau củ kho + cơm thêm. giao rào b4, 11h30. 0773570xxx.",
    "1p 15k thập cẩm ( cơm thêm) giao rào D6 0585115xxx ạ"
]

for sample in test_cases:
    predict_food_order(sample)

```

##  7. Notes on Word Segmentation

As PhoBERT employed the [RDRSegmenter](https://github.com/datquocnguyen/RDRsegmenter) from [VnCoreNLP](https://github.com/vncorenlp/VnCoreNLP) to pre-process its pre-training data, **input texts MUST be word-segmented before being fed into the tokenizer** (e.g., separating syllables of compound words with underscores like `cơm_sườn`).

Feeding raw, unsegmented text directly into `tokenizer.encode()` will break word boundaries and significantly degrade NER accuracy. The example code above handles this pipeline automatically using `py_vncorenlp`.

##  8. Authors & Citation

This project was developed for the **Natural Language Processing (CS221.Q21)** course at the University of Information Technology (UIT) - VNU-HCM.

* **Students:** Võ Thành Lộc (24520989), Nguyễn Anh Nguyên (24521185)
* **Instructor:** Ph.D Nguyễn Trọng Chỉnh
* **Date:** July 2026

If you use this model in your academic research or projects, please cite our project and the original PhoBERT paper:

```bibtex
@misc{cs221_food_order_ner,
  author = {Vo Thanh Loc and Nguyen Anh Nguyen},
  title = {Food Order Extraction: Vietnamese NER using PhoBERT},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = https://huggingface.co/CS221DoAn/vietnamese_food_order_extraction
}

@inproceedings{phobert,
  title     = {{PhoBERT: Pre-trained language models for Vietnamese}},
  author    = {Dat Quoc Nguyen and Anh Tuan Nguyen},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2020},
  year      = {2020},
  pages     = {1037--1042}
}
```