Spaces:
Running
Running
Daryl Lim Claude Opus 4.6 commited on
Commit ·
d999e21
1
Parent(s): 491ad42
Update docs for evaluated-only language set
Browse filesUpdate CLAUDE.md and README.md to reflect 183 evaluated languages
from Table 14, Section A.9.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
CLAUDE.md
CHANGED
|
@@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
|
|
| 4 |
|
| 5 |
## Project Overview
|
| 6 |
|
| 7 |
-
A Hugging Face Spaces app that translates English text to
|
| 8 |
|
| 9 |
## Commands
|
| 10 |
|
|
@@ -34,7 +34,7 @@ pytest -m slow # model tests only
|
|
| 34 |
|
| 35 |
**`app.py`** — Single-file application. Uses `@lru_cache` for lazy loading of the `google/madlad400-3b-mt` tokenizer and model with `float16` precision (no download on import). Translation prepends a language token to the input text (e.g., `<2fr>Hello`) before tokenization and generation. The `@spaces.GPU` decorator allocates GPU on HF Spaces infrastructure.
|
| 36 |
|
| 37 |
-
**`langmap/`** — Package with `langid_mapping.py`, a hand-maintained dictionary mapping
|
| 38 |
|
| 39 |
**`tests/`** — Pytest suite split into fast (`test_langmap.py`) and slow (`test_app.py`). Slow tests require GPU and model download; they are auto-skipped without MPS/CUDA. Fast tests in `test_app.py` verify the module imports without triggering model download.
|
| 40 |
|
|
|
|
| 4 |
|
| 5 |
## Project Overview
|
| 6 |
|
| 7 |
+
A Hugging Face Spaces app that translates English text to 183 evaluated languages (Table 14, Section A.9) from Google's [MADLAD-400](https://arxiv.org/pdf/2309.04662) 3B Seq2Seq model. Built with Gradio and deployed on HF Spaces. Falls back to CPU with a warning when no GPU is available.
|
| 8 |
|
| 9 |
## Commands
|
| 10 |
|
|
|
|
| 34 |
|
| 35 |
**`app.py`** — Single-file application. Uses `@lru_cache` for lazy loading of the `google/madlad400-3b-mt` tokenizer and model with `float16` precision (no download on import). Translation prepends a language token to the input text (e.g., `<2fr>Hello`) before tokenization and generation. The `@spaces.GPU` decorator allocates GPU on HF Spaces infrastructure.
|
| 36 |
|
| 37 |
+
**`langmap/`** — Package with `langid_mapping.py`, a hand-maintained dictionary mapping 183 evaluated language tokens to human-readable language names (sourced from Table 14, Section A.9 of the MADLAD-400 paper; evaluation sets: WMT, Flores-200, NTREX, Gatones). Available languages at runtime are the intersection of this mapping and the model's vocabulary.
|
| 38 |
|
| 39 |
**`tests/`** — Pytest suite split into fast (`test_langmap.py`) and slow (`test_app.py`). Slow tests require GPU and model download; they are auto-skipped without MPS/CUDA. Fast tests in `test_app.py` verify the module imports without triggering model download.
|
| 40 |
|
README.md
CHANGED
|
@@ -8,12 +8,12 @@ sdk_version: 6.8.0
|
|
| 8 |
app_file: app.py
|
| 9 |
pinned: false
|
| 10 |
license: apache-2.0
|
| 11 |
-
short_description: Translate English text to
|
| 12 |
---
|
| 13 |
|
| 14 |
# MADLAD-400 Translation
|
| 15 |
|
| 16 |
-
Translate English text to
|
| 17 |
|
| 18 |
## Requirements
|
| 19 |
|
|
|
|
| 8 |
app_file: app.py
|
| 9 |
pinned: false
|
| 10 |
license: apache-2.0
|
| 11 |
+
short_description: Translate English text to 183 evaluated languages.
|
| 12 |
---
|
| 13 |
|
| 14 |
# MADLAD-400 Translation
|
| 15 |
|
| 16 |
+
Translate English text to 183 evaluated languages (Table 14, Section A.9) using Google's [MADLAD-400](https://arxiv.org/pdf/2309.04662) 3B model.
|
| 17 |
|
| 18 |
## Requirements
|
| 19 |
|