Add files using upload-large-folder tool

Browse files

Files changed (8) hide show

.gitattributes +1 -0
README.md +190 -0
config.json +796 -0
id2label.json +219 -0
openmed-mlx.json +31 -0
tokenizer.json +3 -0
tokenizer_config.json +12 -0
weights.safetensors +3 -0

.gitattributes CHANGED Viewed

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md ADDED Viewed

	@@ -0,0 +1,190 @@

+---
+license: apache-2.0
+base_model: OpenMed/privacy-filter-multilingual-v2
+datasets:
+  - ai4privacy/pii-masking-200k
+  - ai4privacy/pii-masking-400k
+  - ai4privacy/open-pii-masking-500k-ai4privacy
+  - ai4privacy/pii-masking-openpii-1m
+  - nvidia/Nemotron-PII
+  - gretelai/gretel-pii-masking-en-v1
+  - piimb/privy
+pipeline_tag: token-classification
+library_name: openmed
+tags:
+  - openmed
+  - mlx
+  - apple-silicon
+  - token-classification
+  - pii
+  - de-identification
+  - medical
+  - clinical
+  - privacy-filter
+  - multilingual
+  - bf16
+  - full-precision
+language:
+  - ar
+  - bn
+  - de
+  - en
+  - es
+  - fr
+  - hi
+  - it
+  - ja
+  - ko
+  - nl
+  - pt
+  - te
+  - tr
+  - vi
+  - zh
+---
+# OpenMed Privacy Filter Multilingual v2 - MLX BF16
+A native [MLX](https://github.com/ml-explore/mlx) port of [`OpenMed/privacy-filter-multilingual-v2`](https://huggingface.co/OpenMed/privacy-filter-multilingual-v2) for Apple Silicon PII detection and de-identification with OpenMed. This is the unquantized BF16 reference artifact. For the 8-bit sibling, see [`OpenMed/privacy-filter-multilingual-v2-mlx-8bit`](https://huggingface.co/OpenMed/privacy-filter-multilingual-v2-mlx-8bit).
+> Family at a glance:
+> - PyTorch source: [`OpenMed/privacy-filter-multilingual-v2`](https://huggingface.co/OpenMed/privacy-filter-multilingual-v2)
+> - MLX BF16 (this repo): Apple Silicon, full precision, `2.6 GiB` weights
+> - MLX 8-bit: [`OpenMed/privacy-filter-multilingual-v2-mlx-8bit`](https://huggingface.co/OpenMed/privacy-filter-multilingual-v2-mlx-8bit) - Apple Silicon, `1.4 GiB` weights
+## At a glance
+- Source checkpoint: [`OpenMed/privacy-filter-multilingual-v2`](https://huggingface.co/OpenMed/privacy-filter-multilingual-v2)
+- OpenMed MLX repo: [`OpenMed/privacy-filter-multilingual-v2-mlx`](https://huggingface.co/OpenMed/privacy-filter-multilingual-v2-mlx)
+- Label schema: 54 fine-grained multilingual PII categories
+- Output space: 217 BIOES classes (O plus B/I/E/S for each category)
+- Languages: 16 languages from the source card: ar, bn, de, en, es, fr, hi, it, ja, ko, nl, pt, te, tr, vi, zh
+- Weight format: `safetensors`
+- Quantization: none (BF16 reference)
+## Q8 sibling validation
+The 8-bit sibling was compared against this BF16 artifact on 10 golden PII samples. Decoded entity spans matched across all samples. Average Q8/BF16 argmax agreement was 100.00% with average logit MAE 0.1902; average local forward time was 15.1 ms for BF16 vs 8.4 ms for Q8.
+## What it does
+This model is an MLX packaging of [`OpenMed/privacy-filter-multilingual-v2`](https://huggingface.co/OpenMed/privacy-filter-multilingual-v2), the second-generation multilingual checkpoint for fine-grained PII extraction across 16 languages. It uses OpenAI's Privacy Filter architecture and predicts 217 BIOES classes (O plus B/I/E/S for each category). The OpenMed `PrivacyFilterMLXPipeline` runs BIOES-aware Viterbi decoding so callers receive grouped spans instead of raw token tags.
+Label coverage highlights:
+- Identity: FIRSTNAME, MIDDLENAME, LASTNAME, AGE, GENDER, USERNAME, OCCUPATION, ORGANIZATION
+- Contact and address: EMAIL, PHONE, URL, STREET, BUILDINGNUMBER, CITY, COUNTY, STATE, ZIPCODE
+- Financial and crypto: BANKACCOUNT, IBAN, BIC, CREDITCARD, CVV, PIN, BITCOINADDRESS, ETHEREUMADDRESS
+- Vehicle, digital, and auth: VIN, VRM, IPADDRESS, MACADDRESS, IMEI, PASSWORD
+- Date and amount labels such as DATE, DATEOFBIRTH, TIME, AMOUNT, CURRENCY, and CURRENCYCODE
+The full label map is included in `id2label.json`.
+## Architecture
+| Field | Value |
+| --- | --- |
+| Source model type | `openai_privacy_filter` |
+| Source architecture | `OpenAIPrivacyFilterForTokenClassification` |
+| Hidden size | 640 |
+| Transformer layers | 8 |
+| Attention | Grouped-query attention (14 query heads / 2 KV heads, head_dim=64) with attention sinks |
+| FFN | Sparse Mixture-of-Experts - 128 experts, top-4 routing, SwiGLU |
+| Position encoding | YARN-scaled RoPE (`rope_theta=150000`, factor=32) |
+| Context length | 131,072 tokens (initial 4,096) |
+| Tokenizer | `o200k_base` / tiktoken-compatible tokenizer assets, vocab 200,064 |
+| Output head | Linear(640 -> 217) with bias |
+## File set
+| File | Size | Purpose |
+| --- | --- | --- |
+| `weights.safetensors` | 2.6 GiB | MLX weights |
+| `config.json` | 17.6 KiB | Model and OpenMed MLX runtime config |
+| `id2label.json` | 4.8 KiB | Numeric ID to BIOES label mapping |
+| `openmed-mlx.json` | 0.7 KiB | OpenMed MLX artifact manifest |
+| `tokenizer.json` | 27 MiB | Tokenizer asset kept with the artifact |
+| `tokenizer_config.json` | 0.2 KiB | Tokenizer metadata |
+The MLX runtime uses the tiktoken-compatible `o200k_base` tokenizer path. `tokenizer.json` and `tokenizer_config.json` are bundled so consumers can inspect the tokenizer assets and keep the artifact self-contained.
+## Quick start
+### With OpenMed
+```bash
+pip install -U "openmed[mlx]"
+```
+```python
+from openmed import extract_pii, deidentify
+from openmed.core import OpenMedConfig
+model_name = "OpenMed/privacy-filter-multilingual-v2-mlx"
+text = (
+    "Patient Sarah Johnson (DOB 03/15/1985), MRN 4872910, "
+    "phone 415-555-0123, email sarah.johnson@example.com."
+)
+result = extract_pii(
+    text,
+    model_name=model_name,
+    config=OpenMedConfig(backend="mlx"),
+)
+for ent in result.entities:
+    print(ent.label, ent.text, round(ent.confidence, 4))
+masked = deidentify(
+    text,
+    method="mask",
+    model_name=model_name,
+    config=OpenMedConfig(backend="mlx"),
+)
+print(masked.deidentified_text)
+```
+For non-MLX hosts, use the source PyTorch checkpoint [`OpenMed/privacy-filter-multilingual-v2`](https://huggingface.co/OpenMed/privacy-filter-multilingual-v2).
+### Direct MLX usage
+```python
+from huggingface_hub import snapshot_download
+from openmed.mlx.inference import PrivacyFilterMLXPipeline
+model_path = snapshot_download("OpenMed/privacy-filter-multilingual-v2-mlx")
+pipe = PrivacyFilterMLXPipeline(model_path)
+print(pipe("Email me at alice.smith@example.com after 5pm."))
+```
+### Loading from a local snapshot
+```python
+from openmed.mlx.models import load_model
+import mlx.core as mx
+model = load_model("/path/to/privacy-filter-multilingual-v2-mlx")
+ids = mx.array([[1, 100, 200, 300]], dtype=mx.int32)
+mask = mx.ones((1, 4), dtype=mx.bool_)
+logits = model(ids, attention_mask=mask)
+print(logits.shape)
+```
+## Hardware notes
+- Designed for Apple Silicon with MLX.
+- CPU inference may work, but GPU-backed MLX on M-series Macs is the intended runtime.
+- The Python package path is `pip install -U "openmed[mlx]"`.
+## Credits
+This artifact builds on:
+- [`OpenMed/privacy-filter-multilingual-v2`](https://huggingface.co/OpenMed/privacy-filter-multilingual-v2) by OpenMed
+- [`openai/privacy-filter`](https://huggingface.co/openai/privacy-filter) and OpenAI's `opf` training/evaluation tooling
+- The datasets listed in the model-card metadata above
+- Apple's [MLX](https://github.com/ml-explore/mlx) framework
+## License
+Apache 2.0, matching the source checkpoint metadata.

config.json ADDED Viewed

	@@ -0,0 +1,796 @@

+{
+  "architectures": [
+    "OpenAIPrivacyFilterForTokenClassification"
+  ],
+  "attention_bias": true,
+  "attention_dropout": 0.0,
+  "bos_token_id": null,
+  "classifier_dropout": 0.0,
+  "default_n_ctx": 128000,
+  "dtype": "bfloat16",
+  "eos_token_id": 199999,
+  "head_dim": 64,
+  "hidden_act": "silu",
+  "hidden_size": 640,
+  "id2label": {
+    "0": "O",
+    "1": "B-ACCOUNTNAME",
+    "2": "I-ACCOUNTNAME",
+    "3": "E-ACCOUNTNAME",
+    "4": "S-ACCOUNTNAME",
+    "5": "B-AGE",
+    "6": "I-AGE",
+    "7": "E-AGE",
+    "8": "S-AGE",
+    "9": "B-AMOUNT",
+    "10": "I-AMOUNT",
+    "11": "E-AMOUNT",
+    "12": "S-AMOUNT",
+    "13": "B-BANKACCOUNT",
+    "14": "I-BANKACCOUNT",
+    "15": "E-BANKACCOUNT",
+    "16": "S-BANKACCOUNT",
+    "17": "B-BIC",
+    "18": "I-BIC",
+    "19": "E-BIC",
+    "20": "S-BIC",
+    "21": "B-BITCOINADDRESS",
+    "22": "I-BITCOINADDRESS",
+    "23": "E-BITCOINADDRESS",
+    "24": "S-BITCOINADDRESS",
+    "25": "B-BUILDINGNUMBER",
+    "26": "I-BUILDINGNUMBER",
+    "27": "E-BUILDINGNUMBER",
+    "28": "S-BUILDINGNUMBER",
+    "29": "B-CITY",
+    "30": "I-CITY",
+    "31": "E-CITY",
+    "32": "S-CITY",
+    "33": "B-COUNTY",
+    "34": "I-COUNTY",
+    "35": "E-COUNTY",
+    "36": "S-COUNTY",
+    "37": "B-CREDITCARD",
+    "38": "I-CREDITCARD",
+    "39": "E-CREDITCARD",
+    "40": "S-CREDITCARD",
+    "41": "B-CREDITCARDISSUER",
+    "42": "I-CREDITCARDISSUER",
+    "43": "E-CREDITCARDISSUER",
+    "44": "S-CREDITCARDISSUER",
+    "45": "B-CURRENCY",
+    "46": "I-CURRENCY",
+    "47": "E-CURRENCY",
+    "48": "S-CURRENCY",
+    "49": "B-CURRENCYCODE",
+    "50": "I-CURRENCYCODE",
+    "51": "E-CURRENCYCODE",
+    "52": "S-CURRENCYCODE",
+    "53": "B-CURRENCYNAME",
+    "54": "I-CURRENCYNAME",
+    "55": "E-CURRENCYNAME",
+    "56": "S-CURRENCYNAME",
+    "57": "B-CURRENCYSYMBOL",
+    "58": "I-CURRENCYSYMBOL",
+    "59": "E-CURRENCYSYMBOL",
+    "60": "S-CURRENCYSYMBOL",
+    "61": "B-CVV",
+    "62": "I-CVV",
+    "63": "E-CVV",
+    "64": "S-CVV",
+    "65": "B-DATE",
+    "66": "I-DATE",
+    "67": "E-DATE",
+    "68": "S-DATE",
+    "69": "B-DATEOFBIRTH",
+    "70": "I-DATEOFBIRTH",
+    "71": "E-DATEOFBIRTH",
+    "72": "S-DATEOFBIRTH",
+    "73": "B-EMAIL",
+    "74": "I-EMAIL",
+    "75": "E-EMAIL",
+    "76": "S-EMAIL",
+    "77": "B-ETHEREUMADDRESS",
+    "78": "I-ETHEREUMADDRESS",
+    "79": "E-ETHEREUMADDRESS",
+    "80": "S-ETHEREUMADDRESS",
+    "81": "B-EYECOLOR",
+    "82": "I-EYECOLOR",
+    "83": "E-EYECOLOR",
+    "84": "S-EYECOLOR",
+    "85": "B-FIRSTNAME",
+    "86": "I-FIRSTNAME",
+    "87": "E-FIRSTNAME",
+    "88": "S-FIRSTNAME",
+    "89": "B-GENDER",
+    "90": "I-GENDER",
+    "91": "E-GENDER",
+    "92": "S-GENDER",
+    "93": "B-GPSCOORDINATES",
+    "94": "I-GPSCOORDINATES",
+    "95": "E-GPSCOORDINATES",
+    "96": "S-GPSCOORDINATES",
+    "97": "B-HEIGHT",
+    "98": "I-HEIGHT",
+    "99": "E-HEIGHT",
+    "100": "S-HEIGHT",
+    "101": "B-IBAN",
+    "102": "I-IBAN",
+    "103": "E-IBAN",
+    "104": "S-IBAN",
+    "105": "B-IMEI",
+    "106": "I-IMEI",
+    "107": "E-IMEI",
+    "108": "S-IMEI",
+    "109": "B-IPADDRESS",
+    "110": "I-IPADDRESS",
+    "111": "E-IPADDRESS",
+    "112": "S-IPADDRESS",
+    "113": "B-JOBDEPARTMENT",
+    "114": "I-JOBDEPARTMENT",
+    "115": "E-JOBDEPARTMENT",
+    "116": "S-JOBDEPARTMENT",
+    "117": "B-JOBTITLE",
+    "118": "I-JOBTITLE",
+    "119": "E-JOBTITLE",
+    "120": "S-JOBTITLE",
+    "121": "B-LASTNAME",
+    "122": "I-LASTNAME",
+    "123": "E-LASTNAME",
+    "124": "S-LASTNAME",
+    "125": "B-LITECOINADDRESS",
+    "126": "I-LITECOINADDRESS",
+    "127": "E-LITECOINADDRESS",
+    "128": "S-LITECOINADDRESS",
+    "129": "B-MACADDRESS",
+    "130": "I-MACADDRESS",
+    "131": "E-MACADDRESS",
+    "132": "S-MACADDRESS",
+    "133": "B-MASKEDNUMBER",
+    "134": "I-MASKEDNUMBER",
+    "135": "E-MASKEDNUMBER",
+    "136": "S-MASKEDNUMBER",
+    "137": "B-MIDDLENAME",
+    "138": "I-MIDDLENAME",
+    "139": "E-MIDDLENAME",
+    "140": "S-MIDDLENAME",
+    "141": "B-OCCUPATION",
+    "142": "I-OCCUPATION",
+    "143": "E-OCCUPATION",
+    "144": "S-OCCUPATION",
+    "145": "B-ORDINALDIRECTION",
+    "146": "I-ORDINALDIRECTION",
+    "147": "E-ORDINALDIRECTION",
+    "148": "S-ORDINALDIRECTION",
+    "149": "B-ORGANIZATION",
+    "150": "I-ORGANIZATION",
+    "151": "E-ORGANIZATION",
+    "152": "S-ORGANIZATION",
+    "153": "B-PASSWORD",
+    "154": "I-PASSWORD",
+    "155": "E-PASSWORD",
+    "156": "S-PASSWORD",
+    "157": "B-PHONE",
+    "158": "I-PHONE",
+    "159": "E-PHONE",
+    "160": "S-PHONE",
+    "161": "B-PIN",
+    "162": "I-PIN",
+    "163": "E-PIN",
+    "164": "S-PIN",
+    "165": "B-PREFIX",
+    "166": "I-PREFIX",
+    "167": "E-PREFIX",
+    "168": "S-PREFIX",
+    "169": "B-SECONDARYADDRESS",
+    "170": "I-SECONDARYADDRESS",
+    "171": "E-SECONDARYADDRESS",
+    "172": "S-SECONDARYADDRESS",
+    "173": "B-SEX",
+    "174": "I-SEX",
+    "175": "E-SEX",
+    "176": "S-SEX",
+    "177": "B-SSN",
+    "178": "I-SSN",
+    "179": "E-SSN",
+    "180": "S-SSN",
+    "181": "B-STATE",
+    "182": "I-STATE",
+    "183": "E-STATE",
+    "184": "S-STATE",
+    "185": "B-STREET",
+    "186": "I-STREET",
+    "187": "E-STREET",
+    "188": "S-STREET",
+    "189": "B-TIME",
+    "190": "I-TIME",
+    "191": "E-TIME",
+    "192": "S-TIME",
+    "193": "B-URL",
+    "194": "I-URL",
+    "195": "E-URL",
+    "196": "S-URL",
+    "197": "B-USERAGENT",
+    "198": "I-USERAGENT",
+    "199": "E-USERAGENT",
+    "200": "S-USERAGENT",
+    "201": "B-USERNAME",
+    "202": "I-USERNAME",
+    "203": "E-USERNAME",
+    "204": "S-USERNAME",
+    "205": "B-VIN",
+    "206": "I-VIN",
+    "207": "E-VIN",
+    "208": "S-VIN",
+    "209": "B-VRM",
+    "210": "I-VRM",
+    "211": "E-VRM",
+    "212": "S-VRM",
+    "213": "B-ZIPCODE",
+    "214": "I-ZIPCODE",
+    "215": "E-ZIPCODE",
+    "216": "S-ZIPCODE"
+  },
+  "initial_context_length": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 640,
+  "label2id": {
+    "O": 0,
+    "B-ACCOUNTNAME": 1,
+    "I-ACCOUNTNAME": 2,
+    "E-ACCOUNTNAME": 3,
+    "S-ACCOUNTNAME": 4,
+    "B-AGE": 5,
+    "I-AGE": 6,
+    "E-AGE": 7,
+    "S-AGE": 8,
+    "B-AMOUNT": 9,
+    "I-AMOUNT": 10,
+    "E-AMOUNT": 11,
+    "S-AMOUNT": 12,
+    "B-BANKACCOUNT": 13,
+    "I-BANKACCOUNT": 14,
+    "E-BANKACCOUNT": 15,
+    "S-BANKACCOUNT": 16,
+    "B-BIC": 17,
+    "I-BIC": 18,
+    "E-BIC": 19,
+    "S-BIC": 20,
+    "B-BITCOINADDRESS": 21,
+    "I-BITCOINADDRESS": 22,
+    "E-BITCOINADDRESS": 23,
+    "S-BITCOINADDRESS": 24,
+    "B-BUILDINGNUMBER": 25,
+    "I-BUILDINGNUMBER": 26,
+    "E-BUILDINGNUMBER": 27,
+    "S-BUILDINGNUMBER": 28,
+    "B-CITY": 29,
+    "I-CITY": 30,
+    "E-CITY": 31,
+    "S-CITY": 32,
+    "B-COUNTY": 33,
+    "I-COUNTY": 34,
+    "E-COUNTY": 35,
+    "S-COUNTY": 36,
+    "B-CREDITCARD": 37,
+    "I-CREDITCARD": 38,
+    "E-CREDITCARD": 39,
+    "S-CREDITCARD": 40,
+    "B-CREDITCARDISSUER": 41,
+    "I-CREDITCARDISSUER": 42,
+    "E-CREDITCARDISSUER": 43,
+    "S-CREDITCARDISSUER": 44,
+    "B-CURRENCY": 45,
+    "I-CURRENCY": 46,
+    "E-CURRENCY": 47,
+    "S-CURRENCY": 48,
+    "B-CURRENCYCODE": 49,
+    "I-CURRENCYCODE": 50,
+    "E-CURRENCYCODE": 51,
+    "S-CURRENCYCODE": 52,
+    "B-CURRENCYNAME": 53,
+    "I-CURRENCYNAME": 54,
+    "E-CURRENCYNAME": 55,
+    "S-CURRENCYNAME": 56,
+    "B-CURRENCYSYMBOL": 57,
+    "I-CURRENCYSYMBOL": 58,
+    "E-CURRENCYSYMBOL": 59,
+    "S-CURRENCYSYMBOL": 60,
+    "B-CVV": 61,
+    "I-CVV": 62,
+    "E-CVV": 63,
+    "S-CVV": 64,
+    "B-DATE": 65,
+    "I-DATE": 66,
+    "E-DATE": 67,
+    "S-DATE": 68,
+    "B-DATEOFBIRTH": 69,
+    "I-DATEOFBIRTH": 70,
+    "E-DATEOFBIRTH": 71,
+    "S-DATEOFBIRTH": 72,
+    "B-EMAIL": 73,
+    "I-EMAIL": 74,
+    "E-EMAIL": 75,
+    "S-EMAIL": 76,
+    "B-ETHEREUMADDRESS": 77,
+    "I-ETHEREUMADDRESS": 78,
+    "E-ETHEREUMADDRESS": 79,
+    "S-ETHEREUMADDRESS": 80,
+    "B-EYECOLOR": 81,
+    "I-EYECOLOR": 82,
+    "E-EYECOLOR": 83,
+    "S-EYECOLOR": 84,
+    "B-FIRSTNAME": 85,
+    "I-FIRSTNAME": 86,
+    "E-FIRSTNAME": 87,
+    "S-FIRSTNAME": 88,
+    "B-GENDER": 89,
+    "I-GENDER": 90,
+    "E-GENDER": 91,
+    "S-GENDER": 92,
+    "B-GPSCOORDINATES": 93,
+    "I-GPSCOORDINATES": 94,
+    "E-GPSCOORDINATES": 95,
+    "S-GPSCOORDINATES": 96,
+    "B-HEIGHT": 97,
+    "I-HEIGHT": 98,
+    "E-HEIGHT": 99,
+    "S-HEIGHT": 100,
+    "B-IBAN": 101,
+    "I-IBAN": 102,
+    "E-IBAN": 103,
+    "S-IBAN": 104,
+    "B-IMEI": 105,
+    "I-IMEI": 106,
+    "E-IMEI": 107,
+    "S-IMEI": 108,
+    "B-IPADDRESS": 109,
+    "I-IPADDRESS": 110,
+    "E-IPADDRESS": 111,
+    "S-IPADDRESS": 112,
+    "B-JOBDEPARTMENT": 113,
+    "I-JOBDEPARTMENT": 114,
+    "E-JOBDEPARTMENT": 115,
+    "S-JOBDEPARTMENT": 116,
+    "B-JOBTITLE": 117,
+    "I-JOBTITLE": 118,
+    "E-JOBTITLE": 119,
+    "S-JOBTITLE": 120,
+    "B-LASTNAME": 121,
+    "I-LASTNAME": 122,
+    "E-LASTNAME": 123,
+    "S-LASTNAME": 124,
+    "B-LITECOINADDRESS": 125,
+    "I-LITECOINADDRESS": 126,
+    "E-LITECOINADDRESS": 127,
+    "S-LITECOINADDRESS": 128,
+    "B-MACADDRESS": 129,
+    "I-MACADDRESS": 130,
+    "E-MACADDRESS": 131,
+    "S-MACADDRESS": 132,
+    "B-MASKEDNUMBER": 133,
+    "I-MASKEDNUMBER": 134,
+    "E-MASKEDNUMBER": 135,
+    "S-MASKEDNUMBER": 136,
+    "B-MIDDLENAME": 137,
+    "I-MIDDLENAME": 138,
+    "E-MIDDLENAME": 139,
+    "S-MIDDLENAME": 140,
+    "B-OCCUPATION": 141,
+    "I-OCCUPATION": 142,
+    "E-OCCUPATION": 143,
+    "S-OCCUPATION": 144,
+    "B-ORDINALDIRECTION": 145,
+    "I-ORDINALDIRECTION": 146,
+    "E-ORDINALDIRECTION": 147,
+    "S-ORDINALDIRECTION": 148,
+    "B-ORGANIZATION": 149,
+    "I-ORGANIZATION": 150,
+    "E-ORGANIZATION": 151,
+    "S-ORGANIZATION": 152,
+    "B-PASSWORD": 153,
+    "I-PASSWORD": 154,
+    "E-PASSWORD": 155,
+    "S-PASSWORD": 156,
+    "B-PHONE": 157,
+    "I-PHONE": 158,
+    "E-PHONE": 159,
+    "S-PHONE": 160,
+    "B-PIN": 161,
+    "I-PIN": 162,
+    "E-PIN": 163,
+    "S-PIN": 164,
+    "B-PREFIX": 165,
+    "I-PREFIX": 166,
+    "E-PREFIX": 167,
+    "S-PREFIX": 168,
+    "B-SECONDARYADDRESS": 169,
+    "I-SECONDARYADDRESS": 170,
+    "E-SECONDARYADDRESS": 171,
+    "S-SECONDARYADDRESS": 172,
+    "B-SEX": 173,
+    "I-SEX": 174,
+    "E-SEX": 175,
+    "S-SEX": 176,
+    "B-SSN": 177,
+    "I-SSN": 178,
+    "E-SSN": 179,
+    "S-SSN": 180,
+    "B-STATE": 181,
+    "I-STATE": 182,
+    "E-STATE": 183,
+    "S-STATE": 184,
+    "B-STREET": 185,
+    "I-STREET": 186,
+    "E-STREET": 187,
+    "S-STREET": 188,
+    "B-TIME": 189,
+    "I-TIME": 190,
+    "E-TIME": 191,
+    "S-TIME": 192,
+    "B-URL": 193,
+    "I-URL": 194,
+    "E-URL": 195,
+    "S-URL": 196,
+    "B-USERAGENT": 197,
+    "I-USERAGENT": 198,
+    "E-USERAGENT": 199,
+    "S-USERAGENT": 200,
+    "B-USERNAME": 201,
+    "I-USERNAME": 202,
+    "E-USERNAME": 203,
+    "S-USERNAME": 204,
+    "B-VIN": 205,
+    "I-VIN": 206,
+    "E-VIN": 207,
+    "S-VIN": 208,
+    "B-VRM": 209,
+    "I-VRM": 210,
+    "E-VRM": 211,
+    "S-VRM": 212,
+    "B-ZIPCODE": 213,
+    "I-ZIPCODE": 214,
+    "E-ZIPCODE": 215,
+    "S-ZIPCODE": 216
+  },
+  "max_position_embeddings": 131072,
+  "model_type": "openai_privacy_filter",
+  "num_attention_heads": 14,
+  "num_experts_per_tok": 4,
+  "num_hidden_layers": 8,
+  "num_key_value_heads": 2,
+  "num_local_experts": 128,
+  "output_router_logits": false,
+  "pad_token_id": 199999,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "beta_fast": 32.0,
+    "beta_slow": 1.0,
+    "factor": 32.0,
+    "original_max_position_embeddings": 4096,
+    "rope_theta": 150000.0,
+    "rope_type": "yarn",
+    "truncate": false
+  },
+  "router_aux_loss_coef": 0.001,
+  "sliding_window": 128,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.6.0.dev0",
+  "use_cache": true,
+  "vocab_size": 200064,
+  "transformers.js_config": {
+    "use_external_data_format": {
+      "model.onnx": 3,
+      "model_fp16.onnx": 2,
+      "model": 1
+    }
+  },
+  "num_labels": 217,
+  "opf_metadata": {
+    "category_version": "openmed_pii54_multi_v1",
+    "encoding": "o200k_base",
+    "span_class_names": [
+      "O",
+      "ACCOUNTNAME",
+      "AGE",
+      "AMOUNT",
+      "BANKACCOUNT",
+      "BIC",
+      "BITCOINADDRESS",
+      "BUILDINGNUMBER",
+      "CITY",
+      "COUNTY",
+      "CREDITCARD",
+      "CREDITCARDISSUER",
+      "CURRENCY",
+      "CURRENCYCODE",
+      "CURRENCYNAME",
+      "CURRENCYSYMBOL",
+      "CVV",
+      "DATE",
+      "DATEOFBIRTH",
+      "EMAIL",
+      "ETHEREUMADDRESS",
+      "EYECOLOR",
+      "FIRSTNAME",
+      "GENDER",
+      "GPSCOORDINATES",
+      "HEIGHT",
+      "IBAN",
+      "IMEI",
+      "IPADDRESS",
+      "JOBDEPARTMENT",
+      "JOBTITLE",
+      "LASTNAME",
+      "LITECOINADDRESS",
+      "MACADDRESS",
+      "MASKEDNUMBER",
+      "MIDDLENAME",
+      "OCCUPATION",
+      "ORDINALDIRECTION",
+      "ORGANIZATION",
+      "PASSWORD",
+      "PHONE",
+      "PIN",
+      "PREFIX",
+      "SECONDARYADDRESS",
+      "SEX",
+      "SSN",
+      "STATE",
+      "STREET",
+      "TIME",
+      "URL",
+      "USERAGENT",
+      "USERNAME",
+      "VIN",
+      "VRM",
+      "ZIPCODE"
+    ],
+    "ner_class_names": [
+      "O",
+      "B-ACCOUNTNAME",
+      "I-ACCOUNTNAME",
+      "E-ACCOUNTNAME",
+      "S-ACCOUNTNAME",
+      "B-AGE",
+      "I-AGE",
+      "E-AGE",
+      "S-AGE",
+      "B-AMOUNT",
+      "I-AMOUNT",
+      "E-AMOUNT",
+      "S-AMOUNT",
+      "B-BANKACCOUNT",
+      "I-BANKACCOUNT",
+      "E-BANKACCOUNT",
+      "S-BANKACCOUNT",
+      "B-BIC",
+      "I-BIC",
+      "E-BIC",
+      "S-BIC",
+      "B-BITCOINADDRESS",
+      "I-BITCOINADDRESS",
+      "E-BITCOINADDRESS",
+      "S-BITCOINADDRESS",
+      "B-BUILDINGNUMBER",
+      "I-BUILDINGNUMBER",
+      "E-BUILDINGNUMBER",
+      "S-BUILDINGNUMBER",
+      "B-CITY",
+      "I-CITY",
+      "E-CITY",
+      "S-CITY",
+      "B-COUNTY",
+      "I-COUNTY",
+      "E-COUNTY",
+      "S-COUNTY",
+      "B-CREDITCARD",
+      "I-CREDITCARD",
+      "E-CREDITCARD",
+      "S-CREDITCARD",
+      "B-CREDITCARDISSUER",
+      "I-CREDITCARDISSUER",
+      "E-CREDITCARDISSUER",
+      "S-CREDITCARDISSUER",
+      "B-CURRENCY",
+      "I-CURRENCY",
+      "E-CURRENCY",
+      "S-CURRENCY",
+      "B-CURRENCYCODE",
+      "I-CURRENCYCODE",
+      "E-CURRENCYCODE",
+      "S-CURRENCYCODE",
+      "B-CURRENCYNAME",
+      "I-CURRENCYNAME",
+      "E-CURRENCYNAME",
+      "S-CURRENCYNAME",
+      "B-CURRENCYSYMBOL",
+      "I-CURRENCYSYMBOL",
+      "E-CURRENCYSYMBOL",
+      "S-CURRENCYSYMBOL",
+      "B-CVV",
+      "I-CVV",
+      "E-CVV",
+      "S-CVV",
+      "B-DATE",
+      "I-DATE",
+      "E-DATE",
+      "S-DATE",
+      "B-DATEOFBIRTH",
+      "I-DATEOFBIRTH",
+      "E-DATEOFBIRTH",
+      "S-DATEOFBIRTH",
+      "B-EMAIL",
+      "I-EMAIL",
+      "E-EMAIL",
+      "S-EMAIL",
+      "B-ETHEREUMADDRESS",
+      "I-ETHEREUMADDRESS",
+      "E-ETHEREUMADDRESS",
+      "S-ETHEREUMADDRESS",
+      "B-EYECOLOR",
+      "I-EYECOLOR",
+      "E-EYECOLOR",
+      "S-EYECOLOR",
+      "B-FIRSTNAME",
+      "I-FIRSTNAME",
+      "E-FIRSTNAME",
+      "S-FIRSTNAME",
+      "B-GENDER",
+      "I-GENDER",
+      "E-GENDER",
+      "S-GENDER",
+      "B-GPSCOORDINATES",
+      "I-GPSCOORDINATES",
+      "E-GPSCOORDINATES",
+      "S-GPSCOORDINATES",
+      "B-HEIGHT",
+      "I-HEIGHT",
+      "E-HEIGHT",
+      "S-HEIGHT",
+      "B-IBAN",
+      "I-IBAN",
+      "E-IBAN",
+      "S-IBAN",
+      "B-IMEI",
+      "I-IMEI",
+      "E-IMEI",
+      "S-IMEI",
+      "B-IPADDRESS",
+      "I-IPADDRESS",
+      "E-IPADDRESS",
+      "S-IPADDRESS",
+      "B-JOBDEPARTMENT",
+      "I-JOBDEPARTMENT",
+      "E-JOBDEPARTMENT",
+      "S-JOBDEPARTMENT",
+      "B-JOBTITLE",
+      "I-JOBTITLE",
+      "E-JOBTITLE",
+      "S-JOBTITLE",
+      "B-LASTNAME",
+      "I-LASTNAME",
+      "E-LASTNAME",
+      "S-LASTNAME",
+      "B-LITECOINADDRESS",
+      "I-LITECOINADDRESS",
+      "E-LITECOINADDRESS",
+      "S-LITECOINADDRESS",
+      "B-MACADDRESS",
+      "I-MACADDRESS",
+      "E-MACADDRESS",
+      "S-MACADDRESS",
+      "B-MASKEDNUMBER",
+      "I-MASKEDNUMBER",
+      "E-MASKEDNUMBER",
+      "S-MASKEDNUMBER",
+      "B-MIDDLENAME",
+      "I-MIDDLENAME",
+      "E-MIDDLENAME",
+      "S-MIDDLENAME",
+      "B-OCCUPATION",
+      "I-OCCUPATION",
+      "E-OCCUPATION",
+      "S-OCCUPATION",
+      "B-ORDINALDIRECTION",
+      "I-ORDINALDIRECTION",
+      "E-ORDINALDIRECTION",
+      "S-ORDINALDIRECTION",
+      "B-ORGANIZATION",
+      "I-ORGANIZATION",
+      "E-ORGANIZATION",
+      "S-ORGANIZATION",
+      "B-PASSWORD",
+      "I-PASSWORD",
+      "E-PASSWORD",
+      "S-PASSWORD",
+      "B-PHONE",
+      "I-PHONE",
+      "E-PHONE",
+      "S-PHONE",
+      "B-PIN",
+      "I-PIN",
+      "E-PIN",
+      "S-PIN",
+      "B-PREFIX",
+      "I-PREFIX",
+      "E-PREFIX",
+      "S-PREFIX",
+      "B-SECONDARYADDRESS",
+      "I-SECONDARYADDRESS",
+      "E-SECONDARYADDRESS",
+      "S-SECONDARYADDRESS",
+      "B-SEX",
+      "I-SEX",
+      "E-SEX",
+      "S-SEX",
+      "B-SSN",
+      "I-SSN",
+      "E-SSN",
+      "S-SSN",
+      "B-STATE",
+      "I-STATE",
+      "E-STATE",
+      "S-STATE",
+      "B-STREET",
+      "I-STREET",
+      "E-STREET",
+      "S-STREET",
+      "B-TIME",
+      "I-TIME",
+      "E-TIME",
+      "S-TIME",
+      "B-URL",
+      "I-URL",
+      "E-URL",
+      "S-URL",
+      "B-USERAGENT",
+      "I-USERAGENT",
+      "E-USERAGENT",
+      "S-USERAGENT",
+      "B-USERNAME",
+      "I-USERNAME",
+      "E-USERNAME",
+      "S-USERNAME",
+      "B-VIN",
+      "I-VIN",
+      "E-VIN",
+      "S-VIN",
+      "B-VRM",
+      "I-VRM",
+      "E-VRM",
+      "S-VRM",
+      "B-ZIPCODE",
+      "I-ZIPCODE",
+      "E-ZIPCODE",
+      "S-ZIPCODE"
+    ],
+    "inference_contract_version": 1
+  },
+  "classifier_bias": true,
+  "_name_or_path": "OpenMed/privacy-filter-multilingual-v2",
+  "_mlx_task": "token-classification",
+  "_mlx_family": "openai-privacy-filter",
+  "_mlx_model_type": "openai-privacy-filter",
+  "_mlx_runtime": {
+    "experimental": true,
+    "decode": "bioes-viterbi",
+    "tokenizer": "tiktoken"
+  },
+  "num_experts": 128,
+  "experts_per_token": 4,
+  "_mlx_viterbi_biases": {},
+  "rope_theta": 150000.0,
+  "rope_scaling_factor": 32.0,
+  "rope_ntk_alpha": 1.0,
+  "rope_ntk_beta": 32.0,
+  "encoding": "o200k_base",
+  "bidirectional_context": true,
+  "bidirectional_left_context": 64,
+  "bidirectional_right_context": 63,
+  "hidden_dropout_prob": 0.1,
+  "attention_probs_dropout_prob": 0.0,
+  "layer_norm_eps": 1e-12,
+  "swiglu_limit": 7.0,
+  "_mlx_weights_format": "safetensors"
+}

id2label.json ADDED Viewed

	@@ -0,0 +1,219 @@

+{
+  "0": "O",
+  "1": "B-ACCOUNTNAME",
+  "2": "I-ACCOUNTNAME",
+  "3": "E-ACCOUNTNAME",
+  "4": "S-ACCOUNTNAME",
+  "5": "B-AGE",
+  "6": "I-AGE",
+  "7": "E-AGE",
+  "8": "S-AGE",
+  "9": "B-AMOUNT",
+  "10": "I-AMOUNT",
+  "11": "E-AMOUNT",
+  "12": "S-AMOUNT",
+  "13": "B-BANKACCOUNT",
+  "14": "I-BANKACCOUNT",
+  "15": "E-BANKACCOUNT",
+  "16": "S-BANKACCOUNT",
+  "17": "B-BIC",
+  "18": "I-BIC",
+  "19": "E-BIC",
+  "20": "S-BIC",
+  "21": "B-BITCOINADDRESS",
+  "22": "I-BITCOINADDRESS",
+  "23": "E-BITCOINADDRESS",
+  "24": "S-BITCOINADDRESS",
+  "25": "B-BUILDINGNUMBER",
+  "26": "I-BUILDINGNUMBER",
+  "27": "E-BUILDINGNUMBER",
+  "28": "S-BUILDINGNUMBER",
+  "29": "B-CITY",
+  "30": "I-CITY",
+  "31": "E-CITY",
+  "32": "S-CITY",
+  "33": "B-COUNTY",
+  "34": "I-COUNTY",
+  "35": "E-COUNTY",
+  "36": "S-COUNTY",
+  "37": "B-CREDITCARD",
+  "38": "I-CREDITCARD",
+  "39": "E-CREDITCARD",
+  "40": "S-CREDITCARD",
+  "41": "B-CREDITCARDISSUER",
+  "42": "I-CREDITCARDISSUER",
+  "43": "E-CREDITCARDISSUER",
+  "44": "S-CREDITCARDISSUER",
+  "45": "B-CURRENCY",
+  "46": "I-CURRENCY",
+  "47": "E-CURRENCY",
+  "48": "S-CURRENCY",
+  "49": "B-CURRENCYCODE",
+  "50": "I-CURRENCYCODE",
+  "51": "E-CURRENCYCODE",
+  "52": "S-CURRENCYCODE",
+  "53": "B-CURRENCYNAME",
+  "54": "I-CURRENCYNAME",
+  "55": "E-CURRENCYNAME",
+  "56": "S-CURRENCYNAME",
+  "57": "B-CURRENCYSYMBOL",
+  "58": "I-CURRENCYSYMBOL",
+  "59": "E-CURRENCYSYMBOL",
+  "60": "S-CURRENCYSYMBOL",
+  "61": "B-CVV",
+  "62": "I-CVV",
+  "63": "E-CVV",
+  "64": "S-CVV",
+  "65": "B-DATE",
+  "66": "I-DATE",
+  "67": "E-DATE",
+  "68": "S-DATE",
+  "69": "B-DATEOFBIRTH",
+  "70": "I-DATEOFBIRTH",
+  "71": "E-DATEOFBIRTH",
+  "72": "S-DATEOFBIRTH",
+  "73": "B-EMAIL",
+  "74": "I-EMAIL",
+  "75": "E-EMAIL",
+  "76": "S-EMAIL",
+  "77": "B-ETHEREUMADDRESS",
+  "78": "I-ETHEREUMADDRESS",
+  "79": "E-ETHEREUMADDRESS",
+  "80": "S-ETHEREUMADDRESS",
+  "81": "B-EYECOLOR",
+  "82": "I-EYECOLOR",
+  "83": "E-EYECOLOR",
+  "84": "S-EYECOLOR",
+  "85": "B-FIRSTNAME",
+  "86": "I-FIRSTNAME",
+  "87": "E-FIRSTNAME",
+  "88": "S-FIRSTNAME",
+  "89": "B-GENDER",
+  "90": "I-GENDER",
+  "91": "E-GENDER",
+  "92": "S-GENDER",
+  "93": "B-GPSCOORDINATES",
+  "94": "I-GPSCOORDINATES",
+  "95": "E-GPSCOORDINATES",
+  "96": "S-GPSCOORDINATES",
+  "97": "B-HEIGHT",
+  "98": "I-HEIGHT",
+  "99": "E-HEIGHT",
+  "100": "S-HEIGHT",
+  "101": "B-IBAN",
+  "102": "I-IBAN",
+  "103": "E-IBAN",
+  "104": "S-IBAN",
+  "105": "B-IMEI",
+  "106": "I-IMEI",
+  "107": "E-IMEI",
+  "108": "S-IMEI",
+  "109": "B-IPADDRESS",
+  "110": "I-IPADDRESS",
+  "111": "E-IPADDRESS",
+  "112": "S-IPADDRESS",
+  "113": "B-JOBDEPARTMENT",
+  "114": "I-JOBDEPARTMENT",
+  "115": "E-JOBDEPARTMENT",
+  "116": "S-JOBDEPARTMENT",
+  "117": "B-JOBTITLE",
+  "118": "I-JOBTITLE",
+  "119": "E-JOBTITLE",
+  "120": "S-JOBTITLE",
+  "121": "B-LASTNAME",
+  "122": "I-LASTNAME",
+  "123": "E-LASTNAME",
+  "124": "S-LASTNAME",
+  "125": "B-LITECOINADDRESS",
+  "126": "I-LITECOINADDRESS",
+  "127": "E-LITECOINADDRESS",
+  "128": "S-LITECOINADDRESS",
+  "129": "B-MACADDRESS",
+  "130": "I-MACADDRESS",
+  "131": "E-MACADDRESS",
+  "132": "S-MACADDRESS",
+  "133": "B-MASKEDNUMBER",
+  "134": "I-MASKEDNUMBER",
+  "135": "E-MASKEDNUMBER",
+  "136": "S-MASKEDNUMBER",
+  "137": "B-MIDDLENAME",
+  "138": "I-MIDDLENAME",
+  "139": "E-MIDDLENAME",
+  "140": "S-MIDDLENAME",
+  "141": "B-OCCUPATION",
+  "142": "I-OCCUPATION",
+  "143": "E-OCCUPATION",
+  "144": "S-OCCUPATION",
+  "145": "B-ORDINALDIRECTION",
+  "146": "I-ORDINALDIRECTION",
+  "147": "E-ORDINALDIRECTION",
+  "148": "S-ORDINALDIRECTION",
+  "149": "B-ORGANIZATION",
+  "150": "I-ORGANIZATION",
+  "151": "E-ORGANIZATION",
+  "152": "S-ORGANIZATION",
+  "153": "B-PASSWORD",
+  "154": "I-PASSWORD",
+  "155": "E-PASSWORD",
+  "156": "S-PASSWORD",
+  "157": "B-PHONE",
+  "158": "I-PHONE",
+  "159": "E-PHONE",
+  "160": "S-PHONE",
+  "161": "B-PIN",
+  "162": "I-PIN",
+  "163": "E-PIN",
+  "164": "S-PIN",
+  "165": "B-PREFIX",
+  "166": "I-PREFIX",
+  "167": "E-PREFIX",
+  "168": "S-PREFIX",
+  "169": "B-SECONDARYADDRESS",
+  "170": "I-SECONDARYADDRESS",
+  "171": "E-SECONDARYADDRESS",
+  "172": "S-SECONDARYADDRESS",
+  "173": "B-SEX",
+  "174": "I-SEX",
+  "175": "E-SEX",
+  "176": "S-SEX",
+  "177": "B-SSN",
+  "178": "I-SSN",
+  "179": "E-SSN",
+  "180": "S-SSN",
+  "181": "B-STATE",
+  "182": "I-STATE",
+  "183": "E-STATE",
+  "184": "S-STATE",
+  "185": "B-STREET",
+  "186": "I-STREET",
+  "187": "E-STREET",
+  "188": "S-STREET",
+  "189": "B-TIME",
+  "190": "I-TIME",
+  "191": "E-TIME",
+  "192": "S-TIME",
+  "193": "B-URL",
+  "194": "I-URL",
+  "195": "E-URL",
+  "196": "S-URL",
+  "197": "B-USERAGENT",
+  "198": "I-USERAGENT",
+  "199": "E-USERAGENT",
+  "200": "S-USERAGENT",
+  "201": "B-USERNAME",
+  "202": "I-USERNAME",
+  "203": "E-USERNAME",
+  "204": "S-USERNAME",
+  "205": "B-VIN",
+  "206": "I-VIN",
+  "207": "E-VIN",
+  "208": "S-VIN",
+  "209": "B-VRM",
+  "210": "I-VRM",
+  "211": "E-VRM",
+  "212": "S-VRM",
+  "213": "B-ZIPCODE",
+  "214": "I-ZIPCODE",
+  "215": "E-ZIPCODE",
+  "216": "S-ZIPCODE"
+}

openmed-mlx.json ADDED Viewed

	@@ -0,0 +1,31 @@

+{
+  "format": "openmed-mlx",
+  "format_version": 2,
+  "task": "token-classification",
+  "family": "openai-privacy-filter",
+  "source_model_id": "OpenMed/privacy-filter-multilingual-v2",
+  "config_path": "config.json",
+  "label_map_path": "id2label.json",
+  "preferred_weights": "weights.safetensors",
+  "fallback_weights": [
+    "weights.npz"
+  ],
+  "available_weights": [
+    "weights.safetensors"
+  ],
+  "weights_format": "safetensors",
+  "quantization": null,
+  "max_sequence_length": 131072,
+  "tokenizer": {
+    "path": ".",
+    "files": [
+      "tokenizer.json",
+      "tokenizer_config.json"
+    ]
+  },
+  "runtime": {
+    "experimental": true,
+    "decode": "bioes-viterbi",
+    "tokenizer": "tiktoken"
+  }
+}

tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0614fe83cadab421296e664e1f48f4261fa8fef6e03e63bb75c20f38e37d07d3
+size 27868174

tokenizer_config.json ADDED Viewed

	@@ -0,0 +1,12 @@

+{
+  "backend": "tokenizers",
+  "eos_token": "<|endoftext|>",
+  "is_local": false,
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 128000,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "TokenizersBackend"
+}

weights.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:67393657701052bb7cd3414c468cf69b45f773e4cdb8ef1afebd436754c69b11
+size 2799220153