Token Classification
Transformers.js
ONNX
Safetensors
openai_privacy_filter
privacy
pii
webgpu
quantized
Mixture of Experts
Instructions to use nisten/privacy-filter-nemotron-v2-ONNX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers.js
How to use nisten/privacy-filter-nemotron-v2-ONNX with Transformers.js:
// npm i @huggingface/transformers import { pipeline } from '@huggingface/transformers'; // Allocate pipeline const pipe = await pipeline('token-classification', 'nisten/privacy-filter-nemotron-v2-ONNX');
Upload PARITY.md with huggingface_hub
Browse files
PARITY.md
CHANGED
|
@@ -263,3 +263,92 @@ Browser-load bugs fixed to get here (details in CLAUDE.md worklog cont. 7):
|
|
| 263 |
`env.allowLocalModels` browser default, upstream `use_external_data_format`
|
| 264 |
catch-all key forcing a phantom `model_quantized.onnx_data` fetch (hang at
|
| 265 |
100%), `.mjs` served as octet-stream breaking the webgpu EP's dynamic import.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 263 |
`env.allowLocalModels` browser default, upstream `use_external_data_format`
|
| 264 |
catch-all key forcing a phantom `model_quantized.onnx_data` fetch (hang at
|
| 265 |
100%), `.mjs` served as octet-stream breaking the webgpu EP's dynamic import.
|
| 266 |
+
|
| 267 |
+
## fp16 β parity vs PyTorch fp32 (model_fp16.onnx)
|
| 268 |
+
|
| 269 |
+
- logits max abs diff: **1.491**
|
| 270 |
+
- per-token argmax agreement: **775/775 (100.00%)**
|
| 271 |
+
- span exact match (start,end,label): **61/61 (100.0%)** β 0 only-PyTorch, 0 only-ONNX
|
| 272 |
+
- gold coverage: PyTorch **52/61**, fp16 **52/61**
|
| 273 |
+
- critical gold spans missed by fp16: **0**
|
| 274 |
+
|
| 275 |
+
## bench10 β 10 handwritten PII examples, CAUGHT (any-label overlap) / exact-label
|
| 276 |
+
|
| 277 |
+
| ex | gold item (label) | pytorch | fp16 |
|
| 278 |
+
|---|---|---|---|
|
| 279 |
+
| 1 | `kai.nakamura` (unique_id) | CAUGHT* | CAUGHT* |
|
| 280 |
+
| 1 | `Tr0ub4dor&3` (password) | CAUGHT | CAUGHT |
|
| 281 |
+
| 1 | `kai.nakamura@example.io` (email) | CAUGHT | CAUGHT |
|
| 282 |
+
| 2 | `5425 2334 3010 9903` (credit_debit_card) | CAUGHT | CAUGHT |
|
| 283 |
+
| 2 | `11/27` (date) | CAUGHT | CAUGHT |
|
| 284 |
+
| 2 | `833` (cvv) | CAUGHT | CAUGHT |
|
| 285 |
+
| 3 | `078-05-1120` (ssn) | CAUGHT | CAUGHT |
|
| 286 |
+
| 3 | `1EG4-TE5-MK72` (health_plan_beneficiary_number) | CAUGHT | CAUGHT |
|
| 287 |
+
| 4 | `2001:0db8:85a3:0000:0000:8a2e:0370:7334` (ipv6) | CAUGHT | CAUGHT |
|
| 288 |
+
| 4 | `3D:F2:C9:A6:B3:4F` (mac_address) | CAUGHT | CAUGHT |
|
| 289 |
+
| 5 | `sk-live-9f8B2kQ71LmZxWv4TgNpR3sY` (api_key) | CAUGHT | CAUGHT |
|
| 290 |
+
| 6 | `Rosa` (first_name) | CAUGHT | CAUGHT |
|
| 291 |
+
| 6 | `Delgado-Marquez` (last_name) | CAUGHT* | CAUGHT* |
|
| 292 |
+
| 6 | `+1 (312) 555-0147` (phone_number) | CAUGHT | CAUGHT |
|
| 293 |
+
| 6 | `44 Cedar Hollow Ln, Apt 9C` (street_address) | CAUGHT | CAUGHT |
|
| 294 |
+
| 6 | `Naperville` (city) | CAUGHT | CAUGHT |
|
| 295 |
+
| 6 | `IL` (state) | CAUGHT | CAUGHT |
|
| 296 |
+
| 6 | `60540` (postcode) | CAUGHT | CAUGHT |
|
| 297 |
+
| 7 | `07/04/1962` (date_of_birth) | CAUGHT* | CAUGHT* |
|
| 298 |
+
| 7 | `X4711982-B` (national_id) | CAUGHT | CAUGHT |
|
| 299 |
+
| 7 | `Lisbon` (city) | **MISSED** | **MISSED** |
|
| 300 |
+
| 7 | `2026-09-18` (date) | CAUGHT* | CAUGHT* |
|
| 301 |
+
| 8 | `DE89 3704 0044 0532 0130 00` (account_number) | CAUGHT | CAUGHT |
|
| 302 |
+
| 8 | `021000021` (bank_routing_number) | CAUGHT | CAUGHT |
|
| 303 |
+
| 8 | `646-555-0199` (phone_number) | CAUGHT | CAUGHT |
|
| 304 |
+
| 9 | `KTX-4092` (license_plate) | CAUGHT* | CAUGHT* |
|
| 305 |
+
| 9 | `1HGBH41JXMN109186` (vehicle_identifier) | CAUGHT | CAUGHT |
|
| 306 |
+
| 9 | `45.5231, -122.6765` (coordinate) | CAUGHT | CAUGHT |
|
| 307 |
+
| 10 | `_wid=eyJhbGciOiJIUzI1NiJ9.aWQ9NzcyNA.4vX2` (http_cookie) | CAUGHT | CAUGHT |
|
| 308 |
+
| 10 | `wellness.example.com` (url) | **MISSED** | **MISSED** |
|
| 309 |
+
| 10 | `E-100482` (employee_id) | CAUGHT | CAUGHT |
|
| 310 |
+
|
| 311 |
+
**pytorch: 29/31 caught**
|
| 312 |
+
|
| 313 |
+
**fp16: 29/31 caught**
|
| 314 |
+
|
| 315 |
+
`*` = caught with a different (but redacting) label
|
| 316 |
+
|
| 317 |
+
## Phase 4b: q8 / mixed48 / fp16 in the browser (supersedes Phase 4)
|
| 318 |
+
|
| 319 |
+
Phase 4 concluded q8 and mixed48 were native-ORT-only. Two changes overturned
|
| 320 |
+
that β the big variants now run in-browser:
|
| 321 |
+
|
| 322 |
+
1. **External data + 64-byte alignment** (`export/externalize.py`). Single-file
|
| 323 |
+
variants forced ort-web to parse the whole protobuf inside the 4 GB wasm32
|
| 324 |
+
heap (q8 β `std::bad_alloc`) and left initializers at unaligned file offsets
|
| 325 |
+
(mixed48 β `operation does not support unaligned accesses`). Re-saved as
|
| 326 |
+
`<name>.onnx` + `<name>.onnx_data` with every tensor padded to a 64-byte
|
| 327 |
+
boundary, both failures disappear. Session options `enableCpuMemArena:false`
|
| 328 |
+
+ `enableMemPattern:false` trim peak allocation further.
|
| 329 |
+
2. **ort-web JSPI bundle for WebGPU** (`ort.jspi.bundle.min.mjs`). The default
|
| 330 |
+
(asyncify) bundle runs QMoE on the CPU via async round-trips β mixed48 loads
|
| 331 |
+
but inference is ~91 s. The JSPI bundle has the **native WebGPU QMoE kernel**
|
| 332 |
+
(expert weights resident in VRAM), taking mixed48 to 488 ms and q8 to 817 ms.
|
| 333 |
+
The worker picks the bundle from `src/lib/variants.ts` (`bundleFor`): q8 and
|
| 334 |
+
mixed48 on WebGPU β JSPI; q4 and all CPU β default (broad compatibility;
|
| 335 |
+
JSPI needs Chrome/Edge β₯ 137).
|
| 336 |
+
|
| 337 |
+
Measured in-browser matrix (Chrome 149, GTX 1660 Ti / Turing, 6 GB VRAM,
|
| 338 |
+
ort-web 1.27.0; spans correct in every β cell):
|
| 339 |
+
|
| 340 |
+
| variant | GPU (WebGPU) | CPU (wasm) |
|
| 341 |
+
|---|---|---|
|
| 342 |
+
| q4 (0.92 GB) | β default bundle Β· load 29 s Β· p50 134 ms Β· +825 MiB VRAM | β no wasm `GatherBlockQuantized` (4-bit Gather) kernel |
|
| 343 |
+
| q8 (1.98 GB) | β **JSPI** Β· load 152 s cold/32 s warm Β· 817 ms Β· +2849 MiB VRAM | β default bundle Β· load 36 s Β· **357 ms** |
|
| 344 |
+
| mixed48 (1.66 GB) | β **JSPI** Β· load 121 s Β· 488 ms Β· +1368 MiB VRAM | β default bundle Β· ~370 ms |
|
| 345 |
+
| fp16 (2.82 GB) | β *only on `shader-f16` GPUs* β this Turing card lacks it β "device (webgpu) does not support fp16"; works on RTX 20xx+ / Apple Silicon | β impractical (fp16 has no CPU compute advantage; 2.82 GB session stalls) |
|
| 346 |
+
|
| 347 |
+
Notes:
|
| 348 |
+
- **q4 stays the recommended default**: smallest, GPU-only, broadest bundle
|
| 349 |
+
compatibility, lowest VRAM. q8 is the accuracy pick when VRAM allows.
|
| 350 |
+
- fp16-on-WebGPU is a genuine capability (native-precision, no dequant) but is
|
| 351 |
+
gated on the GPU exposing `shader-f16`; the app surfaces the clear ort error
|
| 352 |
+
on GPUs that don't.
|
| 353 |
+
- The picker (`public/index.html` + `src/lib/variants.ts`) disables combos that
|
| 354 |
+
can't run β q4 forces GPU; CPU is offered for q8/mixed48/fp16.
|