File size: 2,692 Bytes
33322c6 330f5bf 33322c6 330f5bf | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 | ---
language: en
license: cc-by-nc-4.0
library_name: transformers.js
tags:
- onnx
- transformers.js
- grammar-correction
- t5
- coedit
- quantized
- text-generation
- text-generation-inference
pipeline_tag: text-generation
base_model:
- grammarly/coedit-large
---
# CoEdIT-Large ONNX (INT8 Quantized)
ONNX export of [grammarly/coedit-large](https://huggingface.co/grammarly/coedit-large) (770M params, flan-t5-large) optimized for [`@huggingface/transformers`](https://github.com/huggingface/transformers.js) v3+.
Includes both FP32 and INT8 quantized versions. The INT8 quantized model is ~780MB total and runs in browser via WASM or WebGPU.
## Usage
```js
import { pipeline } from '@huggingface/transformers';
const pipe = await pipeline('text2text-generation', 'rabden/coedit-large-onnx', {
quantized: true,
dtype: 'q8',
});
const result = await pipe(
'Fix grammatical errors in this sentence: ' +
'The protocol utilize a novel encryption scheme that ensure data integrity across multiple node.',
{
max_new_tokens: 64,
}
);
console.log(result[0].generated_text);
// "The protocol utilizes a novel encryption scheme that ensures data integrity across multiple nodes."
```
## Generation Config
The model has `repetition_penalty: 1.5` baked in by default to prevent repeated output. You can override it:
```js
const result = await pipe(text, {
max_new_tokens: 64,
repetition_penalty: 1.0, // disable
});
```
## Files
| File | Size | Description |
|------|------|-------------|
| `onnx/encoder_model_quantized.onnx` | 326 MB | INT8 quantized encoder |
| `onnx/decoder_model_merged_quantized.onnx` | 454 MB | INT8 quantized decoder (with lm_head) |
| `onnx/encoder_model.onnx` | 1302 MB | FP32 encoder |
| `onnx/decoder_model_merged.onnx` | 1812 MB | FP32 decoder (with lm_head) |
| `config.json` | — | T5 config |
| `generation_config.json` | — | Generation parameters |
| `tokenizer.json` / `spiece.model` | — | T5 tokenizer |
## Performance
Tested on Node.js (WASM backend, Intel Xeon, quantized):
- Load time: ~7s (cached)
- Inference: 300ms–1300ms per sentence (varies with length)
WebGPU backend is faster but requires browser with WebGPU support.
## Model Details
- **Base model:** `google/flan-t5-large` fine-tuned on CoEdIT dataset
- **Architecture:** T5 encoder-decoder (24 layers, d_model=1024, 16 heads)
- **Task:** Text editing via instruction tuning
- **Paper:** [CoEdIT: Text Editing by Task-Specific Instruction Tuning](https://arxiv.org/abs/2305.09857)
- **Original:** [grammarly/coedit-large](https://huggingface.co/grammarly/coedit-large) (gated, requires accepting terms)
## License
CC-BY-NC-4.0 (same as the original model). |