chiaraboretti nielsr HF Staff commited on
Commit
0eeffab
·
1 Parent(s): 76716fa

Add pipeline tag, arxiv metadata and fix usage snippets (#1)

Browse files

- Add pipeline tag, arxiv metadata and fix usage snippets (edc44d3c98e87f7a1b7bba86acab72f8656f409a)


Co-authored-by: Niels Rogge <nielsr@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +9 -2
README.md CHANGED
@@ -1,4 +1,5 @@
1
  ---
 
2
  language:
3
  - en
4
  license: apache-2.0
@@ -11,8 +12,9 @@ tags:
11
  - qwen
12
  - llm
13
  - compression
14
- base_model: Qwen/Qwen3-32B
15
  base_model_relation: quantized
 
 
16
  ---
17
 
18
  <p align="center">
@@ -25,6 +27,9 @@ base_model_relation: quantized
25
  # A-SINQ 4-bit Quantized Qwen3-32B model
26
 
27
  This repository contains the official **4-bit quantized** version of the [`Qwen3-32B`](https://huggingface.co/Qwen/Qwen3-32B) model using the *calibrated* version of **SINQ (Sinkhorn-Normalized Quantization)** method.
 
 
 
28
  SINQ is a novel, fast and high-quality quantization method designed to make any Large Language Models smaller while keeping their accuracy almost intact.
29
 
30
  To support the project please put a star ⭐ in the official [SINQ](https://github.com/huawei-csl/SINQ) github repository.
@@ -48,7 +53,7 @@ To support the project please put a star ⭐ in the official [SINQ](https://gith
48
 
49
  ---
50
 
51
- # 🚀 Usage</span>
52
 
53
  ## Prerequisite
54
  Before running the quantization script, make sure the **SINQ** library is installed.
@@ -58,6 +63,7 @@ Installation instructions and setup details are available in the [SINQ official
58
  You can load and use the model with our wrapper based on the 🤗 Transformers library:
59
 
60
  ```python
 
61
  from transformers import AutoTokenizer
62
  from sinq.patch_model import AutoSINQHFModel
63
 
@@ -83,6 +89,7 @@ print(tokenizer.decode(out_ids[0], skip_special_tokens=True))
83
  The quantized model was obtained using the **SINQ** quantization library, following the steps below:
84
 
85
  ```python
 
86
  from transformers import AutoModelForCausalLM, AutoTokenizer
87
  from sinq.patch_model import AutoSINQHFModel
88
  from sinq.sinqlinear import BaseQuantizeConfig
 
1
  ---
2
+ base_model: Qwen/Qwen3-32B
3
  language:
4
  - en
5
  license: apache-2.0
 
12
  - qwen
13
  - llm
14
  - compression
 
15
  base_model_relation: quantized
16
+ pipeline_tag: text-generation
17
+ arxiv: 2509.22944
18
  ---
19
 
20
  <p align="center">
 
27
  # A-SINQ 4-bit Quantized Qwen3-32B model
28
 
29
  This repository contains the official **4-bit quantized** version of the [`Qwen3-32B`](https://huggingface.co/Qwen/Qwen3-32B) model using the *calibrated* version of **SINQ (Sinkhorn-Normalized Quantization)** method.
30
+
31
+ The method was introduced in the paper [SINQ: Sinkhorn-Normalized Quantization for Calibration-Free Low-Precision LLM Weights](https://huggingface.co/papers/2509.22944) by Lorenz K. Müller, Philippe Bich, Jiawei Zhuang, Ahmet Çelik, Luca Benfenati, and Lukas Cavigelli.
32
+
33
  SINQ is a novel, fast and high-quality quantization method designed to make any Large Language Models smaller while keeping their accuracy almost intact.
34
 
35
  To support the project please put a star ⭐ in the official [SINQ](https://github.com/huawei-csl/SINQ) github repository.
 
53
 
54
  ---
55
 
56
+ # 🚀 Usage
57
 
58
  ## Prerequisite
59
  Before running the quantization script, make sure the **SINQ** library is installed.
 
63
  You can load and use the model with our wrapper based on the 🤗 Transformers library:
64
 
65
  ```python
66
+ import torch
67
  from transformers import AutoTokenizer
68
  from sinq.patch_model import AutoSINQHFModel
69
 
 
89
  The quantized model was obtained using the **SINQ** quantization library, following the steps below:
90
 
91
  ```python
92
+ import torch
93
  from transformers import AutoModelForCausalLM, AutoTokenizer
94
  from sinq.patch_model import AutoSINQHFModel
95
  from sinq.sinqlinear import BaseQuantizeConfig