Create README.md

README.md

@@ -0,0 +1,57 @@
+---
+  license: other
+  base_model: Ewere/Qwen3-30B-A3B-abliterated-erotic
+  tags:
+    - quantized
+    - autoround
+    - int4
+    - qwen3
+    - vllm
+  quantization:
+    method: autoround
+    bits: 4
+---
+
+# Qwen3-30B AutoRound Int4
+
+4-bit AutoRound quantization of Qwen3-30B-A3B-abliterated-erotic.
+
+## Quantization Details
+
+- Method: AutoRound (SignRound optimization)
+- Bits: 4-bit (W4A16 symmetric)
+- Group Size: 128
+- Calibration: 512 samples from NeelNanda/pile-10k
+- Iterations: 200 (light mode)
+
+## Model Size
+
+- Original (FP16): ~60GB
+- Quantized (Int4): ~17GB
+- Compression: 3.5x
+
+## Usage with vLLM
+
+```python
+from vllm import LLM, SamplingParams
+
+llm = LLM(
+    model="yourusername/Qwen3-30B-AutoRound-Int4",
+    quantization="auto-round",
+    dtype="float16",
+    gpu_memory_utilization=0.9,
+    max_model_len=16384,
+    trust_remote_code=True,
+)
+
+sampling_params = SamplingParams(temperature=0.7, max_tokens=512)
+outputs = llm.generate(["Your prompt here"], sampling_params)
+
+Hardware Requirements
+
+- VRAM: 18-20GB (single GPU) or 2x RTX 3090 with tensor parallelism
+- Inference Speed: 35-55 tokens/sec on 2x RTX 3090
+
+Base Model
+
+Based on https://huggingface.co/Ewere/Qwen3-30B-A3B-abliterated-erotic