Add files using upload-large-folder tool

README.md

@@ -0,0 +1,94 @@
+---
+pipeline_tag: text-generation
+library_name: mlx
+license: other
+base_model: kai-os/Grug-35B-A3B
+base_model_relation: quantized
+tags:
+- mlx
+- mlx-lm
+- qwen3
+- qwen3-next
+- mixture-of-experts
+- text-generation
+- quantized
+- 4-bit
+- 6-bit
+- 8-bit
+- apple-silicon
+---
+
+# Grug-35B-A3B MLX
+
+Apple Silicon MLX quantizations of
+[`kai-os/Grug-35B-A3B`](https://huggingface.co/kai-os/Grug-35B-A3B), packaged
+as a single Hugging Face repo with one folder per quantization level.
+
+This repository contains MLX safetensors folders, not GGUF files. The source
+model remains in the upstream Hugging Face repository.
+
+## Available variants
+
+| Variant | Folder | Quantization | Size | Best fit |
+| --- | --- | --- | ---: | --- |
+| MLX 8-bit | `mlx-8bit/` | affine, group size 64 | 34 GB | Highest-quality local MLX run. |
+| MLX 6-bit | `mlx-6bit/` | affine, group size 64 | 26 GB | Balanced memory and quality. |
+| MLX 4-bit | `mlx-4bit/` | affine, group size 64 | 18 GB | Smallest footprint and easiest local run. |
+
+Benchmarks will be added later.
+
+## Usage
+
+Download only the variant you want:
+
+```python
+from pathlib import Path
+from huggingface_hub import snapshot_download
+
+repo_id = "chanderbalaji/Grug-35B-A3B-MLX"
+variant = "mlx-4bit"
+
+snapshot = snapshot_download(
+    repo_id,
+    allow_patterns=[f"{variant}/*"],
+)
+model_path = Path(snapshot) / variant
+print(model_path)
+```
+
+Run with `mlx-lm`:
+
+```bash
+python -m mlx_lm.generate \
+  --model /path/to/downloaded/snapshot/mlx-4bit \
+  --prompt "Reply with a short explanation of what this model is." \
+  --max-tokens 256
+```
+
+## Local compatibility note
+
+The source config uses `model_type: qwen3_5_moe_text`. At conversion time,
+`mlx-lm 0.31.3` did not include a native loader for that exact model type, so
+the conversion used a local compatibility shim mapping the model to the
+existing Qwen3-Next style MLX implementation and adapting the published weight
+names.
+
+Consumers may need an `mlx-lm` build that supports `qwen3_5_moe_text`, or an
+equivalent local compatibility shim, until upstream support is available.
+
+## Provenance and attribution
+
+- Source model: [`kai-os/Grug-35B-A3B`](https://huggingface.co/kai-os/Grug-35B-A3B)
+- Relationship: MLX quantized derivatives of the source model
+- Source revision used locally: `f6349f9c7beba10ad44ce0210b0a0f6fba414a05`
+- Conversion tool: `mlx-lm 0.31.3`
+- Quantization mode: affine, group size 64
+
+Please refer to the source model card for upstream training details, intended
+use, limitations, acknowledgements, and license context.
+
+## Limitations
+
+Quantization can change output quality, numerical behavior, and edge-case
+performance. These files are intended for local MLX inference on Apple Silicon.
+Use the source model repo for the original Transformers/safetensors weights.