darthcrawl/skyfall-31b-v4.2-mlx-4bit

Skyfall-31B-v4.2 · MLX 4-bit

4-bit MLX quantization of TheDrummer/Skyfall-31B-v4.2, produced with mlx_lm.convert. Apple Silicon only.

Use

pip install -U mlx-lm
mlx_lm.generate \
  --model darthcrawl/skyfall-31b-v4.2-mlx-4bit \
  --prompt "Continue this scene: a private investigator returns to her office at 2 AM and finds the door already unlocked."

Or from Python:

from mlx_lm import load, generate

model, tok = load("darthcrawl/skyfall-31b-v4.2-mlx-4bit")
print(generate(model, tok, "Continue this scene: ...", max_tokens=512))

Variants

Repo	Bits	Approx Size
...mlx-4bit	4	~17 GB
...mlx-6bit	6	~25 GB
...mlx-8bit	8	~33 GB

License

Apache 2.0, inherited from the base.