Fastino Nemotron 3.5 Lightning
Collection
8 items • Updated
Fastino-Nemotron-3.5-Lightning-Healthcare-MLX-4bit
This model was converted to MLX format from fastino/Fastino-Nemotron-3.5-Lightning-Healthcare using mlx-lm version 0.31.3.
| Property | Value |
|---|---|
| Quantization | 4-bit affine |
| Group size | 64 |
Use with mlx
pip install mlx-lm
mlx_lm.generate --model Fastino-Nemotron-3.5-Lightning-Healthcare-MLX-4bit --prompt "Hello"
from mlx_lm import load, generate
model, tokenizer = load("Fastino-Nemotron-3.5-Lightning-Healthcare-MLX-4bit")
prompt = "Hello"
if tokenizer.chat_template is not None:
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
response = generate(model, tokenizer, prompt=prompt, verbose=True)
- Downloads last month
- 16
Model size
5B params
Tensor type
BF16
·
U32 ·
F32 ·
Hardware compatibility
Log In to add your hardware
4-bit
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("nicolasembleton/Fastino-Nemotron-3.5-Lightning-Healthcare-MLX-4bit") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True)