Is 1bit possible?
#1
by darthsider - opened
Is it possible to create 1bit version to fit in a single M3 Ultra 512 GB?
MLX only goes down to 2-bit, so you would need to look at reaping/pruning options to fit it on a single device. There are a couple of quants available that attempt to do this, but I haven't tested them personally.
darthsider changed discussion status to closed