Frankenmerge?
#1
by yamatazen - opened
Is this a frankenmerge model?
Basically yeah. But instead of mixing tuned models, I got Nemo Instruct, duplicated some of its layers, and then finetuned it.
So this is not upscaled from the 12B Rocinante model?
Nope. Just plain old Nemo Instruct with the same dataset as Skyfall 4.2.
So you upscaled the Nemo Instruct model and the finetuned it?
Yes. Upscaling lets the model soak up new data without destroying the old weights during training.