model.safetensors.index.json?
Hi,
I'm new to running local models so I apologize if i'm wasting time, but isn't an index required to run this model? I tried both on Exo as well as Inferencer with distributed compute across two 512GB Mac Studios with no luck.
Which version of Inferencer did you try running it on?
This is just an observation comment, not technical:
I can't say for certain, but I think one thing that was getting in my way was the network settings of my Macs. I was not being as pragmatic as I could be when I was trying to hunt down what my problems were. So I changed my settings to be...-> "Using DHCP with Manual Address" instead of Automatic DHCP. I'm pretty sure this helped me both with Exo and Inferencer.
I was using two 512GB Mac Studios to distribute the models across. I used both the models from Inferencer (the has quantized) as well as other models from Hugging Face to do the distribution of the models successfully.
I was then able to go further and connect to the distributed models and do inferencing from a third M2 Mac Ultra.
While it's only a comment, at least you know someone's achieved what you're looking to do, which I think is helpful anyway. At least I hope so.
Which version of Inferencer did you try running it on?
latest version, this was just a few hours ago
This is just an observation comment, not technical:
I can't say for certain, but I think one thing that was getting in my way was the network settings of my Macs. I was not being as pragmatic as I could be when I was trying to hunt down what my problems were. So I changed my settings to be...-> "Using DHCP with Manual Address" instead of Automatic DHCP. I'm pretty sure this helped me both with Exo and Inferencer.
I was using two 512GB Mac Studios to distribute the models across. I used both the models from Inferencer (the has quantized) as well as other models from Hugging Face to do the distribution of the models successfully.
I was then able to go further and connect to the distributed models and do inferencing from a third M2 Mac Ultra.While it's only a comment, at least you know someone's achieved what you're looking to do, which I think is helpful anyway. At least I hope so.
thank you; i am going to keep slamming my head against the wall and (hopefully) it will work <3
Can you provide a screenshot of the error you're seeing?