Instructions to use liminerity/Mistral-quiet-star-GGUF-f16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use liminerity/Mistral-quiet-star-GGUF-f16 with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("liminerity/Mistral-quiet-star-GGUF-f16", dtype="auto") - llama-cpp-python
How to use liminerity/Mistral-quiet-star-GGUF-f16 with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="liminerity/Mistral-quiet-star-GGUF-f16", filename="Mistral-quiet-star-GGUF-f16-unsloth.F16.gguf", )
output = llm( "Once upon a time,", max_tokens=512, echo=True ) print(output)
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use liminerity/Mistral-quiet-star-GGUF-f16 with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf liminerity/Mistral-quiet-star-GGUF-f16:F16 # Run inference directly in the terminal: llama cli -hf liminerity/Mistral-quiet-star-GGUF-f16:F16
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf liminerity/Mistral-quiet-star-GGUF-f16:F16 # Run inference directly in the terminal: llama cli -hf liminerity/Mistral-quiet-star-GGUF-f16:F16
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf liminerity/Mistral-quiet-star-GGUF-f16:F16 # Run inference directly in the terminal: ./llama-cli -hf liminerity/Mistral-quiet-star-GGUF-f16:F16
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf liminerity/Mistral-quiet-star-GGUF-f16:F16 # Run inference directly in the terminal: ./build/bin/llama-cli -hf liminerity/Mistral-quiet-star-GGUF-f16:F16
Use Docker
docker model run hf.co/liminerity/Mistral-quiet-star-GGUF-f16:F16
- LM Studio
- Jan
- Ollama
How to use liminerity/Mistral-quiet-star-GGUF-f16 with Ollama:
ollama run hf.co/liminerity/Mistral-quiet-star-GGUF-f16:F16
- Unsloth Studio
How to use liminerity/Mistral-quiet-star-GGUF-f16 with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for liminerity/Mistral-quiet-star-GGUF-f16 to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for liminerity/Mistral-quiet-star-GGUF-f16 to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for liminerity/Mistral-quiet-star-GGUF-f16 to start chatting
- Atomic Chat new
- Docker Model Runner
How to use liminerity/Mistral-quiet-star-GGUF-f16 with Docker Model Runner:
docker model run hf.co/liminerity/Mistral-quiet-star-GGUF-f16:F16
- Lemonade
How to use liminerity/Mistral-quiet-star-GGUF-f16 with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull liminerity/Mistral-quiet-star-GGUF-f16:F16
Run and chat with the model
lemonade run user.Mistral-quiet-star-GGUF-f16-F16
List all available models
lemonade list
QSTAR
I think this model is proof of my theory that you dont need a special architecture to train a llm to reason. The techniques i imployed to make this could be greatly expanded on and coupled with a agent system to achieve a functioning low cost agi system. -Steps: -Loss:
356 - 0.014300
357 - 0.012400
358 - 0.016800
359 - 0.022200
360 - 0.015000
361 - 0.018300
362 - 0.016000
363 - 0.019000
364 - 0.017600
365 - 0.015600
366 - 0.016700
367 - 0.016400
368 - 0.016600
369 - 0.017600
heres a conversation i had with the model:
system prompt: "You are a helpful assistant who thinks before speaking"
user: "hi there. id like to build a time machine. how might i go about achieving this?"
assistant: "|startthought| To build a time machine, I need to figure out how to move through time just like we move through space. That means finding a way to change my frame of reference from the stationary Earth to one that's moving forward in time. |endthought| One approach is to use Einstein's theory of relativity and massive objects with strong gravitational fields, like black holes or neutron stars. By warping spacetime, these objects can slow down time for an observer passing by. Another idea is to explore quantum mechanics and the notion of entanglement between particles. If I could create a pair of entangled particles and keep one, the other might appear elsewhere in space-time, effectively teleporting it through time. Yet another approach is to look at cosmic rays and high-energy particle collisions, which can create short-lived particles that seem to violate the forward-in-time direction of time. Understanding these phenomena might lead to a way to control time flow. Ultimately, building a functional time machine will require advancing our understanding of these fundamental physical principles and finding a way to apply them technologically. It's a grand challenge that has captivated scientists and science fiction writers alike for centuries."
user: "how might we make use of this phenomena to create a functioning time machine?"
assistant: "|startthought| To make use of these phenomenas to create a funcitoning time machine, we need to find a way to control and harness them technologically. That could mean developing advanced gravitational lensing techniques to manipulate space-time around massive objects, exploring quantum entanglement in more complex systems, or finding a way to generate the high-energy particle collision conditions needed to create tachyons. It's likely that we'll need to understand and combine multiple approaches to make a functional time machine a reality. |endthought| One approach might be to focus on developing advanced gravitational lensing techniques. This could allow us to control the warping of space-time around massive objects like black holes or neutrino stars, creating stable time vortexes that we can pass through. Another idea is to explore quantum entanglement in more complex systems. If we could create and maintain a large-scale entangled state between two particles, we might be able to use one particle as a clock while the other appears elsewhere in space-time, effectively teleporting it through time. Yet another approach is to look at high-energy particle collision phenomenas like those created in large hadron colliders or supernova explosions. These events can generate short-lived particles called tachyons, which seem to move faster than the speed of light and violate the forward-in-time direction of time. If we could understand and control this phenomena, we might be able to create a stable source of tachyons that could be used to power a time machine. Ultimately, making a functional time machine will require advancing our understanding of these fundamental physical principles and finding a way to apply them technologically. It's a grand challenge that has captivaed scientists and science fiction writers alike for centuries."
the data set will need more work but this shows insane potential
Uploaded model
- Developed by: liminerity
- License: apache-2.0
- Finetuned from model : unsloth/mistral-7b-bnb-4bit This model was trained on a alpaca based dataset i made using perplexity and claude 3. The dataset is meant to encourage the llm the think before speaking This mistral model was trained 2x faster with Unsloth and Huggingface's TRL library.
- Downloads last month
- 101
16-bit
Model tree for liminerity/Mistral-quiet-star-GGUF-f16
Base model
unsloth/mistral-7b-bnb-4bit
ollama run hf.co/liminerity/Mistral-quiet-star-GGUF-f16:F16