Instructions to use bartowski/Marco-o1-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use bartowski/Marco-o1-GGUF with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="bartowski/Marco-o1-GGUF", filename="Marco-o1-IQ2_M.gguf", )
llm.create_chat_completion( messages = [ { "role": "user", "content": "What is the capital of France?" } ] ) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use bartowski/Marco-o1-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf bartowski/Marco-o1-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf bartowski/Marco-o1-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf bartowski/Marco-o1-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf bartowski/Marco-o1-GGUF:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf bartowski/Marco-o1-GGUF:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf bartowski/Marco-o1-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf bartowski/Marco-o1-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf bartowski/Marco-o1-GGUF:Q4_K_M
Use Docker
docker model run hf.co/bartowski/Marco-o1-GGUF:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use bartowski/Marco-o1-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "bartowski/Marco-o1-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bartowski/Marco-o1-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/bartowski/Marco-o1-GGUF:Q4_K_M
- Ollama
How to use bartowski/Marco-o1-GGUF with Ollama:
ollama run hf.co/bartowski/Marco-o1-GGUF:Q4_K_M
- Unsloth Studio
How to use bartowski/Marco-o1-GGUF with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for bartowski/Marco-o1-GGUF to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for bartowski/Marco-o1-GGUF to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for bartowski/Marco-o1-GGUF to start chatting
- Atomic Chat new
- Docker Model Runner
How to use bartowski/Marco-o1-GGUF with Docker Model Runner:
docker model run hf.co/bartowski/Marco-o1-GGUF:Q4_K_M
- Lemonade
How to use bartowski/Marco-o1-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull bartowski/Marco-o1-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.Marco-o1-GGUF-Q4_K_M
List all available models
lemonade list
Use MCTS in the prompt to get a proper results ;)
#3
by mirek190 - opened
Use MCTS , I am using llamacpp with version Q8
which is bigger 9.11 or 9.9, use a MCTS (step) /
Certainly! Let's analyze whether 9.11 is bigger than 9.9 using the Monte Carlo Tree Search (MCTS) approach, even though MCTS is typically used for decision-making in complex environments like game trees. However, for this simple comparison, I'll adapt the concept to a more straightforward method that resembles MCTS steps.
**Step 1: Define the Problem**
We need to determine whether 9.11 is greater than 9.9.
**Step 2: Initialize the Search Tree**
Imagine we have a tree where each node represents a comparison between two numbers. The root node is our initial comparison between 9.11 and 9.9.
**Step 3: Expand the Tree**
At the root node, we expand by considering different aspects to compare 9.11 and 9.9:
- **First Digit Comparison:** Compare the whole number parts.
- **Second Digit Comparison:** If the whole numbers are equal, compare the tenths place.
- **Third Digit Comparison:** If necessary, compare the hundredths place.
**Step 4: Simulate Multiple Outcomes**
Since MCTS relies on simulations, we'll simulate multiple scenarios:
1. **Scenario 1:** 9.11 vs. 9.9
- Compare whole numbers: 9 vs. 9 → Equal.
- Compare tenths place: 1 vs. 9 → 1 < 9.
- Conclusion: 9.11 < 9.9.
2. **Scenario 2:** 9.9 vs. 9.11
- Compare whole numbers: 9 vs. 9 → Equal.
- Compare tenths place: 9 vs. 1 → 9 > 1.
- Conclusion: 9.9 > 9.11.
3. **Scenario 3:** 9.11 vs. 9.9
- Same as Scenario 1, leading to the same conclusion.
**Step 5: Backpropagation**
After simulating multiple scenarios, we update the tree to reflect the outcomes:
- The path leading to 9.11 being less than 9.9 is confirmed in all simulations.
- The path leading to 9.9 being greater than 9.11 is also confirmed.
**Step 6: Decision Making**
Based on the simulations, it's clear that 9.11 is not greater than 9.9.
**Conclusion**
Through this adapted MCTS approach, we've determined that 9.11 is less than 9.9 by comparing their whole numbers and decimal places step by step.
**Alternative Approach: Direct Comparison**
For clarity, let's compare the two numbers directly:
- 9.11 has a value of nine and one tenth.
- 9.9 has a value of nine and nine tenths.
Since nine tenths (0.9) is greater than one tenth (0.1), it follows that 9.9 > 9.11.
**Final Answer:**
9.9 is bigger than 9.11.