Instructions to use DavidAU/DarkForest-20B-V3-Ultra-Quality-ImatP2-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use DavidAU/DarkForest-20B-V3-Ultra-Quality-ImatP2-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf DavidAU/DarkForest-20B-V3-Ultra-Quality-ImatP2-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf DavidAU/DarkForest-20B-V3-Ultra-Quality-ImatP2-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf DavidAU/DarkForest-20B-V3-Ultra-Quality-ImatP2-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf DavidAU/DarkForest-20B-V3-Ultra-Quality-ImatP2-GGUF:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf DavidAU/DarkForest-20B-V3-Ultra-Quality-ImatP2-GGUF:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf DavidAU/DarkForest-20B-V3-Ultra-Quality-ImatP2-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf DavidAU/DarkForest-20B-V3-Ultra-Quality-ImatP2-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf DavidAU/DarkForest-20B-V3-Ultra-Quality-ImatP2-GGUF:Q4_K_M
Use Docker
docker model run hf.co/DavidAU/DarkForest-20B-V3-Ultra-Quality-ImatP2-GGUF:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use DavidAU/DarkForest-20B-V3-Ultra-Quality-ImatP2-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "DavidAU/DarkForest-20B-V3-Ultra-Quality-ImatP2-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "DavidAU/DarkForest-20B-V3-Ultra-Quality-ImatP2-GGUF", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/DavidAU/DarkForest-20B-V3-Ultra-Quality-ImatP2-GGUF:Q4_K_M
- Ollama
How to use DavidAU/DarkForest-20B-V3-Ultra-Quality-ImatP2-GGUF with Ollama:
ollama run hf.co/DavidAU/DarkForest-20B-V3-Ultra-Quality-ImatP2-GGUF:Q4_K_M
- Unsloth Studio
How to use DavidAU/DarkForest-20B-V3-Ultra-Quality-ImatP2-GGUF with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for DavidAU/DarkForest-20B-V3-Ultra-Quality-ImatP2-GGUF to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for DavidAU/DarkForest-20B-V3-Ultra-Quality-ImatP2-GGUF to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for DavidAU/DarkForest-20B-V3-Ultra-Quality-ImatP2-GGUF to start chatting
- Docker Model Runner
How to use DavidAU/DarkForest-20B-V3-Ultra-Quality-ImatP2-GGUF with Docker Model Runner:
docker model run hf.co/DavidAU/DarkForest-20B-V3-Ultra-Quality-ImatP2-GGUF:Q4_K_M
- Lemonade
How to use DavidAU/DarkForest-20B-V3-Ultra-Quality-ImatP2-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull DavidAU/DarkForest-20B-V3-Ultra-Quality-ImatP2-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.DarkForest-20B-V3-Ultra-Quality-ImatP2-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Ultra High Quality - 20 B Dark Forest Version 3.0 - 32 bit upscale - Imatrix Plus 2
Fully rebuilt from master files, including full merge(s) to maintain full 32 bit precision right up until it is compressed into GGUF files which results on a top to bottom upgrade.
The result is superior performance in instruction following, reasoning, depth, nuance and emotion.
This is the Imatrix Plus 2 repo. Imatrix Plus 2 uses the Ultra Quality 32 bit upscale as a base with applied in house datasets to further enhance the model's quality and performance.
NOTE: There are three original versions of "Dark Forest 20B", this is an upscale of the third version, with links below to 1st and 2nd versions also upscaled.
On average this means a q4km operates at Q6 levels and Q6 and Q8 exceeds original model full precision performance.
Perplexity drop (lower is better) is close to 10% (over 752 points for q4km) for all quants.
That means precision has been enhanced for all 20 billion parameters which affects "brain density" / "function", instruction following and output quality.
Imatrix quants to follow shortly.
For more details, including a list of enhancements see our other 32 bit upscale of "Space Whale 20B" rebuild here:
[ https://huggingface.co/DavidAU/Psyonic-Cetacean-Ultra-Quality-20b-GGUF ]
For Version 1 of Dark Forest Ultra Quality 32 bit upscale please go here:
[ https://huggingface.co/DavidAU/Dark-Forest-V1-Ultra-Quality-20b-GGUF ]
For Version 1 of Dark Forest Ultra Quality 32 bit upscale please go here:
[ https://huggingface.co/TeeZee/DarkForest-20B-v2.0 ]
Special thanks to "TEEZEE" for making a both fantasic models of "Dark Forest".
Info from the original model card:
Warning: This model can produce NSFW content!
Results:
- main difference to v1.0 - model has much better sense of humor.
- produces SFW nad NSFW content without issues, switches context seamlessly.
- good at following instructions.
- good at tracking multiple characters in one scene.
- very creative, scenarios produced are mature and complicated, model doesn't shy from writing about PTSD, menatal issues or complicated relationships.
- NSFW output is more creative and suprising than typical limaRP output.
- definitely for mature audiences, not only because of vivid NSFW content but also because of overall maturity of stories it produces.
- This is NOT Harry Potter level storytelling.
For original model spec and information please visit:
[ https://huggingface.co/TeeZee/DarkForest-20B-v3.0 ]
Highest Quality Settings / Optimal Operation Guide / Parameters and Samplers
This a "Class 2" model:
For all settings used for this model (including specifics for its "class"), including example generation(s) and for advanced settings guide (which many times addresses any model issue(s)), including methods to improve model performance for all use case(s) as well as chat, roleplay and other use case(s) please see:
- Downloads last month
- 208
1-bit
2-bit
3-bit
4-bit
5-bit
6-bit