NeuralDaredevil-8B-abliterated-GGUF

Instructions to use thunderdick/NeuralDaredevil-8B-abliterated-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use thunderdick/NeuralDaredevil-8B-abliterated-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf thunderdick/NeuralDaredevil-8B-abliterated-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf thunderdick/NeuralDaredevil-8B-abliterated-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf thunderdick/NeuralDaredevil-8B-abliterated-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf thunderdick/NeuralDaredevil-8B-abliterated-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf thunderdick/NeuralDaredevil-8B-abliterated-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf thunderdick/NeuralDaredevil-8B-abliterated-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf thunderdick/NeuralDaredevil-8B-abliterated-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf thunderdick/NeuralDaredevil-8B-abliterated-GGUF:Q4_K_M

Use Docker

docker model run hf.co/thunderdick/NeuralDaredevil-8B-abliterated-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use thunderdick/NeuralDaredevil-8B-abliterated-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "thunderdick/NeuralDaredevil-8B-abliterated-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "thunderdick/NeuralDaredevil-8B-abliterated-GGUF",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/thunderdick/NeuralDaredevil-8B-abliterated-GGUF:Q4_K_M

Ollama
How to use thunderdick/NeuralDaredevil-8B-abliterated-GGUF with Ollama:
```
ollama run hf.co/thunderdick/NeuralDaredevil-8B-abliterated-GGUF:Q4_K_M
```

Unsloth Studio

How to use thunderdick/NeuralDaredevil-8B-abliterated-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for thunderdick/NeuralDaredevil-8B-abliterated-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for thunderdick/NeuralDaredevil-8B-abliterated-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for thunderdick/NeuralDaredevil-8B-abliterated-GGUF to start chatting

Atomic Chat new
Docker Model Runner
How to use thunderdick/NeuralDaredevil-8B-abliterated-GGUF with Docker Model Runner:
```
docker model run hf.co/thunderdick/NeuralDaredevil-8B-abliterated-GGUF:Q4_K_M
```

Lemonade

How to use thunderdick/NeuralDaredevil-8B-abliterated-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull thunderdick/NeuralDaredevil-8B-abliterated-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.NeuralDaredevil-8B-abliterated-GGUF-Q4_K_M

List all available models

lemonade list

thunderdick

munish0838 commited on May 8

Commit

ca2140f

0 Parent(s):

Duplicate from QuantFactory/NeuralDaredevil-8B-abliterated-GGUF

Browse files

Co-authored-by: Munish Kumar <munish0838@users.noreply.huggingface.co>

Files changed (16) hide show

.gitattributes +49 -0
NeuralDaredevil-8B-abliterated.Q2_K.gguf +3 -0
NeuralDaredevil-8B-abliterated.Q3_K_L.gguf +3 -0
NeuralDaredevil-8B-abliterated.Q3_K_M.gguf +3 -0
NeuralDaredevil-8B-abliterated.Q3_K_S.gguf +3 -0
NeuralDaredevil-8B-abliterated.Q4_0.gguf +3 -0
NeuralDaredevil-8B-abliterated.Q4_1.gguf +3 -0
NeuralDaredevil-8B-abliterated.Q4_K_M.gguf +3 -0
NeuralDaredevil-8B-abliterated.Q4_K_S.gguf +3 -0
NeuralDaredevil-8B-abliterated.Q5_0.gguf +3 -0
NeuralDaredevil-8B-abliterated.Q5_1.gguf +3 -0
NeuralDaredevil-8B-abliterated.Q5_K_M.gguf +3 -0
NeuralDaredevil-8B-abliterated.Q5_K_S.gguf +3 -0
NeuralDaredevil-8B-abliterated.Q6_K.gguf +3 -0
NeuralDaredevil-8B-abliterated.Q8_0.gguf +3 -0
README.md +161 -0

.gitattributes ADDED Viewed

	@@ -0,0 +1,49 @@

+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+NeuralDaredevil-8B-abliterated.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+NeuralDaredevil-8B-abliterated.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+NeuralDaredevil-8B-abliterated.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+NeuralDaredevil-8B-abliterated.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+NeuralDaredevil-8B-abliterated.Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
+NeuralDaredevil-8B-abliterated.Q5_1.gguf filter=lfs diff=lfs merge=lfs -text
+NeuralDaredevil-8B-abliterated.Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
+NeuralDaredevil-8B-abliterated.Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+NeuralDaredevil-8B-abliterated.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+NeuralDaredevil-8B-abliterated.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+NeuralDaredevil-8B-abliterated.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+NeuralDaredevil-8B-abliterated.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+NeuralDaredevil-8B-abliterated.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+NeuralDaredevil-8B-abliterated.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text

NeuralDaredevil-8B-abliterated.Q2_K.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:47ca9f33deef6601a5db84d74c992d1a6aea3049d326cadc87fd7a7e1d6c231b
+size 3179130720

NeuralDaredevil-8B-abliterated.Q3_K_L.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:054ee469dd1f1bf97f3b495fef2c01c09ff3501a1092eb43990cbc7d45fe1c18
+size 4321955680

NeuralDaredevil-8B-abliterated.Q3_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:99374997817be791d8c0b616bc960725e6fe68712d754c353eee9926ee372d9c
+size 4018917216

NeuralDaredevil-8B-abliterated.Q3_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e2b81272251eed29ef22216fc8d4ac2808c295e8daf42cd7804a321e50650e3d
+size 3664498528

NeuralDaredevil-8B-abliterated.Q4_0.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e000a3b1b5c3259a0581b0780e85abc7e5d881657d6eb49dd570258905fd4779
+size 4661210976

NeuralDaredevil-8B-abliterated.Q4_1.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9a6cdaf1bbbf9b92943dad6116b3cd4a22b50e1f7fb6a69231cbdfca33e017e5
+size 5130252128

NeuralDaredevil-8B-abliterated.Q4_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:12f4af9d66817d7d300bd9a181e4fe66f7ecf7ea972049f2cbd0554cdc3ecf05
+size 4920733536

NeuralDaredevil-8B-abliterated.Q4_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:82b38b174cb17ae6f577a25983adc789922b8944eafb388db3f24fcc324ce4c5
+size 4692668256

NeuralDaredevil-8B-abliterated.Q5_0.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:028bb7fabddc2e13f98e8a19757031005952002b176495bf4ff01b17d8672464
+size 5599293280

NeuralDaredevil-8B-abliterated.Q5_1.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:86901a0e22874fc4a42f35568c5720033d8b40fa53203413c9913c964fc6f665
+size 6068334432

NeuralDaredevil-8B-abliterated.Q5_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a0c6e6450e37e4e681bbc14216bda0be4e5445633bca2635949908a37125076a
+size 5732986720

NeuralDaredevil-8B-abliterated.Q5_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:79822a7cd4e003615f9fc8f3ff0d4fc8fe800bd6225da605dfcd7bb764b1931d
+size 5599293280

NeuralDaredevil-8B-abliterated.Q6_K.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a81b3d5a20cd0dfcff5baf64caa20efd2eb83014e9ca7a3ae87e7caae3f3881f
+size 6596005728

NeuralDaredevil-8B-abliterated.Q8_0.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bfd63d210c889e48775341ba9fdb0a5ee02c1d1e7a0828697b9b4e1846c5b19f
+size 8540770144

README.md ADDED Viewed

	@@ -0,0 +1,161 @@

+---
+license: other
+tags:
+- dpo
+base_model: mlabonne/NeuralDaredevil-8B-abliterated
+datasets:
+- mlabonne/orpo-dpo-mix-40k
+model-index:
+- name: Daredevil-8B-abliterated-dpomix
+  results:
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: AI2 Reasoning Challenge (25-Shot)
+      type: ai2_arc
+      config: ARC-Challenge
+      split: test
+      args:
+        num_few_shot: 25
+    metrics:
+    - type: acc_norm
+      value: 69.28
+      name: normalized accuracy
+    source:
+      url: >-
+        https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=mlabonne/Daredevil-8B-abliterated-dpomix
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: HellaSwag (10-Shot)
+      type: hellaswag
+      split: validation
+      args:
+        num_few_shot: 10
+    metrics:
+    - type: acc_norm
+      value: 85.05
+      name: normalized accuracy
+    source:
+      url: >-
+        https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=mlabonne/Daredevil-8B-abliterated-dpomix
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: MMLU (5-Shot)
+      type: cais/mmlu
+      config: all
+      split: test
+      args:
+        num_few_shot: 5
+    metrics:
+    - type: acc
+      value: 69.1
+      name: accuracy
+    source:
+      url: >-
+        https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=mlabonne/Daredevil-8B-abliterated-dpomix
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: TruthfulQA (0-shot)
+      type: truthful_qa
+      config: multiple_choice
+      split: validation
+      args:
+        num_few_shot: 0
+    metrics:
+    - type: mc2
+      value: 60
+    source:
+      url: >-
+        https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=mlabonne/Daredevil-8B-abliterated-dpomix
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: Winogrande (5-shot)
+      type: winogrande
+      config: winogrande_xl
+      split: validation
+      args:
+        num_few_shot: 5
+    metrics:
+    - type: acc
+      value: 78.69
+      name: accuracy
+    source:
+      url: >-
+        https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=mlabonne/Daredevil-8B-abliterated-dpomix
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: GSM8k (5-shot)
+      type: gsm8k
+      config: main
+      split: test
+      args:
+        num_few_shot: 5
+    metrics:
+    - type: acc
+      value: 71.8
+      name: accuracy
+    source:
+      url: >-
+        https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=mlabonne/Daredevil-8B-abliterated-dpomix
+      name: Open LLM Leaderboard
+pipeline_tag: text-generation
+---
+# NeuralDaredevil-8B-abliterated-GGUF
+This is quantized version of [mlabonne/NeuralDaredevil-8B-abliterated](https://huggingface.co/mlabonne/NeuralDaredevil-8B-abliterated) created using llama.cpp
+# Model Description
+![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/gFEhcIDSKa3AWpkNfH91q.jpeg)
+This is a DPO fine-tune of [mlabonne/Daredevil-8-abliterated](https://huggingface.co/mlabonne/Daredevil-8B-abliterated), trained on one epoch of [mlabonne/orpo-dpo-mix-40k](https://huggingface.co/datasets/mlabonne/orpo-dpo-mix-40k).
+The DPO fine-tuning successfully recovers the performance loss due to the abliteration process, making it an excellent uncensored model.
+## 🔎 Applications
+NeuralDaredevil-8B-abliterated performs better than the Instruct model on my tests.
+You can use it for any application that doesn't require alignment, like role-playing. Tested on LM Studio using the "Llama 3" preset.
+## 🏆 Evaluation
+### Open LLM Leaderboard
+NeuralDaredevil-8B is the best-performing uncensored 8B model on the Open LLM Leaderboard (MMLU score).
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/HQtd51mJfVRhJ0lJFLceM.png)
+### Nous
+Evaluation performed using [LLM AutoEval](https://github.com/mlabonne/llm-autoeval). See the entire leaderboard [here](https://huggingface.co/spaces/mlabonne/Yet_Another_LLM_Leaderboard).
+| Model | Average | AGIEval | GPT4All | TruthfulQA | Bigbench |
+|---|---:|---:|---:|---:|---:|
+| [**mlabonne/NeuralDaredevil-8B-abliterated**](https://huggingface.co/mlabonne/NeuralDaredevil-8B-abliterated) [📄](https://gist.github.com/mlabonne/ae0bf16936cef900b72964b33c99edbc) | **55.87** | **43.73** | **73.6** | **59.36** | **46.8** |
+| [mlabonne/Daredevil-8B](https://huggingface.co/mlabonne/Daredevil-8B) [📄](https://gist.github.com/mlabonne/080f9c5f153ea57a7ab7d932cf896f21) | 55.87 | 44.13 | 73.52 | 59.05 | 46.77 |
+| [mlabonne/Daredevil-8B-abliterated](https://huggingface.co/mlabonne/Daredevil-8B-abliterated) [📄](https://gist.github.com/mlabonne/32cdd8460804662c856bcb2a20acd49e) | 55.06 | 43.29 | 73.33 | 57.47 | 46.17 |
+| [NousResearch/Hermes-2-Theta-Llama-3-8B](https://huggingface.co/NousResearch/Hermes-2-Theta-Llama-3-8B) [📄](https://gist.github.com/mlabonne/5df2a3051dd6eb3368a77b684635dc05) | 54.28 | 43.9 | 72.62 | 56.36 | 44.23 |
+| [openchat/openchat-3.6-8b-20240522](https://huggingface.co/openchat/openchat-3.6-8b-20240522) [📄](https://gist.github.com/mlabonne/95eef8e8d26b7b17910dcb78e1c95f4a) | 53.49 | 44.03 | 73.67 | 49.78 | 46.48 |
+| [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) [📄](https://gist.github.com/mlabonne/8329284d86035e6019edb11eb0933628) | 51.34 | 41.22 | 69.86 | 51.65 | 42.64 |
+| [meta-llama/Meta-Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) [📄](https://gist.github.com/mlabonne/616b6245137a9cfc4ea80e4c6e55d847) | 45.42 | 31.1 | 69.95 | 43.91 | 36.7 |
+## 🌳 Model family tree
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/ekwRGgnjzEOyprT8sEBFt.png)