Instructions to use aloobun/Cypher-Laser-Mixtral-2x1.8B-v0.1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use aloobun/Cypher-Laser-Mixtral-2x1.8B-v0.1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="aloobun/Cypher-Laser-Mixtral-2x1.8B-v0.1")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForMultimodalLM

tokenizer = AutoTokenizer.from_pretrained("aloobun/Cypher-Laser-Mixtral-2x1.8B-v0.1")
model = AutoModelForMultimodalLM.from_pretrained("aloobun/Cypher-Laser-Mixtral-2x1.8B-v0.1")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use aloobun/Cypher-Laser-Mixtral-2x1.8B-v0.1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "aloobun/Cypher-Laser-Mixtral-2x1.8B-v0.1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aloobun/Cypher-Laser-Mixtral-2x1.8B-v0.1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/aloobun/Cypher-Laser-Mixtral-2x1.8B-v0.1

SGLang

How to use aloobun/Cypher-Laser-Mixtral-2x1.8B-v0.1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "aloobun/Cypher-Laser-Mixtral-2x1.8B-v0.1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aloobun/Cypher-Laser-Mixtral-2x1.8B-v0.1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "aloobun/Cypher-Laser-Mixtral-2x1.8B-v0.1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aloobun/Cypher-Laser-Mixtral-2x1.8B-v0.1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use aloobun/Cypher-Laser-Mixtral-2x1.8B-v0.1 with Docker Model Runner:
```
docker model run hf.co/aloobun/Cypher-Laser-Mixtral-2x1.8B-v0.1
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

This work is based on implementation of the LASER technique, by @fernandofernandes & @ehartford.

I 'lasered' two models first and then merged them.

Cypher-Laser-Mixtral-2x1.8B-v0.1

Cypher-Laser-Mixtral-2x1.8B-v0.1 is a Mixure of Experts (MoE) made with the following models using LazyMergekit:

🧩 Configuration

base_model: aloobun/Cypher-Mini-Laser-1.8B
gate_mode: hidden
dtype: bfloat16
experts:
  - source_model: aloobun/Cypher-Mini-Laser-1.8B
    positive_prompts:
      - "Write a Python script that sorts a list of integers using the bubble sort algorithm."
      - "Write a JavaScript function that redirects a web page to another page after 5 seconds."
      - "Describe the steps to troubleshoot a fluid dynamics issue with a water fountain."
      - "Write a short story about a knight's quest to find a lost treasure, and then summarize it in one paragraph."
      - "Summarize the following article with details and clarity."
      - "Tell me about your favorite book and why you like it."
      
  - source_model: aloobun/Cypher-CoT-Laser-1.8B
    positive_prompts:
      - "Liam saw an animal running on the farm. Q: Is it true that The animal could be a horse."
      - "Based on the following paragraph can we conclude that the sentence below is true?"
      - "According to the article, how do dolphins communicate with each other?"
      - "Solve this math problem Solve 7644 = 4648*d - 4557*d for d."
      - "If we have 3 marbles, and two roll under the counter, and one is found, how many marbles are there?"
      - "What is the result of 25 divided by 5?"
      - "Is it morally justifiable to lie to protect someone's feelings?"
      - "Determine if the sentence is true based on the text below. Choose from options."
      - "What might a person do if they forget their umbrella on a rainy day?"
      - "Which of the following is an example of renewable energy: a) Coal, b) Solar, c) Oil, d) Natural gas?"
      - "What is the capital of Canada? a) Toronto, b) Ottawa, c) Montreal, d) Vancouver."
      - "Which of these animals is a mammal? a) Snake, b) Dolphin, c) Turtle, d) Frog."
      - "Given a story, answer the question about the story."
      - "Given a prompt and four completions, select the completion that is the most plausible in continuing or answering the prompt."

Format:

<|system|></s><|prompt|></s><|answer|>

Downloads last month: 64

Safetensors

Model size

3B params

Tensor type

BF16

Model tree for aloobun/Cypher-Laser-Mixtral-2x1.8B-v0.1

aloobun/Cypher-CoT-Laser-1.8B

aloobun/Cypher-Mini-Laser-1.8B

Merge model

this model

Quantizations

1 model

aloobun
/

Cypher-Laser-Mixtral-2x1.8B-v0.1

Cypher-Laser-Mixtral-2x1.8B-v0.1

🧩 Configuration

Format:

Model tree for aloobun/Cypher-Laser-Mixtral-2x1.8B-v0.1

Datasets used to train aloobun/Cypher-Laser-Mixtral-2x1.8B-v0.1