Instructions to use micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5")
model = AutoModelForCausalLM.from_pretrained("micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5

SGLang

How to use micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5 with Docker Model Runner:
```
docker model run hf.co/micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5

CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5 is a fine-tuned coding assistant built on top of CodeMate-Qwen-1.5B-8K.

The model was further trained on converted Claude Fable 5 coding traces to improve:

Code generation
Code explanation
Debugging
Multi-turn coding conversations
Software engineering reasoning

Model Details

Base Model: micymike/codemate-qwen-1.5B-8k
Architecture: Qwen2 Causal LM
Training Method: LoRA fine-tuning merged into full weights
Precision: BF16
Configured Context Length: 32,768 tokens

Context Configuration

This model has been configured for a 32K context window using YaRN RoPE scaling.

from transformers import AutoConfig

config = AutoConfig.from_pretrained(
    "micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5"
)

print(config.max_position_embeddings)
print(config.rope_scaling)

Current configuration:

{
    "rope_type": "yarn",
    "factor": 4.0,
    "original_max_position_embeddings": 8192,
    "rope_theta": 1000000.0
}

Note: Long-context performance beyond the original context length should be evaluated carefully for specific workloads.

Dataset

The model was trained on converted Claude Fable 5 coding traces formatted into OpenAI-style conversations.

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5"

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

messages = [
    {
        "role": "system",
        "content": "You are CodeMate, an expert programming assistant."
    },
    {
        "role": "user",
        "content": "Write a Python function to compute edit distance."
    }
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.7,
    top_p=0.9,
    do_sample=True
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Limitations

Experimental research model.
Long-context capabilities require further evaluation.
May generate incorrect or insecure code.

Acknowledgements

Built upon:

Qwen2
Transformers
PEFT
Hugging Face
llama.cpp
Claude Fable traces

Disclaimer

This project is an independent research effort and is not affiliated with or endorsed by Anthropic, Claude, Alibaba, or Qwen.

Author

Built by micymike 🇰🇪

Downloads last month: 96

Safetensors

Model size

2B params

Tensor type

BF16

Model tree for micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5

Base model

micymike/codemate-qwen-1.5B

Finetuned

micymike/codemate-qwen-1.5B-8k

Finetuned

(1)

this model

Quantizations

2 models