README.md

---
language:
- en
library_name: mlx
pipeline_tag: text-generation
license: apache-2.0
base_model: Qwen/Qwen3.6-35B-A3B
base_model_relation: finetune
datasets:
- aisamdasu/algocean-fable5-traces
tags:
- mlx
- qwen3.6
- qwen3.5-moe
- lora
- sft
- tool-use
- agentic
- langgraph
- fable5
---

# Algocean-Qwen3.6-35B-A3B-mlx

Merged MLX release of `Qwen/Qwen3.6-35B-A3B` fine-tuned with a LoRA SFT run on the `aisamdasu/algocean-fable5-traces` trace mix.

This MLX artifact is text-generation focused. The upstream model card describes the base as a language model with a vision encoder, but the current `mlx-lm` Qwen3.5-MoE conversion stores the language-model weights and omits `model.visual`.

## Use

```bash
pip install mlx-lm
```

```python
from mlx_lm import load, generate

model, tokenizer = load("aisamdasu/Algocean-Qwen3.6-35B-A3B-mlx")

messages = [{"role": "user", "content": "Create a concise LangGraph plan for a repo refactor."}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
response = generate(model, tokenizer, prompt=prompt, max_tokens=512, verbose=True)
```

## Artifact

| item | value |
|---|---:|
| Format | MLX safetensors |
| Shards | 14 |
| Weight size | 69,321,221,376 bytes |
| Parameters in MLX index | 34,660,608,768 |
| Visual weights | omitted |
| Integrity file | `SHA256SUMS` |

## Training

| key | value |
|---|---|
| Base model | `Qwen/Qwen3.6-35B-A3B` |
| Dataset | `aisamdasu/algocean-fable5-traces` |
| Train / eval rows | 30,265 / 512 |
| Selected source rows | Crownelius 26,938 + kelexine 3,839 |
| Dedup skipped | 1,721 |
| Fine-tune method | LoRA SFT |
| LoRA target modules | `q_proj`, `v_proj` |
| LoRA rank / alpha / dropout | 16 / 32 / 0.0 |
| Max sequence length | 16,384 |
| Optimizer / schedule | AdamW fused / cosine |
| Learning rate / warmup | 1e-5 / 0.03 |
| Batch / grad accumulation | 1 / 1 |
| Max steps | 4,000 |
| Eval / save interval | 200 / 200 |
| GPU | Modal B200 x1 |
| Runtime | 7,934.71 sec |

## Checkpoint Selection

The published weights use checkpoint `3600`, not the final step `4000`.

| step | eval loss | eval PPL | note |
|---:|---:|---:|---|
| 200 | 1.300712 | 3.671911 | first eval |
| 3200 | 1.067631 | 2.908480 | near plateau |
| 3400 | 1.067996 | 2.909543 | regression |
| 3600 | 1.066911 | 2.906388 | selected best |
| 3800 | 1.067495 | 2.908084 | worse than best |
| 4000 | 1.067005 | 2.906661 | final, slightly worse |

Best checkpoint rationale:

- `3600` has the lowest validation loss: `1.066911`.
- Final step `4000` is very close but still higher: `+0.000094` eval loss.
- The curve flattened after `3200`, so selecting `3600` keeps the best validation point without chasing extra train loss.

![Eval loss vs step](./eval_loss_vs_step.png)

![Train vs eval loss](./train_vs_eval_loss.png)

![Learning rate vs step](./learning_rate_vs_step.png)

## Metrics

| metric | value |
|---|---:|
| Best eval loss | 1.066911 |
| Best eval perplexity | 2.906388 |
| Final eval loss | 1.067005 |
| Final eval perplexity | 2.906661 |
| Trainer average train loss | 0.494604 |
| Last logged train loss | 0.996137 |
| Train samples/sec | 0.504 |
| Train steps/sec | 0.504 |
| Eval samples/sec at final eval | 2.088 |

## Comparison Notes

This release is the best checkpoint from the completed Algocean LoRA run. It is not claiming a public benchmark win over the base model or the upstream OptiQ model; those require the same held-out evaluation harness on all models. The selection here is based on the run's validation loss, where checkpoint `3600` is the lowest-loss point among all saved checkpoints.

## Intended Use

This model is intended for local MLX inference experiments around agentic coding, tool-use traces, and LangGraph-style planning. It is not a general safety-tuned assistant release.