upload checkpoint

README.md

@@ -0,0 +1,158 @@
+---
+language:
+- en
+library_name: transformers
+tags:
+- stepfun
+- MOE
+- pruning
+- compression
+license: other
+name: cerebras/Step-3.5-Flash-REAP-121B-A11B
+description: >
+  This model was obtained by uniformly pruning 40% of experts in Step-3.5-Flash using the REAP method.
+readme: >
+  https://huggingface.co/cerebras/Step-3.5-Flash-REAP-121B-A11B/main/README.md
+pipeline_tag: text-generation
+base_model:
+- stepfun-ai/Step-3.5-Flash
+---
+
+<p align="center">
+  <em>𓌳 <strong>REAP</strong>𓌳  the Experts: Why Pruning Prevails for One-Shot MoE Compression</em><br>
+  <img src="https://i.imgur.com/rmzG3gg.png" alt="REAP" width="75%">
+</p>
+
+# Step-3.5-Flash-REAP-121B-A11B
+
+## ✨ Highlights
+
+Introducing **Step-3.5-Flash-REAP-121B-A11B**, a **memory-efficient compressed variant** of Step-3.5-Flash that maintains near-identical performance while being **40% lighter**.
+
+This model was created using **REAP (Router-weighted Expert Activation Pruning)**, a novel expert pruning method that selectively removes redundant experts while preserving the router's independent control over remaining experts. Key features include:
+
+- **Near-Lossless Performance**: Maintains almost identical accuracy on code generation, agentic coding, and function calling tasks compared to the full 196B model
+- **40% Memory Reduction**: Compressed from 196B to 121B parameters, significantly lowering deployment costs and memory requirements
+- **Preserved Capabilities**: Retains all core functionalities including code generation, math & reasoning and tool calling.
+- **Drop-in Compatibility**: Works with vanilla vLLM - no source modifications or custom patches required
+- **Optimized for Real-World Use**: Particularly effective for resource-constrained environments, local deployments, and academic research
+---
+## 📋 Model Overview
+
+**Step-3.5-Flash-REAP-121B-A11B** has the following specifications:
+
+- **Base Model**: Step-3.5-Flash
+- **Compression Method**: REAP (Router-weighted Expert Activation Pruning)
+- **Compression Ratio**: 40% expert pruning
+- **Type**: Sparse Mixture-of-Experts (SMoE) Causal Language Model
+- **Number of Parameters**: 121B total, 11B activated per token
+- **Number of Layers**: 45
+- **Number of Attention Heads**: 64
+- **Number of Experts**: 173 (uniformly pruned from 288)
+- **Number of Activated Experts**: 8 per token
+- **Context Length**: 262,144 tokens
+- **License**: Apache 2.0
+
+---
+
+## 📊 Evaluations
+
+<table>
+  <thead>
+    <tr>
+      <th align="left">Benchmark</th>
+      <th align="center">Step-3.5-Flash</th>
+      <th align="center"><a href="https://huggingface.co/cerebras/Step-3.5-Flash-REAP-149B-A11B">Step-3.5-Flash-REAP-149B-A11B</a></th>
+      <th align="center"><a href="https://huggingface.co/cerebras/Step-3.5-Flash-REAP-121B-A11B">Step-3.5-Flash-REAP-121B-A11B</a></th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td><strong>Compression</strong></td>
+      <td align="center">—</td>
+      <td align="center">25%</td>
+      <td align="center">40%</td>
+    </tr>
+    <tr>
+      <td colspan="5" align="center"><strong>Coding</strong></td>
+    </tr>
+    <tr>
+      <td><strong>HumanEval</strong></td>
+      <td align="center">98.2</td>
+      <td align="center">97.0</td>
+      <td align="center">95.7</td>
+    </tr>
+    <tr>
+      <td><strong>HumanEval+</strong></td>
+      <td align="center">93.9</td>
+      <td align="center">90.9</td>
+      <td align="center">91.5</td>
+    </tr>
+  </tbody>
+</table>
+
+---
+
+## 🚀 Deployment
+
+You can deploy the model directly using the **latest vLLM** (that supports Step-3.5-Flash), no source modifications or custom patches required.
+
+```bash
+vllm serve cerebras/Step-3.5-Flash-REAP-121B-A11B \
+    --tensor-parallel-size 8 \
+    --tool-call-parser step3p5 \
+    --reasoning-parser step3p5 \
+    --trust-remote-code \
+    --enable_expert_parallel \
+    --disable-cascade-attn \
+    --enable-auto-tool-choice
+```
+
+If you encounter insufficient memory when running this model, you might need to set a lower value for `--max-num-seqs` flag (e.g. set to 64). For more information, refer to the [official vLLM deployment guide](https://huggingface.co/stepfun-ai/Step-3.5-Flash#61-vllm).
+
+## 🧩 Model Creation
+
+This checkpoint was created by applying the **REAP (Router-weighted Expert Activation Pruning)** method uniformly across all Mixture-of-Experts (MoE) blocks of **Step-3.5-Flash**, with a **40% pruning rate**.
+
+### How REAP Works
+
+REAP selects experts to prune based on a novel **saliency criterion** that considers both:
+- **Router gate values**: How frequently and strongly the router activates each expert
+- **Expert activation norms**: The magnitude of each expert's output contributions
+
+This dual consideration ensures that experts contributing minimally to the layer's output are pruned, while preserving those that play critical roles in the model's computations.
+
+### Key Advantages
+
+- **One-Shot Compression**: No fine-tuning required after pruning - the model is immediately ready for deployment
+- **Preserved Router Control**: Unlike expert merging methods, REAP maintains the router's independent, input-dependent control over remaining experts, avoiding "functional subspace collapse"
+- **Generative Task Superiority**: REAP significantly outperforms expert merging approaches on generative benchmarks (code generation, creative writing, mathematical reasoning) while maintaining competitive performance on discriminative tasks
+
+📚 For more details, refer to the following resources:
+
+- [🧾 arXiv Preprint](https://arxiv.org/abs/2510.13999)
+- [🧾 REAP Blog](https://www.cerebras.ai/blog/reap)
+- [💻 REAP Codebase (GitHub)](https://github.com/CerebrasResearch/reap)
+
+---
+
+## ⚖️ License
+
+This model is derived from
+**[`stepfun-ai/Step-3.5-Flash`](https://huggingface.co/stepfun-ai/Step-3.5-Flash)**
+and distributed under the **Apache 2.0 license**.
+
+---
+
+## 🧾 Citation
+
+If you use this checkpoint, please cite the REAP paper:
+
+```bibtex
+@article{lasby-reap,
+  title={REAP the Experts: Why Pruning Prevails for One-Shot MoE compression},
+  author={Lasby, Mike and Lazarevich, Ivan and Sinnadurai, Nish and Lie, Sean and Ioannou, Yani and Thangarasa, Vithursan},
+  journal={arXiv preprint arXiv:2510.13999},
+  year={2025}
+}
+```

chat_template.jinja

@@ -0,0 +1,80 @@
+{% macro render_content(content) %}{% if content is none %}{{- '' }}{% elif content is string %}{{- content }}{% elif content is mapping %}{{- content['value'] if 'value' in content else content['text'] }}{% elif content is iterable %}{% for item in content %}{% if item.type == 'text' %}{{- item['value'] if 'value' in item else item['text'] }}{% elif item.type == 'image' %}<im_patch>{% endif %}{% endfor %}{% endif %}{% endmacro %}
+{{bos_token}}{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- render_content(messages[0].content) + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou have access to the following functions in JSONSchema format:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson(ensure_ascii=False) }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...>\n...\n</function> block must be nested within <tool_call>\n...\n</tool_call> XML tags\n- Required parameters MUST be specified\n</IMPORTANT><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + render_content(messages[0].content) + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" and render_content(message.content) is string and not(render_content(message.content).startswith('<tool_response>') and render_content(message.content).endswith('</tool_response>')) %}
+        {%- set ns.multi_step_tool = false %}
+        {%- set ns.last_query_index = index %}
+    {%- endif %}
+{%- endfor %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content) %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {%- set role_name = 'observation' if (message.role == "system" and not loop.first and message.name == 'observation') else message.role %}
+        {{- '<|im_start|>' + role_name + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = render_content(message.reasoning_content) %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- else %}
+                {%- set reasoning_content = '' %}
+            {%- endif %}
+        {%- endif %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- if tool_call.arguments is defined %}
+                    {%- set arguments = tool_call.arguments %}
+                    {%- for args_name, args_value in arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson(ensure_ascii=False) | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>tool_response\n' }}
+        {%- endif %}
+        {{- '<tool_response>' }}
+        {{- content }}
+        {{- '</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n<think>\n' }}
+{%- endif %}

config.json

@@ -0,0 +1,312 @@
+{
+  "architectures": [
+    "Step3p5ForCausalLM"
+  ],
+  "model_type": "step3p5",
+  "auto_map": {
+    "AutoConfig": "configuration_step3p5.Step3p5Config",
+    "AutoModelForCausalLM": "modeling_step3p5.Step3p5ForCausalLM"
+  },
+  "rope_scaling": {
+    "rope_type": "llama3",
+    "factor": 2.0,
+    "original_max_position_embeddings": 131072,
+    "low_freq_factor": 1.0,
+    "high_freq_factor": 32.0
+  },
+  "yarn_only_types": ["full_attention"],
+  "hidden_size": 4096,
+  "intermediate_size": 11264,
+  "num_hidden_layers": 45,
+  "max_seq_len": 262144,
+  "vocab_size": 128896,
+  "torch_dtype": "bfloat16",
+  "use_qk_norm": true,
+  "moe_layers_enum": "3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44",
+  "num_attention_heads": 64,
+  "num_attention_groups": 8,
+  "head_dim": 128,
+  "use_moe": true,
+  "moe_num_experts": 173,
+  "moe_top_k": 8,
+  "moe_intermediate_size": 1280,
+  "share_expert_dim": 1280,
+  "moe_layer_offset": 0,
+  "moe_every_n_layer": 1,
+  "norm_expert_weight": true,
+  "moe_router_activation": "sigmoid",
+  "moe_router_scaling_factor": 3.0,
+  "att_impl_type": "GQA",
+  "rope_theta": [
+    5000000.0,
+    10000.0,
+    10000.0,
+    10000.0,
+    5000000.0,
+    10000.0,
+    10000.0,
+    10000.0,
+    5000000.0,
+    10000.0,
+    10000.0,
+    10000.0,
+    5000000.0,
+    10000.0,
+    10000.0,
+    10000.0,
+    5000000.0,
+    10000.0,
+    10000.0,
+    10000.0,
+    5000000.0,
+    10000.0,
+    10000.0,
+    10000.0,
+    5000000.0,
+    10000.0,
+    10000.0,
+    10000.0,
+    5000000.0,
+    10000.0,
+    10000.0,
+    10000.0,
+    5000000.0,
+    10000.0,
+    10000.0,
+    10000.0,
+    5000000.0,
+    10000.0,
+    10000.0,
+    10000.0,
+    5000000.0,
+    10000.0,
+    10000.0,
+    10000.0,
+    5000000.0,
+    10000.0,
+    10000.0,
+    10000.0
+  ],
+  "use_head_wise_attn_gate": true,
+  "sliding_window": 512,
+  "use_moe_router_bias": true,
+  "need_fp32_gate": true,
+  "sink": false,
+  "layer_types": [
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention"
+  ],
+  "use_rope_layers": [],
+  "num_nextn_predict_layers": 3,
+  "partial_rotary_factors": [
+    0.5,
+    1.0,
+    1.0,
+    1.0,
+    0.5,
+    1.0,
+    1.0,
+    1.0,
+    0.5,
+    1.0,
+    1.0,
+    1.0,
+    0.5,
+    1.0,
+    1.0,
+    1.0,
+    0.5,
+    1.0,
+    1.0,
+    1.0,
+    0.5,
+    1.0,
+    1.0,
+    1.0,
+    0.5,
+    1.0,
+    1.0,
+    1.0,
+    0.5,
+    1.0,
+    1.0,
+    1.0,
+    0.5,
+    1.0,
+    1.0,
+    1.0,
+    0.5,
+    1.0,
+    1.0,
+    1.0,
+    0.5,
+    1.0,
+    1.0,
+    1.0,
+    0.5,
+    1.0,
+    1.0,
+    1.0
+  ],
+  "eos_token_id": [
+    1,
+    2,
+    128007
+  ],
+  "bos_token_id": 0,
+  "attention_other_setting": {
+    "attention_type": "sliding_attention",
+    "num_attention_heads": 96,
+    "num_attention_groups": 8,
+    "head_dim": 128,
+    "true_head_dim": 128
+  },
+  "swiglu_limits": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    7,
+    7,
+    0.0,
+    0.0,
+    0.0
+  ],
+  "swiglu_limits_shared": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    16,
+    0.0,
+    0.0,
+    0.0
+  ],
+  "zero_centered": true,
+  "max_position_embeddings": 262144
+}

configuration_step3p5.py

@@ -0,0 +1,59 @@
+from typing import Any, Optional, Union
+
+from transformers.configuration_utils import PretrainedConfig
+
+
+
+class Step3p5Config(PretrainedConfig):
+    model_type = "step3p5"
+    architectures = ["Step3p5ForCausalLM"]
+
+    def __init__(
+        self,
+        hidden_size: int = 4096,
+        intermediate_size: int = 11264,
+        num_attention_heads: int = 64,
+        num_attention_groups: int = 8,
+        num_hidden_layers: int = 45,
+        max_seq_len: int = 128000,
+        vocab_size: int = 128815,
+        rms_norm_eps: float = 1e-5,
+        moe_intermediate_size: int = 1280,
+        moe_num_experts: int = 288,
+        moe_top_k: int = 8,
+        rope_theta: float = 10000,
+        rope_scaling: Optional[dict[str, Any]] = None,
+        max_position_embeddings: int = 128000,
+        share_expert_dims: int = 1280,
+        head_dim: int = 128,
+        norm_expert_weight: bool = True,
+        layer_types: list[str] = None,
+        sliding_window: Optional[int] = None,
+        moe_layers_enum: tuple[int] = (3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14,
+                                       15, 16, 17, 18, 19, 20, 21, 22, 23, 24,
+                                       25, 26, 27, 28, 29, 30, 31, 32, 33, 34,
+                                       35, 36, 37, 38, 39, 40, 41, 42, 43, 44),
+        **kwargs,
+    ) -> None:
+        self.hidden_size = hidden_size
+        self.intermediate_size = intermediate_size
+        self.num_attention_heads = num_attention_heads
+        self.num_attention_groups = num_attention_groups
+        self.num_hidden_layers = num_hidden_layers
+        self.max_seq_len = max_seq_len
+        self.vocab_size = vocab_size
+        self.rms_norm_eps = rms_norm_eps
+        self.moe_intermediate_size = moe_intermediate_size
+        self.moe_num_experts = moe_num_experts
+        self.moe_top_k = moe_top_k
+        self.rope_theta = rope_theta
+        self.rope_scaling = rope_scaling
+        self.max_position_embeddings = max_position_embeddings
+        self.share_expert_dim = share_expert_dims
+        self.head_dim = head_dim
+        self.norm_expert_weight = norm_expert_weight
+        self.moe_layers_enum = moe_layers_enum
+        self.layer_types = layer_types
+        self.sliding_window = sliding_window
+        super().__init__(**kwargs)
+

model-00001-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5624cc147930c8323253ee4d57648a01cce8aaaeceb96142dc9b35a1c0307e0d
+size 4510089164

model-00002-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:878af09c54e5d19261bc14b930a9096b574aa9647ba7749891a10a218b50fcd2
+size 3812486132

model-00003-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ed5ba6a280e951ba5100c2dd2d638d2f77faf7c8057bafe9cff5a36ba5610fe9
+size 3628073232

model-00004-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f33e4f3dafd65138d3a69e68f330024232f0a1aa7dfc5b9b559ed2a7a7dbe3b4
+size 3879857140

model-00005-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dd5a6bd5e8ff1b6a80e93bb9f72baad578ad1bd7fa1a7f49e139e7b40e509353
+size 3879857140

model-00006-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3aba70cdd5f8012f350419c43b4112f9eec4c7c29f8fef873583370323897ad4
+size 3628073232

model-00007-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aeef836ba88f32cf8b8164e7562ade441c83e7db00905b62af217952f83646d0
+size 3879857140

model-00008-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a8fe428e15b4b1058ae4a2c9a29a238bdbb992c84b2e13f20d9e269ded488e94
+size 3812486132

model-00009-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f318861f022f1222933bce4e5eca4905b15268d9df5a69be721b9dd84e6dc417
+size 3628073232

model-00010-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:76b7feb6b3439905da4d503ae8206844ba862ed4d30507341ea2422c43aa8311
+size 3879857140

model-00011-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:24ad37964b706e6ff09f9bf3e12c08d392ca1204db65a469ca9df881b4892a2a
+size 3879857132

model-00012-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:70a8a02b1bf87b9f2ebb5c993737ee9738bfa7f7636600a909d69ea0a4bfacd7
+size 3628073232

model-00013-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d5fb52f0a9e06f35975d2612f6583aac9bc97a6b8481113d2b79afc73ed8104a
+size 3879857156

model-00014-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4a9e6e81edd0bbc9f48f984c61f28b4af0e1ead088d1e9d6ff2b2cf3650ac68c
+size 3812486148

model-00015-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9458a4a0573bab7b3dd74cdff2a5786751b9a1540fa8aeda90a2a9a079355b64
+size 3628073232

model-00016-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:76c2cfce27b2ae747fa0b1ee02ada6105c2dc163a3a8dc7da7d954c17e1cc377
+size 3879857156

model-00017-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f1e097988027795c58526e29345a158c7ab59613b0e624ba8297989c78dce373
+size 3879857156

model-00018-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc03bf51fef611e82b263274be17a46bcfa4e7b773a046aada486985b2da0211
+size 3628073232

model-00019-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f30ccd2eb205e71750bfce444ecc25094f710bc6818116b41ae4c912bd90f135
+size 3879857156

model-00020-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a28d11a2a43cae5bfcf9a1d9f77f09c02c9765b0e188093ee9970ee6373bb364
+size 3812486148

model-00021-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:96485bdedd6aca4ebc0a87cd43853eef9d73ec7db08fa3e89b97a69a4f60d251
+size 3628073232

model-00022-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e92a5ffa1daa21496498b9d839fdfb11f743363ea0a14ed3c5fde4971434d5b7
+size 3879857156

model-00023-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:36c654d456ad814a9e8b58eb7d378843df90ec29d941e2c6eb6a06c3a5c0a108
+size 3879857156

model-00024-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5d6e1336c73bc505bce0051afc26afb8e221275c314e7d5e88ca2e5f28102cf5
+size 3628073232

model-00025-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:804149b9b8b8a96c09f76694e41523a2b758287d4b13624a2d634d840e86df63
+size 3879857156

model-00026-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7ce46fada0373a3ed448ad89650d26416ea9228e8ce32c4fdf4ce8d18aac7736
+size 3812486148

model-00027-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bea7991e88d048115a15da99dc99980dd08cd2b89211f9ec132e32f8eecb7ad5
+size 3628073232

model-00028-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9e3700a09bda9f2e00dad75f3c04d1ee345017d71f1546079cccc6f1230add93
+size 3879857156

model-00029-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ae194ab8f09191acc0ac45a800a07a7f8847dd6e967499769857cc9d7ae21213
+size 3879857156

model-00030-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eaf3aaa21eecdf43bd9d1f8889d59a42ff37bed749b39033bc348d7f41696cbd
+size 3628073232

model-00031-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8e4d5c2f6978b364bce503aa9572c60911367ddf374091d27400c9900a04d525
+size 3879857156

model-00032-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:409d3bfa66fba803c7bad2abf70afc391460a14d22d81bbc7a2f9f5b2555be98
+size 3812486148

model-00033-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0ee1b861698948be6c96325e99d993fb15fe276732af8c1c1db66576e4dd5be3
+size 3628073232

model-00034-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c38897aac0df2cf728744232f9e3669cf66158d7bc25a80e284aa9982966301f
+size 3879857156

model-00035-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a3f6bdb8b5975e386f789154fd6ade2d19db8aaf86ad919e375c4030a7bb9c6b
+size 3879857156

model-00036-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b41e79f0b46c1a69ae5449f2216c226219359d63284c730b0b19a92f51db1f97
+size 3628073232

model-00037-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:573a7c3eb4c5a9d408eba874042aae12fd81773e87defdcc5b37a3dbeda7bf24
+size 3879857156

model-00038-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9fae56646790270cc39c42148b056fa1cd5794a1e38a1884b6df14dad821f6bc
+size 3812486148

model-00039-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:96997535de9cfb69a2d83e2b189f88dec260a194bca89c0eb279fdc983ed059a
+size 3628073232

model-00040-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2a427495d37e030ee33dfb122e2f0f1a0e6cab7ef06510aa2be1dbc88ad9e8ce
+size 3879857156

model-00041-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:708369d113576a113fefdaaa5cecb8d890acbd337815fcdd6198ab4fb5944e67
+size 3879857156

model-00042-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:061e43bfb0c323d13e5a1a0019d0827ba375f841f027a1ce998a2243cfdd7c10
+size 3628073232

model-00043-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0972fbc505205d6809e00d7881c33e9c7db4d1ca483fb5bd0a7638ea27fee064
+size 3879857156

model-00044-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d68df4561ec5f6c9525674d7ac9caf76de4ac46e0b80e66bcd2e0fb347f3c4e6
+size 3812486148

model-00045-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:84b273b8cf2ca84d92be432644e4782ca50293953c52e05d13c7c0a0a1e4050f
+size 3628073232

model-00046-of-00064.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:86c4d517487a6b6fed23934215b32795aa4f42eda3e6afb22329486cfe9d72b6
+size 3879857156