Ateron/Gemma-4-Dark-Thoughts-V2-31B

🚬 Dark-Thoughts V2

"Shady corners of mind"

🎥 Overview

So, despite creativity the first version had issues with tool calling and brains in long context role-play. I fixed this in V2 and changed recipe, putting more experiments on focus and filters.

💿 Mergekit Configuration

Below is the exact mergekit_config.yml recipe used to synthesize this model:

Phase 1: Spark

Ignition of creativity with MeroMero V2 and Dark-Scarlett V2

dare_ties Recipe 1

merge_method: dare_ties
base_model: F:\AI\Merge\Gemma-4-it
tokenizer_source: base
dtype: bfloat16
parameters:
  lambda: 1.0
models:
  - model: F:\AI\Merge\Gemma-4-MeroMero-V2
    parameters:
      density: 0.50
      weight:
        - filter: "model.layers.4."
          value: 0.60
        - filter: "model.layers.9."
          value: 0.60
        - filter: "model.layers.14."
          value: 0.60
        - filter: "model.layers.19."
          value: 0.60
        - filter: "model.layers.24."
          value: 0.60
        - filter: "model.layers.29."
          value: 0.60
        - filter: "model.layers.34."
          value: 0.60
        - filter: "model.layers.39."
          value: 0.50
        - filter: "model.layers.44."
          value: 0.50
        - filter: "model.layers.49."
          value: 0.40
        - filter: "model.layers.54."
          value: 0.40
        - filter: "model.layers.59."
          value: 0.40
        - value: 0.50

  - model: F:\AI\Merge\Gemma-4-Dark-Scarlett-V2
    parameters:
      density: 0.50
      weight:
        - filter: "model.layers.4."
          value: 0.40
        - filter: "model.layers.9."
          value: 0.40
        - filter: "model.layers.14."
          value: 0.40
        - filter: "model.layers.19."
          value: 0.40
        - filter: "model.layers.24."
          value: 0.40
        - filter: "model.layers.29."
          value: 0.40
        - filter: "model.layers.34."
          value: 0.40
        - filter: "model.layers.39."
          value: 0.50
        - filter: "model.layers.44."
          value: 0.50
        - filter: "model.layers.49."
          value: 0.60
        - filter: "model.layers.54."
          value: 0.60
        - filter: "model.layers.59."
          value: 0.60
        - value: 0.50

Phase 2: Form

Polisher of thoughts from Scotoma v2

dare_ties Recipe 2

merge_method: dare_ties
base_model: F:\AI\Merge\Gemma-4-it
tokenizer_source: base
dtype: bfloat16
parameters:
  lambda: 1.0
models:
  - model: F:\AI\Merge\Gemma-4-Scotoma-V2
    parameters:
      density: 0.60
      weight:
      - {filter: mlp, value: 0.2}
      - {filter: self_attn, value: 0.2}
      - {filter: q_proj, value: 0.5}
      - {filter: k_proj, value: 0.5}
      - {filter: v_proj, value: 0.6}
      - {filter: o_proj, value: 0.8}
      - value: [0.60, 0.60, 0.70, 0.80, 0.80]
  - model: F:\AI\Merge\Dark-Mero-V3
    parameters:
      density: 0.40
      weight:
      - {filter: mlp, value: 0.8}
      - {filter: self_attn, value: 0.8}
      - {filter: q_proj, value: 0.5}
      - {filter: k_proj, value: 0.5}
      - {filter: v_proj, value: 0.4}
      - {filter: o_proj, value: 0.2}
      - value: [0.40, 0.40, 0.30, 0.20, 0.20]

🤝 Special Thanks

• Google DeepMind: For providing the base model.
• The ReadyArt: Team of good people and creators of Scotoma and Dark Scarlett.
• To Zerofata: For keeping merging possible and for wonderful MeroMero 2.
• To the community: For giving feedbacks and positivity.
• To Nimbz: This cat for being around in tough times .