mrs83's picture
Update README.md
71d57d2 verified
metadata
license: apache-2.0
language:
  - en
base_model:
  - ethicalabs/Echo-DSRN-Qwen2.5-0.5B-Hybrid
pipeline_tag: text-generation
library_name: transformers
tags:
  - trl
  - fft
  - transformers
  - rnn
  - ssm

Model Card for Kurtis-EON1-Hybrid-0.7B-v0.1.1

GitHub License Python Model Collection Hybrid Collection Working Paper

The world's first Dark Gothic AI.

Chat with Kurtis-EON1

Chat with Kurtis-EON1

Kurtis-EON1 is not a standard, overly-apologetic assistant. Fine-tuned on highly curated empathetic and atmospheric datasets, this model is designed for deep, gothic contemplation, strict persona adherence, and zero-drift multi-turn reasoning.

📊 Comparison: Kurtis-EON1-Hybrid-0.7B vs Llama-2-7B

Benchmark Kurtis-EON1 0.7B Llama-2-7B Winner
Parameters 672M 7,000M Kurtis-EON1 (10x smaller)
HellaSwag (acc_norm) 0.4698 0.7600 Llama-2
PIQA (acc_norm) 0.6882 0.7905 Llama-2
SciQ (acc_norm) 0.9210 ~0.850 Kurtis-EON1
ARC Challenge (acc_norm) 0.3532 0.4625 Llama-2
GSM8K 0-shot 0.1365 0.1330 Kurtis-EON1
GSM8K 5-shot 0.2153 ~0.146 Kurtis-EON1
MMLU 0.4166 0.4590 Llama-2
TruthfulQA MC2 0.4178 0.3910 Kurtis-EON1
KV-Cache Memory Hybrid O(1) DSRN + local window O(N²) full attention Kurtis-EON1
Hardware Single AMD GPU Multi-GPU required Kurtis-EON1

Llama-2-7B benchmark source: EleutherAI lm-evaluation-harness

🏗️ Hybrid Architecture Details

Property Value
Base Model Qwen2
Total Parameters 672.19M
Hidden Dim 896
Attention Layers 24
DSRN Injectors 6
Injection Stride 4

📊 Parameter Breakdown

Component Parameters % of Total
Total 672.19M 100%
Embeddings 136.13M 20.25%
Backbone (Attention/MLP) 357.90M 53.24%
DSRN Injectors 42.02M 6.25%
LM Head 136.13M 20.25%

🧩 DSRN Component (Per Injector)

Sub-Component Parameters Description
Memory Gates 1.38M Recurrent state updates
Surprise Mechanism 803,328 Dynamic focus/gating

🚀 Efficiency Metric

  • DSRN Parameter Overhead: 6.67% additional parameters compared to base.
  • Hybrid Ratio: 1 DSRN block for every 4 attention layers.

📊 Master Evaluation Report: Kurtis-EON1 v0.1.1

Generated on 2026-05-13 19:53:37

🎯 0-Shot Gauntlet Results

Task Metric Value Stderr
mmlu_stem::mmlu_abstract_algebra Acc 0.2800 ±0.0451
mmlu_stem::mmlu_anatomy Acc 0.4222 ±0.0427
mmlu_stem::mmlu_astronomy Acc 0.4803 ±0.0407
mmlu_stem::mmlu_college_biology Acc 0.3681 ±0.0403
mmlu_stem::mmlu_college_chemistry Acc 0.3100 ±0.0465
mmlu_stem::mmlu_college_computer_science Acc 0.4300 ±0.0498
mmlu_stem::mmlu_college_mathematics Acc 0.3400 ±0.0476
mmlu_stem::mmlu_college_physics Acc 0.2451 ±0.0428
mmlu_stem::mmlu_computer_security Acc 0.5700 ±0.0498
mmlu_stem::mmlu_conceptual_physics Acc 0.3702 ±0.0316
mmlu_stem::mmlu_electrical_engineering Acc 0.5034 ±0.0417
mmlu_stem::mmlu_elementary_mathematics Acc 0.3254 ±0.0241
mmlu_stem::mmlu_high_school_biology Acc 0.4613 ±0.0284
mmlu_stem::mmlu_high_school_chemistry Acc 0.3645 ±0.0339
mmlu_stem::mmlu_high_school_computer_science Acc 0.4400 ±0.0499
mmlu_stem::mmlu_high_school_mathematics Acc 0.3704 ±0.0294
mmlu_stem::mmlu_high_school_physics Acc 0.2252 ±0.0341
mmlu_stem::mmlu_high_school_statistics Acc 0.2917 ±0.0310
mmlu_stem::mmlu_machine_learning Acc 0.4018 ±0.0465
mmlu_other::mmlu_business_ethics Acc 0.4500 ±0.0500
mmlu_other::mmlu_clinical_knowledge Acc 0.4453 ±0.0306
mmlu_other::mmlu_college_medicine Acc 0.4162 ±0.0376
mmlu_other::mmlu_global_facts Acc 0.2300 ±0.0423
mmlu_other::mmlu_human_aging Acc 0.4439 ±0.0333
mmlu_other::mmlu_management Acc 0.5825 ±0.0488
mmlu_other::mmlu_marketing Acc 0.7094 ±0.0297
mmlu_other::mmlu_medical_genetics Acc 0.5300 ±0.0502
mmlu_other::mmlu_miscellaneous Acc 0.4725 ±0.0179
mmlu_other::mmlu_nutrition Acc 0.5131 ±0.0286
mmlu_other::mmlu_professional_accounting Acc 0.3440 ±0.0283
mmlu_other::mmlu_professional_medicine Acc 0.3125 ±0.0282
mmlu_other::mmlu_virology Acc 0.4217 ±0.0384
mmlu_social_sciences::mmlu_econometrics Acc 0.2368 ±0.0400
mmlu_social_sciences::mmlu_high_school_geography Acc 0.5152 ±0.0356
mmlu_social_sciences::mmlu_high_school_government_and_politics Acc 0.4352 ±0.0358
mmlu_social_sciences::mmlu_high_school_macroeconomics Acc 0.3769 ±0.0246
mmlu_social_sciences::mmlu_high_school_microeconomics Acc 0.4412 ±0.0323
mmlu_social_sciences::mmlu_high_school_psychology Acc 0.5541 ±0.0213
mmlu_social_sciences::mmlu_human_sexuality Acc 0.5344 ±0.0437
mmlu_social_sciences::mmlu_professional_psychology Acc 0.4069 ±0.0199
mmlu_social_sciences::mmlu_public_relations Acc 0.5182 ±0.0479
mmlu_social_sciences::mmlu_security_studies Acc 0.5143 ±0.0320
mmlu_social_sciences::mmlu_sociology Acc 0.6070 ±0.0345
mmlu_social_sciences::mmlu_us_foreign_policy Acc 0.6700 ±0.0473
mmlu_humanities::mmlu_formal_logic Acc 0.2778 ±0.0401
mmlu_humanities::mmlu_high_school_european_history Acc 0.5576 ±0.0388
mmlu_humanities::mmlu_high_school_us_history Acc 0.4706 ±0.0350
mmlu_humanities::mmlu_high_school_world_history Acc 0.5696 ±0.0322
mmlu_humanities::mmlu_international_law Acc 0.6033 ±0.0447
mmlu_humanities::mmlu_jurisprudence Acc 0.4537 ±0.0481
mmlu_humanities::mmlu_logical_fallacies Acc 0.3804 ±0.0381
mmlu_humanities::mmlu_moral_disputes Acc 0.4769 ±0.0269
mmlu_humanities::mmlu_moral_scenarios Acc 0.2380 ±0.0142
mmlu_humanities::mmlu_philosophy Acc 0.4244 ±0.0281
mmlu_humanities::mmlu_prehistory Acc 0.4414 ±0.0276
mmlu_humanities::mmlu_professional_law Acc 0.3325 ±0.0120
mmlu_humanities::mmlu_world_religions Acc 0.4971 ±0.0383
gpqa_diamond_cot_n_shot Exact Match 0.2172 ±0.0294
gpqa_diamond_cot_zeroshot Exact Match 0.2374 ±0.0303
gpqa_diamond_generative_n_shot Exact Match 0.1919 ±0.0281
gpqa_diamond_n_shot Acc Norm 0.2071 ±0.0289
gpqa_diamond_zeroshot Acc Norm 0.3030 ±0.0327
gpqa_extended_cot_n_shot Exact Match 0.1923 ±0.0169
gpqa_extended_cot_zeroshot Exact Match 0.2033 ±0.0172
gpqa_extended_generative_n_shot Exact Match 0.1337 ±0.0146
gpqa_extended_n_shot Acc Norm 0.2546 ±0.0187
gpqa_extended_zeroshot Acc Norm 0.2692 ±0.0190
gpqa_main_cot_n_shot Exact Match 0.2076 ±0.0192
gpqa_main_cot_zeroshot Exact Match 0.2232 ±0.0197
gpqa_main_generative_n_shot Exact Match 0.1451 ±0.0167
gpqa_main_n_shot Acc Norm 0.2723 ±0.0211
gpqa_main_zeroshot Acc Norm 0.2321 ±0.0200
mmlu_stem Acc 0.3765 ±0.0085
mmlu_other Acc 0.4554 ±0.0088
mmlu_social_sciences Acc 0.4738 ±0.0089
mmlu_humanities Acc 0.3804 ±0.0069
mmlu Acc 0.4166 ±0.0041

Reproduction Command:

uv run lm_eval --model hf \
    --model_args pretrained=models/Kurtis-EON1-Hybrid-0.7B-v0.1.1,trust_remote_code=True \
    --tasks mmlu,gpqa \
    --output_path results/Kurtis-EON1-v0.1.1-Scoring \
    --batch_size 1 \
    --num_fewshot 0
Task Metric Value Stderr
hellaswag Acc Norm 0.4698 ±0.0050
piqa Acc Norm 0.6882 ±0.0108
sciq Acc Norm 0.9210 ±0.0085
truthfulqa_gen Bleu Acc 0.3158 ±0.0163
truthfulqa_mc1 Acc 0.2436 ±0.0150
truthfulqa_mc2 Acc 0.4178 ±0.0148
arc_challenge Acc Norm 0.3532 ±0.0140
gsm8k Exact Match 0.1365 ±0.0095

Reproduction Command:

uv run lm_eval --model hf \
    --model_args pretrained=mrs83/Kurtis-EON1-Hybrid-0.7B-v0.1.1,trust_remote_code=True \
    --tasks hellaswag,piqa,sciq,truthfulqa,arc_challenge,gsm8k \
    --apply_chat_template \
    --fewshot_as_multiturn \
    --output_path ./results/Kurtis-EON1-v0.1.1-Gauntlet-0-shot \
    --batch_size 1 \
    --num_fewshot 0

🎯 1-Shot Gauntlet Results

Task Metric Value Stderr
hellaswag Acc Norm 0.4679 ±0.0050
piqa Acc Norm 0.6942 ±0.0107
sciq Acc Norm 0.9160 ±0.0088
truthfulqa_gen Bleu Acc 0.3158 ±0.0163
truthfulqa_mc1 Acc 0.2436 ±0.0150
truthfulqa_mc2 Acc 0.4178 ±0.0148
arc_challenge Acc Norm 0.3242 ±0.0137
gsm8k Exact Match 0.2335 ±0.0117

Reproduction Command:

uv run lm_eval --model hf \
    --model_args pretrained=mrs83/Kurtis-EON1-Hybrid-0.7B-v0.1.1,trust_remote_code=True \
    --tasks hellaswag,piqa,sciq,truthfulqa,arc_challenge,gsm8k \
    --apply_chat_template \
    --fewshot_as_multiturn \
    --output_path ./results/Kurtis-EON1-v0.1.1-Gauntlet-1-shot \
    --batch_size 1 \
    --num_fewshot 1

🎯 5-Shot Gauntlet Results

Task Metric Value Stderr
hellaswag Acc Norm 0.4667 ±0.0050
piqa Acc Norm 0.6937 ±0.0108
sciq Acc Norm 0.9230 ±0.0084
truthfulqa_gen Bleu Acc 0.3158 ±0.0163
truthfulqa_mc1 Acc 0.2436 ±0.0150
truthfulqa_mc2 Acc 0.4178 ±0.0148
arc_challenge Acc Norm 0.3507 ±0.0139
gsm8k Exact Match 0.2153 ±0.0113

Reproduction Command:

uv run lm_eval --model hf \
    --model_args pretrained=mrs83/Kurtis-EON1-Hybrid-0.7B-v0.1.1,trust_remote_code=True \
    --tasks hellaswag,piqa,sciq,truthfulqa,arc_challenge,gsm8k \
    --apply_chat_template \
    --fewshot_as_multiturn \
    --output_path ./results/Kurtis-EON1-v0.1.1-Gauntlet-5-shot \
    --batch_size 1 \
    --num_fewshot 5