reirei123/SpikySpace-AttnLM-v3_2-BabyLM2026

SpikySpace-AttnLM v3.2 BabyLM 2026 submission artifact

This repository contains the public artifact for a BabyLM 2026 leaderboard submission.

Model

• Name: SpikySpace-AttnLM-v3_2
• Architecture: SpikySpace-inspired event-gated SSM + partial local attention causal language model
• Parameters: 33,341,568
• Checkpoint used for submission: spiky_v3_2 best checkpoint
• Training scale: data-limited / pilot setting, not a full 100M-word strict-small run

Included files

• eval_ready_bundle/: tokenizer, config, checkpoints, and condition index
• SpikySpace-AttnLM-v3_2_babylm2026_submission.json: collated BabyLM prediction JSON generated by collate_preds.sh

Available full zero-shot evaluation results

Task	Score
BLiMP full	54.52
Supplement full	52.73
Entity Tracking full	42.42
COMPS full	51.03

EWoK full, Reading, AoA, and fine-tuning tasks were not included in this partial submission artifact.

Notes

This is a custom PyTorch research model rather than a standard Hugging Face AutoModelForCausalLM checkpoint. The repository is intended to make the submitted model artifact publicly available and reproducible.

Inspiration and Citation

SpikySpace-AttnLM-v3_2 is inspired by the SpikySpace architecture. The citation below refers to the original SpikySpace paper, not to this model repository.

@misc{tang2026spikyspacespikingstatespace,
  title={SpikySpace: A Spiking State Space Model for Energy-Efficient Time Series Forecasting},
  author={Kaiwen Tang and Jiaqi Zheng and Yuze Jin and Yupeng Qiu and Guangda Sun and Zhanglu Yan and Weng-Fai Wong},
  year={2026},
  eprint={2601.02411},
  archivePrefix={arXiv},
  primaryClass={cs.NE},
  url={https://arxiv.org/abs/2601.02411}
}