BananaMind 2 Pro is training! The current checkpoint (ONLY 20% DONE) GETS #6 On the entire Open SLM Leaderboard. We are going to release the first public preview on August 2-4 (estimated from speed)
We're excited to release BananaMindBench Leaderboard, our leaderboard for BananaMind Base Bench 1.1. It measures model performance on a variety of different tasks: Language Completion Common sense too World Knowledge Context Tracking Quantitative Logical Reasoning Code Completion Each has a different score and 1 overall score. Submit your own model: BananaMind/BananaMindBench-Leaderboard Check it out: BananaMind/BananaMindBench-Leaderboard
We're excited to release BananaMind Base Bench 1.1 A new benchmark for base language models with 350 text-completion examples across seven categories. Models are scored using continuation likelihood and receive an Overall Elo score. Initial results: BananaMind-2-Medium: 1034 BananaMind-2-Mini: 974 Supra-50M-Base: 973 Supra-1.5-50M-Base-exp: 948 BananaMind-2-Nano: 910 The official script downloads the gated dataset directly from Hugging Face. The dataset is for benchmarking only and may not be used for model training. BananaMind/BananaMind-Base-Bench-1.1
We're excited to announce BananaMind 2V, our small vision model series! These models are NOT released yet. We will release them in mid-august! BananaMind 2V will include: BananaMind 2V 256M, the flagship based on BananaMind 2 Pro (BananaMind 2 Pro is not released yet). BananaMind 2V 100M, our mid model, based on BananaMind 2 Medium. BananaMind 2V 50M, our smallest vision model, based on BananaMind 2 Mini. These are currently unreleased and will release in mid-august. Our training will start after BananaMind 2 Pro has finished training.
We're excited to release BananaMind 2 Medium and BananaMind 2 Medium Chat!
They’re both 50M parameter models trained on 50B tokens from FineWeb-Edu, DCLM, Cosmopedia v2, FineMath-4+ and NPSet-2 Python-Edu.
The base model reached 61.86% on PIQA, 43.81% on ARC Easy and 32.43% on HellaSwag. The Chat version was fine-tuned on Smol-SmolTalk and scored 38% overall on our internal instruction benchmark, with 56% on multi-turn, 60% on context recall and 80% on code.
Lets get 80 followers on my account and 30 on BananaMind. When we hit that we are going to release BananaMind 2 Medium tomorrow. Me: @Banaxi-Tech BananaMind:
BananaMind 2 Nano is the smallest member of the BananaMind 2.0 family — a 10M-parameter language model that shows how much you can squeeze out of a tiny footprint. It uses the family's digit-isolated tokenizer, so it keeps solid arithmetic despite its size, and it's small enough to run just about anywhere.
Trained on 30B tokens in about a day on a single RTX 5070 Ti (16GB), 4096-token context.
We're excited to release BananaMind 2 MoE, a new addition to the BananaMind 2 series! BananaMind 2 MoE is a sparse mixture-of-experts model with 25M total parameters but only 2M active per token, Like the rest of the series, it uses our custom digit-aware BPE tokenizer that keeps every digit isolated, fixing the core arithmetic weakness of our earlier models. It's trained on 30B tokens from FineWeb-Edu, DCLM, Cosmopedia-v2 and FineMath-4+, and outperforms Pythia-31M on average despite activating just 2M parameters. Check it out at BananaMind/BananaMind-2-MoE
BananaMind 2 Nano is Coming Next. Already training. Apache 2.0.
We are announcing 3 more models in our BananaMind 2 Family of models! BananaMind 2 Nano, a small 10M parameter model, fits on your Pentium 4 BananaMind 2 Medium, our medium model, 50M parameters BananaMind 2 MoE, 25M parameters, 2M active per tokens as fast as a 2M.
Because of this our release dates have changed a bit our currently estimates are: BananaMind 2 MoE July 16-18 BananaMind 2 Nano July 18-20 BananaMind 2 Medium July 24-28 BananaMind 2 Pro August 10-16 Keep in mind these dates are estimates and we don't have a speed number currently, we will post for details going forward!
We're excited to release BananaMind 2 Mini the first model in our BananaMind 2 series!
BananaMind 2 Mini features a custom digit-aware BPE tokenizer that keeps every digit isolated, fixing the core arithmetic weakness of our previous models. It's trained on 30B tokens from FineWeb-Edu, DCLM, Cosmopedia-v2 and FineMath-4+, and already outperforms Pythia-31M despite having fewer parameters. Check it out at BananaMind/BananaMind-2-Mini
BananaMind 2 Mini is now officially training! Live Dashboard: https://huggingface.co/BananaMind/BananaMind-2-Mini-Status (updates every few minutes) We are training it on 30B Tokens and a mix of DCLM, FineWeb Edu, FineMath, and Cosmopedia! We estimate the training to take around 2 days. Follow us to know when it releases!
Today we are releasing BananaMind-KV1-8M-2Bit-Experimental, a KV-cache-aware trained model that stores its generation KV cache in 2-bit precision instead of the usual 16-bit precision.
Result: 5.33x smaller KV cache vs FP16, with 0.0916 mean KLD against a 16-bit KV cache reference on WikiText-2.
The important part: this is not just post-training KV cache quantization. Instead we take the BitNet approach.
KV1 is trained with a 2-bit-aware K/V path. Instead of training a normal model and quantizing the cache afterwards, the model learns during training to operate under the low-bit KV constraint, closer in spirit to the BitNet idea of training for the low-bit regime.
During generation, each K/V vector is quantized into 4 affine levels and packed into uint8 tensors, with four 2-bit values stored per byte.
WikiText-2 eval vs 16-bit KV cache reference:
Mean KLD: 0.0916 nats/token Mean KLD: 0.1322 bits/token Average KV cache shrink vs FP16: 5.33x Evaluated positions: 372,675
If this actually gets used in models like Qwen or Gemma, then it may be possible to run 128K or even 256K Context on a Normal Machine! Try it here: BananaMind/BananaMind-KV1-8M-2Bit-Experimental
A new model is coming! Its going to take a long time on my 5070 Ti so expect a release in ~1 month. We think this model is going to be SOTA For its size. Our Mini Version will be 25M Parameters and Pro with 140M. The Pro version has a 3072 Context Window (Extensible to up to 6K with RoPE) And the Mini version has a context window of 4096 (Up to 8K with RoPE) Meanwhile we are currently working on a Instruct Version of our BananaMind 1.5 Base.
📱 TinyPhoneLM - LLMs on a Phone I built TinyPhoneLM because I wanted to see how far tiny local LMs can go on a real Android phone. Not just a server app. Not just an API wrapper. Not “AI on your phone” that secretly sends everything somewhere else.
TinyPhoneLM allows you to run small language models directly on android. It uses llama.cpp via JNI. We have alot of options for default models + custom GGUF Import Supported. I am running Qwen3.5 4B Locally on my Redmi Note 12 Pro 5G at 4 tokens per second, that may seem slow but that it even runs on my phone is insane. I can also run Qwen3.5 0.8B at 10TPS! Look at this Chart From Artificial Analysis. Qwen3.5 4B is Better than GPT 4.1 and GPT 5 Mini at minimal reasoning! And even the smallest 800M Parameter Qwen3.5 0.8B still beats GPT 3.5 Turbo!
The bad news: To get it on the play store we need 12 Testers
Please only submit your Google Play email if you have a Android phone If you want to test TinyPhoneLM, enter your Google Play email here:
OLD POST CONTENTS EDITED: We currently have a new AI Model and we are currently training it. We are training it on 27B tokens and are currently 8% done. Follow us to be notified when it releases 🚀 Some Info: Parameters 75M GPU: RTX Pro 6000 We expect to be able to release it in the coming dayshttps://huggingface.co/BananaMind/BananaMind-1.5-Base
MiniBananaMind-v2-9M is a tiny causal language model trained fully from scratch on FineWeb-Edu.
It has only ~8.9M parameters, but was trained for ~3.54B tokens after retokenization using a custom 8k byte-level BPE tokenizer. We trained it on a RTX 5070 Ti in just 4h 34m! Go check it out at BananaMind/MiniBananaMind-v2-9M
