--- language: - en license: eupl-1.2 tags: - safetensors - 4-bit - transformers - 8-bit - gguf - mlx base_model: - google/gemma-4-E2B-it base_model_relation: quantized pipeline_tag: any-to-any datasets: - lthn/LEM-research --- # Lemer A [Gemma 4 E2B](https://huggingface.co/google/gemma-4-E2B-it) finetune by [lthn.ai](https://lthn.ai) — EUPL-1.2 ## Benchmarks ### Lemer vs Stock Gemma 4 E2B (bf16) Columns: **(Think, Temperature)** — `G4` = Stock Gemma 4 E2B, `Lemer` = LEK-activated | | G4(1,0) | G4(1,1) | G4(0,0) | G4(0,1) | Lemer(1,0) | Lemer(1,1) | Lemer(0,0) | Lemer(0,1) | | :---- | :----: | :----: | :----: | :----: | :----: | :----: | :----: | :----: | | Biology | 40.0% | TBC | TBC | TBC | **60.0%** | [**65.0%**](https://huggingface.co/lthn/lemer/blob/bf16/README.md#mmlu-pro-bf16-best-per-category "65.0") | TBC | TBC | | Math | 10.0% | 30.0% | 15.0% | 10.0% | **55.0%** | [**80.0%**](https://huggingface.co/lthn/lemer/blob/bf16/README.md#mmlu-pro-bf16-best-per-category "80.0") | **25.0%** | **25.0%** | | Business | TBC | TBC | TBC | TBC | TBC | [**50.0%**](https://huggingface.co/lthn/lemer/blob/bf16/README.md#mmlu-pro-bf16-best-per-category "50.0") | TBC | TBC | | Chemistry | TBC | TBC | TBC | TBC | TBC | [**35.0%**](https://huggingface.co/lthn/lemer/blob/bf16/README.md#mmlu-pro-bf16-best-per-category "35.0") | TBC | TBC | | Computer Science | TBC | TBC | TBC | TBC | TBC | [**65.0%**](https://huggingface.co/lthn/lemer/blob/bf16/README.md#mmlu-pro-bf16-best-per-category "65.0") | TBC | TBC | | Economics | TBC | TBC | TBC | TBC | TBC | [**45.0%**](https://huggingface.co/lthn/lemer/blob/bf16/README.md#mmlu-pro-bf16-best-per-category "45.0") | TBC | TBC | | Engineering | TBC | TBC | TBC | TBC | TBC | [**55.0%**](https://huggingface.co/lthn/lemer/blob/bf16/README.md#mmlu-pro-bf16-best-per-category "55.0") | TBC | TBC | | Health | TBC | TBC | TBC | TBC | TBC | [**35.0%**](https://huggingface.co/lthn/lemer/blob/bf16/README.md#mmlu-pro-bf16-best-per-category "35.0") | TBC | TBC | | History | TBC | TBC | TBC | TBC | TBC | [**15.0%**](https://huggingface.co/lthn/lemer/blob/bf16/README.md#mmlu-pro-bf16-best-per-category "15.0") | TBC | TBC | | Law | TBC | TBC | TBC | TBC | TBC | [**25.0%**](https://huggingface.co/lthn/lemer/blob/bf16/README.md#mmlu-pro-bf16-best-per-category "25.0") | TBC | TBC | | Other | TBC | TBC | TBC | TBC | TBC | [**55.0%**](https://huggingface.co/lthn/lemer/blob/bf16/README.md#mmlu-pro-bf16-best-per-category "55.0") | TBC | TBC | | Philosophy | TBC | TBC | TBC | TBC | TBC | [**30.0%**](https://huggingface.co/lthn/lemer/blob/bf16/README.md#mmlu-pro-bf16-best-per-category "30.0") | TBC | TBC | | Physics | TBC | TBC | TBC | TBC | TBC | [**50.0%**](https://huggingface.co/lthn/lemer/blob/bf16/README.md#mmlu-pro-bf16-best-per-category "50.0") | TBC | TBC | | Psychology | TBC | TBC | TBC | TBC | TBC | [**45.0%**](https://huggingface.co/lthn/lemer/blob/bf16/README.md#mmlu-pro-bf16-best-per-category "45.0") | TBC | TBC | | **Average** | TBC | TBC | TBC | TBC | TBC | [**46.4%**](https://huggingface.co/lthn/lemer/blob/bf16/README.md#mmlu-pro-bf16-best-per-category "46.4") | TBC | TBC | [TIGER-Lab/MMLU-Pro](https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro) test split, 20 samples per category, 0-shot. Evaluated using [rapid-mlx](https://github.com/LetheanNetwork/Rapid-MLX) + [OpenAI SDK](https://github.com/openai/openai-python) + Google [parse_response()](https://huggingface.co/google/gemma-4-E2B-it). Source: [eval.py](https://github.com/LetheanNetwork/LEM/blob/main/eval.py) ### Lemer Quantisation Benchmarks (MMLU-Pro, all categories) | | [bf16](https://huggingface.co/lthn/lemer/tree/bf16) | [8bit](https://huggingface.co/lthn/lemer/tree/8bit) | [6bit](https://huggingface.co/lthn/lemer/tree/6bit) | [5bit](https://huggingface.co/lthn/lemer/tree/5bit) | [4bit](https://huggingface.co/lthn/lemer/tree/4bit) | [mxfp8](https://huggingface.co/lthn/lemer/tree/mxfp8) | [mxfp4](https://huggingface.co/lthn/lemer/tree/mxfp4) | [nvfp4](https://huggingface.co/lthn/lemer/tree/nvfp4) | | :---- | :----: | :----: | :----: | :----: | :----: | :----: | :----: | :----: | | Biology | 65.0% | TBC | TBC | TBC | TBC | TBC | TBC | TBC | | Math | 60.0% | TBC | TBC | TBC | TBC | TBC | TBC | TBC | | Business | 45.0% | TBC | TBC | TBC | TBC | TBC | TBC | TBC | | Chemistry | 20.0% | TBC | TBC | TBC | TBC | TBC | TBC | TBC | | Computer Science | 50.0% | TBC | TBC | TBC | TBC | TBC | TBC | TBC | | Economics | 25.0% | TBC | TBC | TBC | TBC | TBC | TBC | TBC | | Engineering | 50.0% | TBC | TBC | TBC | TBC | TBC | TBC | TBC | | Health | 25.0% | TBC | TBC | TBC | TBC | TBC | TBC | TBC | | History | 10.0% | TBC | TBC | TBC | TBC | TBC | TBC | TBC | | Law | 25.0% | TBC | TBC | TBC | TBC | TBC | TBC | TBC | | Other | 45.0% | TBC | TBC | TBC | TBC | TBC | TBC | TBC | | Philosophy | 20.0% | TBC | TBC | TBC | TBC | TBC | TBC | TBC | | Physics | 50.0% | TBC | TBC | TBC | TBC | TBC | TBC | TBC | | Psychology | 25.0% | TBC | TBC | TBC | TBC | TBC | TBC | TBC | | **Average** | **36.8%** | TBC | TBC | TBC | TBC | TBC | TBC | TBC | ## Use ### MLX (recommended for Apple Silicon) ```bash pip install mlx-lm ``` ```python from mlx_lm import load, generate model, tokenizer = load("lthn/lemer", revision="4bit") response = generate(model, tokenizer, prompt="Hello", max_tokens=200) ``` ### Rapid-MLX (OpenAI-compatible server) ```bash pip install rapid-mlx rapid-mlx serve lthn/lemer --port 8100 ``` ```python from openai import OpenAI client = OpenAI(base_url="http://localhost:8100/v1", api_key="not-needed") response = client.chat.completions.create( model="default", messages=[{"role": "user", "content": "Hello"}], ) print(response.choices[0].message.content) ``` ### HF Transformers ```python from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained("lthn/lemer", revision="bf16-hf") tokenizer = AutoTokenizer.from_pretrained("lthn/lemer", revision="bf16-hf") ``` ## Branches ### MLX | Branch | Size | |--------|------| | `bf16` | 8.7G | | `8bit` | 4.6G | | `6bit` | 3.6G | | `5bit` | 3.0G | | `4bit` | 2.5G | | `mxfp8` | 4.5G | | `mxfp4` | 2.3G | | `nvfp4` | 2.5G | ### GGUF | Branch | Size | |--------|------| | `bf16-gguf` | 8.7G | | `8bit-gguf` | 4.6G | | `6bit-gguf` | 3.6G | | `5bit-gguf` | 3.0G | | `4bit-gguf` | 2.5G | | `3bit-gguf` | 2.0G | ### HF Transformers | Branch | Size | |--------|------| | `bf16-hf` | 8.7G | ## Base [google/gemma-4-E2B-it](https://huggingface.co/google/gemma-4-E2B-it) ## More - [lthn.ai](https://lthn.ai) - [Lethean Network](https://github.com/LetheanNetwork) ## Licence Training data and adapter: [EUPL-1.2](https://joinup.ec.europa.eu/collection/eupl/eupl-text-eupl-12) Base model: [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)