| --- |
| license: llama3 |
| tags: |
| - llama3 |
| - ternary |
| - trillim |
| - cpu-inference |
| base_model: HF1BitLLM/Llama3-8B-1.58-100B-tokens |
| extra_gated_prompt: >- |
| This model is derived from Meta Llama 3. By accessing this model you agree to |
| the Meta Llama 3 Community License Agreement. |
| --- |
| |
| # Llama3-TRNQ |
|
|
| Built with Meta Llama 3. |
|
|
| Ternary-quantized version of [HF1BitLLM/Llama3-8B-1.58-100B-tokens](https://huggingface.co/HF1BitLLM/Llama3-8B-1.58-100B-tokens), packaged for the [Trillim DarkNet](https://huggingface.co/Trillim) inference engine. The source model is a BitNet 1.58b quantization of [Meta-Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B), trained on the FineWeb-edu dataset. |
|
|
| This model runs entirely on CPU — no GPU required. |
|
|
| ## Model Details |
|
|
| | | | |
| |---|---| |
| | **Architecture** | Llama (LlamaForCausalLM) | |
| | **Parameters** | ~8B | |
| | **Hidden size** | 4096 | |
| | **Layers** | 32 | |
| | **Attention heads** | 32 (8 KV heads) | |
| | **Context length** | 8192 | |
| | **Quantization** | Ternary ({-1, 0, 1}) | |
| | **Source model** | [HF1BitLLM/Llama3-8B-1.58-100B-tokens](https://huggingface.co/HF1BitLLM/Llama3-8B-1.58-100B-tokens) | |
| | **License** | [Meta Llama 3 Community License](https://llama.meta.com/llama3/license/) | |
|
|
| ## Usage |
|
|
| ```bash |
| pip install trillim |
| trillim pull Trillim/Llama3-TRNQ |
| trillim serve Trillim/Llama3-TRNQ |
| ``` |
|
|
| This starts an OpenAI-compatible API server at `http://127.0.0.1:8000`. |
|
|
| For interactive CLI chat: |
|
|
| ```bash |
| trillim chat Trillim/Llama3-TRNQ |
| ``` |
|
|
| ## What's in this repo |
|
|
| | File | Description | |
| |---|---| |
| | `qmodel.tensors` | Ternary-quantized weights in Trillim format | |
| | `rope.cache` | Precomputed RoPE embeddings | |
| | `config.json` | Model configuration | |
| | `tokenizer.json` | Tokenizer | |
| | `tokenizer_config.json` | Tokenizer configuration | |
| | `trillim_config.json` | Trillim metadata | |
|
|
| ## License |
|
|
| This model is derived from Meta Llama 3 and is subject to the [Meta Llama 3 Community License Agreement](https://llama.meta.com/llama3/license/). |
|
|
| ### Meta Llama 3 Community License Agreement |
|
|
| Meta Llama 3 is licensed under the Meta Llama 3 Community License, Copyright (c) Meta Platforms, Inc. All Rights Reserved. |
|
|
| **Acceptable Use Policy:** Your use of Llama 3 must comply with applicable laws and regulations and you must comply with Meta's [Acceptable Use Policy](https://llama.meta.com/llama3/use-policy/). |
|
|
| **License terms:** The full license text is available at [https://llama.meta.com/llama3/license/](https://llama.meta.com/llama3/license/). Key terms include: |
|
|
| - You may use, reproduce, distribute, and create derivative works of Llama 3. |
| - If you distribute or make Llama 3 available to a third party, you must provide a copy of this Agreement and include "Built with Meta Llama 3" in related materials. |
| - You must not use Llama 3 to improve any other large language model (excluding Llama 3 derivatives). |
| - If the monthly active users of your products or services exceed 700 million, you must request a license from Meta. |
|
|
| For the complete license terms, please refer to the [official license](https://llama.meta.com/llama3/license/). |
|
|
