File size: 3,098 Bytes
646db13 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 | ---
license: llama3
tags:
- llama3
- ternary
- trillim
- cpu-inference
base_model: HF1BitLLM/Llama3-8B-1.58-100B-tokens
extra_gated_prompt: >-
This model is derived from Meta Llama 3. By accessing this model you agree to
the Meta Llama 3 Community License Agreement.
---
# Llama3-TRNQ
Built with Meta Llama 3.
Ternary-quantized version of [HF1BitLLM/Llama3-8B-1.58-100B-tokens](https://huggingface.co/HF1BitLLM/Llama3-8B-1.58-100B-tokens), packaged for the [Trillim DarkNet](https://huggingface.co/Trillim) inference engine. The source model is a BitNet 1.58b quantization of [Meta-Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B), trained on the FineWeb-edu dataset.
This model runs entirely on CPU — no GPU required.
## Model Details
| | |
|---|---|
| **Architecture** | Llama (LlamaForCausalLM) |
| **Parameters** | ~8B |
| **Hidden size** | 4096 |
| **Layers** | 32 |
| **Attention heads** | 32 (8 KV heads) |
| **Context length** | 8192 |
| **Quantization** | Ternary ({-1, 0, 1}) |
| **Source model** | [HF1BitLLM/Llama3-8B-1.58-100B-tokens](https://huggingface.co/HF1BitLLM/Llama3-8B-1.58-100B-tokens) |
| **License** | [Meta Llama 3 Community License](https://llama.meta.com/llama3/license/) |
## Usage
```bash
pip install trillim
trillim pull Trillim/Llama3-TRNQ
trillim serve Trillim/Llama3-TRNQ
```
This starts an OpenAI-compatible API server at `http://127.0.0.1:8000`.
For interactive CLI chat:
```bash
trillim chat Trillim/Llama3-TRNQ
```
## What's in this repo
| File | Description |
|---|---|
| `qmodel.tensors` | Ternary-quantized weights in Trillim format |
| `rope.cache` | Precomputed RoPE embeddings |
| `config.json` | Model configuration |
| `tokenizer.json` | Tokenizer |
| `tokenizer_config.json` | Tokenizer configuration |
| `trillim_config.json` | Trillim metadata |
## License
This model is derived from Meta Llama 3 and is subject to the [Meta Llama 3 Community License Agreement](https://llama.meta.com/llama3/license/).
### Meta Llama 3 Community License Agreement
Meta Llama 3 is licensed under the Meta Llama 3 Community License, Copyright (c) Meta Platforms, Inc. All Rights Reserved.
**Acceptable Use Policy:** Your use of Llama 3 must comply with applicable laws and regulations and you must comply with Meta's [Acceptable Use Policy](https://llama.meta.com/llama3/use-policy/).
**License terms:** The full license text is available at [https://llama.meta.com/llama3/license/](https://llama.meta.com/llama3/license/). Key terms include:
- You may use, reproduce, distribute, and create derivative works of Llama 3.
- If you distribute or make Llama 3 available to a third party, you must provide a copy of this Agreement and include "Built with Meta Llama 3" in related materials.
- You must not use Llama 3 to improve any other large language model (excluding Llama 3 derivatives).
- If the monthly active users of your products or services exceed 700 million, you must request a license from Meta.
For the complete license terms, please refer to the [official license](https://llama.meta.com/llama3/license/).
|