---
base_model: microsoft/bitnet-b1.58-2B-4T
license: mit
tags:
- bitnet
- apple-silicon
- metal
- swift
---

# Bitnet-b1.58-2B-metal-weight

## Introduction

This repository contains .bin files extracted from [microsoft/bitnet-b1.58-2B-4T](https://huggingface.co/microsoft/bitnet-b1.58-2B-4T). The weights are stock and ready to use in my project [jossnet-bitnet](https://github.com/jossweb/jossnet-bitnet). This project is an implementation of the BitNet model for Apple Silicon, using Metal.

## Structure


    bitnet-b1.58-2B-metal-weight/
    ├── layers/
    │    ├── layer_X/
    │         ├── RMS_attn.bin
    │         ├── RMS_input.bin
    │         ├── RMS_mlp_sub.bin
    │         ├── RMS_post_attn.bin
    │         ├── Scale_down.bin
    │         ├── Scale_gate.bin
    │         ├── Scale_k.bin
    │         ├── Scale_o.bin
    │         ├── Scale_q.bin
    │         ├── Scale_up.bin
    │         ├── Scale_v.bin
    │         ├── W_down.bin
    │         ├── W_gate.bin
    │         ├── W_k.bin
    │         ├── W_o.bin
    │         ├── W_q.bin
    │         ├── W_up.bin
    │         └── W_v.bin
    ├── embeddings.bin
    ├── weights_lm_head.bin
    ├── weights_RMS_final.bin
    ├── weights_RMS_mlp.bin
    ├── weights_W_down.bin
    ├── weights_W_gate.bin
    ├── weights_W_k.bin
    ├── weights_W_o.bin
    ├── weights_W_q.bin
    ├── weights_W_up.bin
    ├── weights_W_v.bin
    └── LICENCE
        

## Licence & references
* Original Model: [microsoft/bitnet-b1.58-2B-4T](https://huggingface.co/microsoft/bitnet-b1.58-2B-4T)
* License: MIT License (Copyright (c) Microsoft Corporation)