| --- |
| license: bigscience-bloom-rail-1.0 |
| language: |
| - ak |
| - ar |
| - as |
| - bm |
| - bn |
| - ca |
| - code |
| - en |
| - es |
| - eu |
| - fon |
| - fr |
| - gu |
| - hi |
| - id |
| - ig |
| - ki |
| - kn |
| - lg |
| - ln |
| - ml |
| - mr |
| - ne |
| - nso |
| - ny |
| - or |
| - pa |
| - pt |
| - rn |
| - rw |
| - sn |
| - st |
| - sw |
| - ta |
| - te |
| - tn |
| - ts |
| - tum |
| - tw |
| - ur |
| - vi |
| - wo |
| - xh |
| - yo |
| - zh |
| - zhs |
| - zht |
| - zu |
| pipeline_tag: text-generation |
| --- |
| |
| <h1 style='text-align: center '>BLOOM LM - 8bit</h1> |
| <h2 style='text-align: center '><em>BigScience Large Open-science Open-access Multilingual Language Model - 8bit</em> </h2> |
| <h3 style='text-align: center '>Model Card</h3> |
| <img src="https://s3.amazonaws.com/moonup/production/uploads/1657124309515-5f17f0a0925b9863e28ad517.png" alt="BigScience Logo" width="800" style="margin-left:'auto' margin-right:'auto' display:'block'"/> |
|
|
| Version 1.0 / 26.May.2022 |
|
|
| Related paper: https://arxiv.org/abs/2208.07339 |
|
|
| ## TL;DR |
|
|
| This repository contains 8bit weights of `bloom-1b7` model. You can load this model using `transformers==4.28.0` and `bitsandbytes>0.37.2` out of the box ! |
|
|
| ```python |
| # pip install accelerate bitsandbytes |
| from transformers import AutoModelForCausalLM |
| |
| model = AutoModelForCausalLM.from_pretrained("ybelkada/bloom-1b7-8bit") |
| ``` |
|
|
| ## How to push 8bit weights? |
|
|
| First, make sure you are using `transformers` & `bitsandbytes` versions stated above. Then load your 8bit model as usual using `load_in_8bit=True`! |
|
|
| ```python |
| # pip install accelerate bitsandbytes |
| from transformers import AutoModelForCausalLM |
| |
| model = AutoModelForCausalLM.from_pretrained("bigscience/bloom-1b7", device_map="auto", load_in_8bit=True) |
| ``` |
|
|
| Then just call `push_to_hub` method or `save_pretrained` method if you want to save your 8bit model locally |
|
|
| ```python |
| model.push_to_hub("{your_username}/bloom-1b7-8bit") |
| ``` |
|
|
| That's it! |
|
|
| ## What is inside the model's `state_dict`? |
| |
| Inside the state dict of the model (`pytorch_model.bin` file) you have |
|
|
| - the quantized `int8` weights |
| - the quantization statistics in `float16` |
|
|
