LiteRT-LM
marissaw commited on
Commit
5d1dc65
·
verified ·
1 Parent(s): 753814e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -0
README.md CHANGED
@@ -6,6 +6,15 @@ license: apache-2.0
6
 
7
  Main Model Card: [google/gemma-4-E2B-it](https://huggingface.co/google/gemma-4-E2B-it)
8
 
 
 
 
 
 
 
 
 
 
9
 
10
  ## Try Gemma 4 E2B
11
 
 
6
 
7
  Main Model Card: [google/gemma-4-E2B-it](https://huggingface.co/google/gemma-4-E2B-it)
8
 
9
+ This model card provides the Gemma 4 E2B model in a way that is ready for deployment on Android, iOS, Desktop IoT and Web.
10
+
11
+ Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models. This particular Gemma 4 model is small so it is ideal for on-device use cases. By running this model on device, users can have private access to Generative AI technology without even requiring an internet connection.
12
+
13
+ These models are provided in the .litertlm format for use with the LiteRT-LM framework. LiteRT-LM is a specialized orchestration layer built directly on top of LiteRT, Google’s high-performance multi-platform runtime trusted by millions of Android and edge developers. LiteRT provides the foundational hardware acceleration via XNNPack for CPU and ML Drift for GPU. LiteRT-LM adds the specialized GenAI libraries and APIs, such as KV-cache management, prompt templating, and function calling. This integrated stack is the same technology powering the Google AI Edge Gallery showcase app.
14
+
15
+ The model file size is 2.58 GB, which consists of a text decoder with 0.79 GB of weights and 1.1GB of embedding parameters. LiteRT-LM framework always keeps main weights in memory, but it only memory maps the embedding parameters as only a fraction of these are required for each inference. The vision and audio models are loaded as needed to further reduce memory consumption.
16
+
17
+
18
 
19
  ## Try Gemma 4 E2B
20