BroAlanTaps
/

Stage1-PCC-Lite-16x

Feature Extraction

text-generation

Model card Files Files and versions

Model Card for Model ID

PCC-Lite-16x Model:

Compressor: Continue pre-training based on GPT2-Large

Downloads last month: 6

Safetensors

Model size

0.8B params

Tensor type

F32

·

Model tree for BroAlanTaps/Stage1-PCC-Lite-16x

Base model

openai-community/gpt2-large

Finetuned

(131)

this model

Dataset used to train BroAlanTaps/Stage1-PCC-Lite-16x

Collection including BroAlanTaps/Stage1-PCC-Lite-16x

PCC-Pretrained

Pretraining Context Compressor for Large Language Models with Embedding-Based Memory • 13 items • Updated Mar 2 • 1