RichardErkhov/ilsp_-_Meltemi-7B-v1-gguf

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Meltemi-7B-v1 - GGUF

• Model creator: https://huggingface.co/ilsp/
• Original model: https://huggingface.co/ilsp/Meltemi-7B-v1/

Name	Quant method	Size
Meltemi-7B-v1.Q2_K.gguf	Q2_K	2.66GB
Meltemi-7B-v1.IQ3_XS.gguf	IQ3_XS	2.95GB
Meltemi-7B-v1.IQ3_S.gguf	IQ3_S	3.11GB
Meltemi-7B-v1.Q3_K_S.gguf	Q3_K_S	3.09GB
Meltemi-7B-v1.IQ3_M.gguf	IQ3_M	3.2GB
Meltemi-7B-v1.Q3_K.gguf	Q3_K	3.42GB
Meltemi-7B-v1.Q3_K_M.gguf	Q3_K_M	3.42GB
Meltemi-7B-v1.Q3_K_L.gguf	Q3_K_L	3.7GB
Meltemi-7B-v1.IQ4_XS.gguf	IQ4_XS	3.83GB
Meltemi-7B-v1.Q4_0.gguf	Q4_0	3.98GB
Meltemi-7B-v1.IQ4_NL.gguf	IQ4_NL	4.03GB
Meltemi-7B-v1.Q4_K_S.gguf	Q4_K_S	4.01GB
Meltemi-7B-v1.Q4_K.gguf	Q4_K	4.22GB
Meltemi-7B-v1.Q4_K_M.gguf	Q4_K_M	4.22GB
Meltemi-7B-v1.Q4_1.gguf	Q4_1	4.4GB
Meltemi-7B-v1.Q5_0.gguf	Q5_0	4.83GB
Meltemi-7B-v1.Q5_K_S.gguf	Q5_K_S	4.83GB
Meltemi-7B-v1.Q5_K.gguf	Q5_K	4.95GB
Meltemi-7B-v1.Q5_K_M.gguf	Q5_K_M	4.95GB
Meltemi-7B-v1.Q5_1.gguf	Q5_1	5.25GB
Meltemi-7B-v1.Q6_K.gguf	Q6_K	5.72GB
Meltemi-7B-v1.Q8_0.gguf	Q8_0	7.41GB

Original model description:

license: apache-2.0 language: - el - en library_name: transformers pipeline_tag: text-generation

Meltemi: A large foundation Language Model for the Greek language

We introduce Meltemi, the first Greek Large Language Model (LLM) trained by the Institute for Language and Speech Processing at Athena Research & Innovation Center. Meltemi is built on top of Mistral-7B, extending its capabilities for Greek through continual pretraining on a large corpus of high-quality and locally relevant Greek texts. We present Meltemi-7B-v1, as well as an instruction fine-tuned version Meltemi-7B-Instruct-v1.

Model Information

• Vocabulary extension of the Mistral-7B tokenizer with Greek tokens
• 8192 context length
• We extend the pretraining of Mistral-7B with added proficiency for the Greek language, by utilizing a large corpus consisting of approximately 40 billion tokens.
  • This corpus includes 28.5 billion monolingual Greek tokens, constructed from publicly available resources. Additionaly, to mitigate catastrophic forgetting and ensure that the model has bilingual capabilities, we use additional sub-corpora with monolingual English texts (10.5 billion tokens) and Greek-English parallel data (600 million tokens).
  • This corpus has been processed, filtered, and deduplicated to ensure data quality (a detailed description of our data processing pipeline will be published in our upcoming paper) and is outlined below:

Sub-corpus	# Tokens	Percentage
Greek	28,555,902,360	72.0%
English	10,478,414,033	26.4%
Parallel	633,816,023	1.6%
Total	39,668,132,416	100%

Usage

Please make sure that the BOS token is always included in the tokenized prompts. This might not be the default setting in all evaluation or fine-tuning frameworks.

Evaluation

The evaluation suite we created includes 6 test sets. The suite is integrated with lm-eval-harness.

Our evaluation suite includes:

• Four machine-translated versions (ARC Greek, Truthful QA Greek, HellaSwag Greek, MMLU Greek) of established English benchmarks for language understanding and reasoning (ARC Challenge, Truthful QA, Hellaswag, MMLU).
• An existing benchmark for question answering in Greek (Belebele)
• A novel benchmark created by the ILSP team for medical question answering based on the medical exams of DOATAP (Medical MCQA).

Our evaluation for Meltemi-7B is performed in a few-shot setting, consistent with the settings in the Open LLM leaderboard. We can see that our training enhances performance across all Greek test sets by a +14.9% average improvement. The results for the Greek test sets are shown in the following table:

	Medical MCQA EL (15-shot)	Belebele EL (5-shot)	HellaSwag EL (10-shot)	ARC-Challenge EL (25-shot)	TruthfulQA MC2 EL (0-shot)	MMLU EL (5-shot)	Average
Mistral 7B	29.8%	45.0%	36.5%	27.1%	45.8%	35%	36.5%
Meltemi 7B	41.0%	63.6%	61.6%	43.2%	52.1%	47%	51.4%

Ethical Considerations

This model has not been aligned with human preferences, and therefore might generate misleading, harmful, and toxic content.

Acknowledgements

The ILSP team utilized Amazon's cloud computing services, which were made available via GRNET under the OCRE Cloud framework, providing Amazon Web Services for the Greek Academic and Research Community.

Citation

@misc{voukoutis2024meltemiopenlargelanguage,
      title={Meltemi: The first open Large Language Model for Greek}, 
      author={Leon Voukoutis and Dimitris Roussis and Georgios Paraskevopoulos and Sokratis Sofianopoulos and Prokopis Prokopidis and Vassilis Papavasileiou and Athanasios Katsamanis and Stelios Piperidis and Vassilis Katsouros},
      year={2024},
      eprint={2407.20743},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2407.20743}, 
}