README.md

---
base_model:
- Qwen/Qwen2.5-14B
license: apache-2.0
model-index:
- name: Virtuoso-Small
  results:
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: IFEval (0-Shot)
      type: HuggingFaceH4/ifeval
      args:
        num_few_shot: 0
    metrics:
    - type: inst_level_strict_acc and prompt_level_strict_acc
      value: 79.35
      name: strict accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/Virtuoso-Small
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: BBH (3-Shot)
      type: BBH
      args:
        num_few_shot: 3
    metrics:
    - type: acc_norm
      value: 50.4
      name: normalized accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/Virtuoso-Small
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MATH Lvl 5 (4-Shot)
      type: hendrycks/competition_math
      args:
        num_few_shot: 4
    metrics:
    - type: exact_match
      value: 34.29
      name: exact match
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/Virtuoso-Small
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: GPQA (0-shot)
      type: Idavidrein/gpqa
      args:
        num_few_shot: 0
    metrics:
    - type: acc_norm
      value: 11.52
      name: acc_norm
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/Virtuoso-Small
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MuSR (0-shot)
      type: TAUR-Lab/MuSR
      args:
        num_few_shot: 0
    metrics:
    - type: acc_norm
      value: 14.44
      name: acc_norm
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/Virtuoso-Small
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MMLU-PRO (5-shot)
      type: TIGER-Lab/MMLU-Pro
      config: main
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 46.57
      name: accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/Virtuoso-Small
      name: Open LLM Leaderboard
---
# Virtuoso-Small-RK3588-1.1.2

This version of Virtuoso-Small has been converted to run on the RK3588 NPU using ['w8a8', 'w8a8_g128', 'w8a8_g256', 'w8a8_g512'] quantization.
This model has been optimized with the following LoRA: 

Compatible with RKLLM version: 1.1.4

## Useful links:
[Official RKLLM GitHub](https://github.com/airockchip/rknn-llm) 

[RockhipNPU Reddit](https://reddit.com/r/RockchipNPU) 

[EZRKNN-LLM](https://github.com/Pelochus/ezrknn-llm/) 

Pretty much anything by these folks: [marty1885](https://github.com/marty1885) and [happyme531](https://huggingface.co/happyme531) 

Converted using https://github.com/c0zaut/ez-er-rkllm-toolkit 

# Original Model Card for base model, Virtuoso-Small, below:


<div align="center">
  <img src="https://i.ibb.co/pXD6Bcv/SW2-U-g-QQLSH1-ZAbxhs-Iu-A.webp" alt="Virtuoso-Small" style="border-radius: 10px; box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2), 0 6px 20px 0 rgba(0, 0, 0, 0.19); max-width: 100%; height: auto;">
</div>

GGUF Available [Here](https://huggingface.co/arcee-ai/Virtuoso-Small-GGUF)

# Virtuoso-Small

Virtuoso-Small is the debut public release of the Virtuoso series of models by Arcee.ai, designed to bring cutting-edge generative AI capabilities to organizations and developers in a compact, efficient form. With 14 billion parameters, Virtuoso-Small is an accessible entry point for high-quality instruction-following, complex reasoning, and business-oriented generative AI tasks. Its larger siblings, Virtuoso-Medium and Virtuoso-Large, offer even greater capabilities and are available via API at [models.arcee.ai](https://models.arcee.ai).

## Key Features

- **Compact and Efficient**: With 14 billion parameters, Virtuoso-Small provides a high-performance solution optimized for smaller hardware configurations without sacrificing quality.
- **Business-Oriented**: Tailored for use cases such as customer support, content creation, and technical assistance, Virtuoso-Small meets the demands of modern enterprises.
- **Scalable Ecosystem**: Part of the Virtuoso series, Virtuoso-Small is fully interoperable with its larger siblings, Forte and Prime, enabling seamless scaling as your needs grow.

---

## Deployment Options

Virtuoso-Small is available under the Apache-2.0 license and can be deployed locally or accessed through an API at [models.arcee.ai](https://models.arcee.ai). For larger-scale or more demanding applications, consider Virtuoso-Forte or Virtuoso-Prime.


# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_arcee-ai__Virtuoso-Small)

|      Metric       |Value|
|-------------------|----:|
|Avg.               |39.43|
|IFEval (0-Shot)    |79.35|
|BBH (3-Shot)       |50.40|
|MATH Lvl 5 (4-Shot)|34.29|
|GPQA (0-shot)      |11.52|
|MuSR (0-shot)      |14.44|
|MMLU-PRO (5-shot)  |46.57|