lucyknada commited on
Commit
8c54577
·
verified ·
1 Parent(s): ad90bc7

Upload ./README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +142 -0
README.md ADDED
@@ -0,0 +1,142 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model:
4
+ - Qwen/Qwen2.5-14B
5
+ model-index:
6
+ - name: Virtuoso-Small
7
+ results:
8
+ - task:
9
+ type: text-generation
10
+ name: Text Generation
11
+ dataset:
12
+ name: IFEval (0-Shot)
13
+ type: HuggingFaceH4/ifeval
14
+ args:
15
+ num_few_shot: 0
16
+ metrics:
17
+ - type: inst_level_strict_acc and prompt_level_strict_acc
18
+ value: 79.35
19
+ name: strict accuracy
20
+ source:
21
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/Virtuoso-Small
22
+ name: Open LLM Leaderboard
23
+ - task:
24
+ type: text-generation
25
+ name: Text Generation
26
+ dataset:
27
+ name: BBH (3-Shot)
28
+ type: BBH
29
+ args:
30
+ num_few_shot: 3
31
+ metrics:
32
+ - type: acc_norm
33
+ value: 50.4
34
+ name: normalized accuracy
35
+ source:
36
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/Virtuoso-Small
37
+ name: Open LLM Leaderboard
38
+ - task:
39
+ type: text-generation
40
+ name: Text Generation
41
+ dataset:
42
+ name: MATH Lvl 5 (4-Shot)
43
+ type: hendrycks/competition_math
44
+ args:
45
+ num_few_shot: 4
46
+ metrics:
47
+ - type: exact_match
48
+ value: 34.29
49
+ name: exact match
50
+ source:
51
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/Virtuoso-Small
52
+ name: Open LLM Leaderboard
53
+ - task:
54
+ type: text-generation
55
+ name: Text Generation
56
+ dataset:
57
+ name: GPQA (0-shot)
58
+ type: Idavidrein/gpqa
59
+ args:
60
+ num_few_shot: 0
61
+ metrics:
62
+ - type: acc_norm
63
+ value: 11.52
64
+ name: acc_norm
65
+ source:
66
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/Virtuoso-Small
67
+ name: Open LLM Leaderboard
68
+ - task:
69
+ type: text-generation
70
+ name: Text Generation
71
+ dataset:
72
+ name: MuSR (0-shot)
73
+ type: TAUR-Lab/MuSR
74
+ args:
75
+ num_few_shot: 0
76
+ metrics:
77
+ - type: acc_norm
78
+ value: 14.44
79
+ name: acc_norm
80
+ source:
81
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/Virtuoso-Small
82
+ name: Open LLM Leaderboard
83
+ - task:
84
+ type: text-generation
85
+ name: Text Generation
86
+ dataset:
87
+ name: MMLU-PRO (5-shot)
88
+ type: TIGER-Lab/MMLU-Pro
89
+ config: main
90
+ split: test
91
+ args:
92
+ num_few_shot: 5
93
+ metrics:
94
+ - type: acc
95
+ value: 46.57
96
+ name: accuracy
97
+ source:
98
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/Virtuoso-Small
99
+ name: Open LLM Leaderboard
100
+ ---
101
+ ### exl2 quant (measurement.json in main branch)
102
+ ---
103
+ ### check revisions for quants
104
+ ---
105
+
106
+
107
+ <div align="center">
108
+ <img src="https://i.ibb.co/pXD6Bcv/SW2-U-g-QQLSH1-ZAbxhs-Iu-A.webp" alt="Virtuoso-Small" style="border-radius: 10px; box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2), 0 6px 20px 0 rgba(0, 0, 0, 0.19); max-width: 100%; height: auto;">
109
+ </div>
110
+
111
+ GGUF Available [Here](https://huggingface.co/arcee-ai/Virtuoso-Small-GGUF)
112
+
113
+ # Virtuoso-Small
114
+
115
+ Virtuoso-Small is the debut public release of the Virtuoso series of models by Arcee.ai, designed to bring cutting-edge generative AI capabilities to organizations and developers in a compact, efficient form. With 14 billion parameters, Virtuoso-Small is an accessible entry point for high-quality instruction-following, complex reasoning, and business-oriented generative AI tasks. Its larger siblings, Virtuoso-Medium and Virtuoso-Large, offer even greater capabilities and are available via API at [models.arcee.ai](https://models.arcee.ai).
116
+
117
+ ## Key Features
118
+
119
+ - **Compact and Efficient**: With 14 billion parameters, Virtuoso-Small provides a high-performance solution optimized for smaller hardware configurations without sacrificing quality.
120
+ - **Business-Oriented**: Tailored for use cases such as customer support, content creation, and technical assistance, Virtuoso-Small meets the demands of modern enterprises.
121
+ - **Scalable Ecosystem**: Part of the Virtuoso series, Virtuoso-Small is fully interoperable with its larger siblings, Forte and Prime, enabling seamless scaling as your needs grow.
122
+
123
+ ---
124
+
125
+ ## Deployment Options
126
+
127
+ Virtuoso-Small is available under the Apache-2.0 license and can be deployed locally or accessed through an API at [models.arcee.ai](https://models.arcee.ai). For larger-scale or more demanding applications, consider Virtuoso-Forte or Virtuoso-Prime.
128
+
129
+
130
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
131
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_arcee-ai__Virtuoso-Small)
132
+
133
+ | Metric |Value|
134
+ |-------------------|----:|
135
+ |Avg. |39.43|
136
+ |IFEval (0-Shot) |79.35|
137
+ |BBH (3-Shot) |50.40|
138
+ |MATH Lvl 5 (4-Shot)|34.29|
139
+ |GPQA (0-shot) |11.52|
140
+ |MuSR (0-shot) |14.44|
141
+ |MMLU-PRO (5-shot) |46.57|
142
+