Crataco commited on
Commit
15092aa
·
verified ·
1 Parent(s): 48a1101

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +16 -0
README.md CHANGED
@@ -25,6 +25,22 @@ license: other
25
 
26
  This is [StableLM 2 Chat 1.6B](https://huggingface.co/stabilityai/stablelm-2-1_6b-chat), quantized with the help of imatrix so it could offer better performance for being quantized, and have quantization levels available for lower-memory devices to run. [Kalomaze's "groups_merged.txt"](https://github.com/ggerganov/llama.cpp/discussions/5263#discussioncomment-8395384) was used for the importance matrix, with context set to 4,096 (the context length according to [their paper](https://drive.google.com/file/d/1JYJHszhS8EFChTbNAf8xmqhKjogWRrQF/view)).
27
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
28
  Original model card below.
29
 
30
  ***
 
25
 
26
  This is [StableLM 2 Chat 1.6B](https://huggingface.co/stabilityai/stablelm-2-1_6b-chat), quantized with the help of imatrix so it could offer better performance for being quantized, and have quantization levels available for lower-memory devices to run. [Kalomaze's "groups_merged.txt"](https://github.com/ggerganov/llama.cpp/discussions/5263#discussioncomment-8395384) was used for the importance matrix, with context set to 4,096 (the context length according to [their paper](https://drive.google.com/file/d/1JYJHszhS8EFChTbNAf8xmqhKjogWRrQF/view)).
27
 
28
+ Here's a chart that provides an approximation of the HellaSwag score (out of 1,000 tasks). Thanks to the randomization of tasks, it may be slightly unprecise:
29
+ |Quantization|HellaSwag|
30
+ |------------|---------|
31
+ |IQ1_S |35.4% |
32
+ |IQ1_M |38.7% |
33
+ |IQ2_XXS |51.2% |
34
+ |IQ2_XS |51.8% |
35
+ |IQ2_S |56.8% |
36
+ |IQ2_M |59.3% |
37
+ |Q2_K_S |55.2% |
38
+ |Q2_K |59.0% |
39
+ |IQ3_XXS |60.8% |
40
+ |Q4_0 |64.0% |
41
+ |Q4_K_M |66.0% |
42
+ |Q5_K_M |65.8% |
43
+
44
  Original model card below.
45
 
46
  ***