OPEA
/

Qwen2.5-72B-Instruct-int2-sym-inc

intel/auto-round

Model card Files Files and versions Community

cicdatopea commited on 17 days ago

Commit

8638c91

·

verified ·

1 Parent(s): 016d9a0

Update README.md

Files changed (1) hide show

README.md +7 -1

README.md CHANGED Viewed

@@ -107,8 +107,14 @@ auto-round --model "OPEA/Qwen2.5-72B-Instruct-int2-sym-inc" --eval --eval_bs 16
 | Metric                                     |  BF16  |  INT4  |
 | :----------------------------------------- | :----: | :----: |
-| Avg                                        | 0.7171 | 0.6843 |
 | mmlu                                       | 0.8334 | 0.7634 |
 | lambada_openai                             | 0.7518 | 0.7215 |
 | hellaswag                                  | 0.7031 | 0.6464 |
 | winogrande                                 | 0.7601 | 0.7553 |

 | Metric                                     |  BF16  |  INT4  |
 | :----------------------------------------- | :----: | :----: |
+| Avg                                        | 0.7413 | 0.6835 |
+| leaderboard_mmlu_pro 5 shots	             | 0.5919 | 0.4602 |
+| leaderboard_ifeval inst_level_strict_acc	 | 0.7770 | 0.6715 |
+| leaderboard_ifeval prompt_level_strict_acc | 0.6858 | 0.5379 |
+| gsm8k(5shot) strict match	                 | 0.9037 | 0.8666 |
 | mmlu                                       | 0.8334 | 0.7634 |
+| cmmlu                                      | 0.8727 | 0.7772 |
+| ceval-valid                                | 0.8975 | 0.7786 |
 | lambada_openai                             | 0.7518 | 0.7215 |
 | hellaswag                                  | 0.7031 | 0.6464 |
 | winogrande                                 | 0.7601 | 0.7553 |