Title says 8-bit while model card sentence says 4-bit
#1
by
StanHatko
- opened
The model title has Int8 in the name and it has the tag "8-bit precision". However, the first sentence of the model card says:
This is a GPTQ-quantized 4-bit version of huihui-ai/Llama-3.3-70B-Instruct-abliterated-finetuned.
If this is an 8-bit model, can you please fix the first sentence of the model card? Or the other way around if it's a 4-bit model?
Sorry for the confusion. This is copied from the 4-bit format, and it has been corrected. You can check the quantization
settings in the config.json file for more details.
Thank you for quickly responding and fixing this.
StanHatko
changed discussion status to
closed