Is it possible to use this model with huggingface's transformers library?
#2
by
Jenish-23
- opened
Can we skip lmdeploy dependency?
It would be great if you can provide an example to use this model either with autoawq, llm-awq or transformers library.
Thank you for your question! Currently, we only support loading AWQ models with lmdeploy. We appreciate your suggestion and will consider expanding support in the future.