Google Colab pro+ TPU and JAIS 30B
#10
by
HanaRasheed
- opened
Hello, Could you please advise in the following matter: I am trying to run the model on google colab pro+ using TPU run time, but it is taking much time to run it and see the output.
I'm sorry
@HanaRasheed
to hear that but the available memory per TPU core (even in colab pro+) is insufficient for such a large model in FP32 !
Please check quantizations from the community, (almazrooei33/jais-family-30b-16k-chat-4bit), (https://huggingface.co/amgadhasan/jais-30b-chat-v3-fp16) or (https://huggingface.co/mradermacher/jais-30b-chat-v3-fp16-GGUF) for example.
alielfilali01
changed discussion status to
closed