bartowski

/

QVQ-72B-Preview-GGUF

like 51

I was going to say you can run directly in ollama from HF but I actually don't know how that works with vision models.. will look into it !

thank you!

I was going to say you can run directly in ollama from HF but I actually don't know how that works with vision models.. will look into it !

well i imported the gguff into openwebui and....

this happens every time and here is the translation:

, Hello little fairies~
Your editor is online again
Recently I'm following "The Next Stop Is Happiness"
It's really too sweet
Song Weilong and Song Qian have a full sense of CP

wait... this is diffrent a while ago it was yapping about a cook and giving me recipies

Hi, seems like we have a problem parsing GGUF file on hugging face backend. I'll report it to the team

So, this is a real bug? I thought this was a problem on my side, thanks a lot! Now I don't have to debug my Ollama or OWUI.

Yes it's a bug on HF side. I don't know yet when we can deploy a fix, given that many of us are on vacation.

In the meantime, you can try adding the chat template yourself via a Modelfile, ref ollama docs

FROM hf.co/bartowski/QVQ-72B-Preview-GGUF
TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
"""
PARAMETER stop "<|im_end|>"

thank you and also, is the problem with all the ggufs or just the q4km's?

it's for all quants

any updates?

I'll try to take a look tomorrow at the process

❤️ thanks a lot! you are a legend!

Haven't been able to figure it out :( was able to upload a file, but not sure how vision is meant to be uploaded/work, if anyone has insight feel free to let me know but the documentation seems quite lacking

Hi @bartowski , sorry for the late response. It seems like the fix can be applied from HF server side. I'll discuss this with our team this week.