How come the suggestion of using Koboldcpp with this model?
Hi! Sorry for the silly question, but I was wondering if there were any particular reasons for your suggestion to use the model with Koboldcpp. Usually, I've always used Ooba, and I was wondering why this particular suggestion! Btw I really like the model, it's my "go-to" at the moment! (GGUF Q8).
Thanks :)
Hello! No worries at all :D It's not silly and I'm happy to answer. I just suggested to use Koboldcpp because it's the one I used to test the model, but it should really be fine to use with any backend you like. I just like Koboldcpp's Context Shifting option, because it helps load the context faster, and I believe Ooba doesn't have that option yet? Also, the XTC samplers are new and might not be available in Ooba too. Although the model should work fine without it, I find that responses are much better with XTC. It's been so long since the last time I used Ooba so I'm not updated anymore, sorry about that!
I'm happy to see you and many people like the model! I really didn't expect to get such feedback since it's my first merge. Thank you for giving it a try and happy chatting ╰(▔∀▔)╯
Thank you so much for the response! So, I'm quite ignorant on the matter, but I think that the recent versions of Ooba support both context shifting and XTC... but again, I'm truly ignorant, so I can't put a finger on it ahah! Anyway, perfect, understood! Thanks again for the reply and, well, for the model ahah~ Good luck with future merges ^^