altomek/Lo101-3B-AnD-GGUF
Text Generation
•
Updated
•
66
Nice, thanks for the link! Yes, just wanted to be sure I need software update to make use of this. Looks like I may like IQ4_NL more in near future :P
Huh interesting, however all inference engines need to adapt newer llama.cpp version correct? Q4_0 and IQ4_NL? Just scrolled throught the pull request. How do you know IQ4_NL should work this way also?