@kaia@brotka.st TheBloke does an awesome job releasing quantized version of the models. But I am not sure if those will work with llama.cpp. I use the scripts provided by llama.cpp to produce a quantized model.
https://huggingface.co/TheBloke
Embed Notice
HTML Code
Corresponding Notice
- Embed this notice
Emi Yusa (gnuxeava@mk.absturztau.be)'s status on Wednesday, 12-Jul-2023 22:44:44 JSTEmi Yusa