@bert_hubert you can get a surprising amount of mileage with local models with stuff like ggml, although it still consumes a consirable amount of hard drive space.

still, local AI is here now. now we just gotta get the people who train them to embrace sparsity so we can get building them cheaper too :ablobcatbongokeyboard: