Embed Notice
HTML Code
Corresponding Notice
- Embed this notice
guizzy (in exile) (guizzy@shitposter.club)'s status on Wednesday, 14-Feb-2024 01:00:05 JSTguizzy (in exile) @Moon @kaia @4censord To run 70B models or Mixtral entirely on GPU at higher bit precision (4, 5, 6 bit quants), especially with a large max context, you need more than 24gb.