Replying to @antirez
This prefill may not be fast in absolute sense but, on thi hardware, is almost the double I can get running llama.cpp
1
3
828
I can't contain my excitement @antirez , I know that you and the community are used to far superior speeds, but I've neve ever imagine to be able to run a 35B model on a simple CPU, with only 14.4G of ram
2
37
Replying to @ivanfioravanti @fal
I was warching video like youtu.be/Vt8jTbWJfdM?is=kXNV… and it was umbelivable the progresses made, both on quality and length
8




