every time a new >1T model comes out and someone jokes about unsloth’s use of “locally” i want to <removed by the production team> istg locally doesn’t mean “on your laptop”, it means “on your own hardware” it doesn’t say anything about the class of hardware or the investment needed
Replying to @astrra.space@bsky.brid.gy
your homelab is locally
your GPU farm is locally
SSD streaming is locally, too!
literally what the fuck are we doing here, they’re saying exactly what they mean
in theory, yes
in practise, hardware to run it with reasonable speed would cost much higher than any individual can afford
and if it runs with like 1minute/token it's not useful at all