Comment by Catloafdev
2 hours ago
Neat! But, what do you do with a 0.5tk/s LLM?
Have you tried running it via llamacpp or other software that supports naive SSD offloading to compare speeds?
2 hours ago
Neat! But, what do you do with a 0.5tk/s LLM?
Have you tried running it via llamacpp or other software that supports naive SSD offloading to compare speeds?
Have it summarise the week overnight for the meeting in the morning. Then have it summarise the meeting transcription overnight for the report tomorrow. Then someone else will have it summarise the report overnight to read on a 6" handheld screen in the small office the next morning after breakfast.
You could use it for long run tasks while you don’t use the laptop.