Comment by argee

Some examples (keep in mind this is all indefinitely free for me, no burning quota away):

1. Getting information (such as information about hardware unfamiliar to me) when not connected to the internet, which happens occasionally in my case.

2. Continuing to learn Rust by way of toy examples, puzzles, and comparing aspects of various solutions, for example from LeetCode.

3. Reformatting data, for example from a PDF to a markdown table, or converting receipt images to text.

4. Simple translation/explanation (e.g. I'm teaching my wife one of the languages I speak but sometimes may not know/have the words to explain the full nuance of a translated word).

5. Summarization. One of the webnovels I'm reading has some very boring parts I don't want to slog through, in those cases I simply make the LLM summarize that part and move on.

Etc., you get the idea. It's not unusable for coding, but it would make many mistakes when making a whole feature and the context lengths are limited to around 30k-40k tokens by my RAM. I could give it access to the web but I simply use an online model when I need that sort of thing, again partly due to the context limit.

Edit: The MLX version of Gemma 4 26b a4b does about 62 tok/s.