Comment by thrw93747572007

It sounds like they are doing something similar to what I described in my other post below. Personal media station.

That can be done on hardware that quite a lot of people basically just have and don't use 24/7 to the max - because it is their gaming machine or their programming and compiling workhorse, for example. Of course you are paying for additional electricity but even with napkin-math instead of a "proper" calculation, you are unlikely to pay more for running your own instead of something commercial (and that can be offset further with some of the "modern" electricity contracts and/or PV and battery storage). Especially if we are talking about a stack that runs most of/all the time when you are not using your machine and makes LLM calls regularly while running.

The work in software/admin to get whatever you want set up is similiar no matter which infrastructure you use.