Comment by qsbuilder
11 hours ago
I always wonder is it safe to run one of these models on a personal pc, or do you guys recommend something like docker, sorry a bit new to all of this.
11 hours ago
I always wonder is it safe to run one of these models on a personal pc, or do you guys recommend something like docker, sorry a bit new to all of this.
Yes, it's fine to run a model on bare metal. The model is just a token predictor. Leave out the fine semantics about this; it's a function taking a set of input tokens to output tokens.
It can't mess with your computer or files until you hook it to a harness, which interprets some of the model's output as commands to execute. So, model on bare metal, harness in a container or VM.