Comment by embedding-shape

> What I want is to have a directory with models and bind mount that readonly into inference containers.

Yeah, that's basically what I'm doing, + over network (via Samba). My weights all live on a separate host, which has two Samba shares, one with write access and one read-only. The write one is mounted on my host, and the container where I run the agent mounts the read-only one (and have the source code it works on copied over to the container on boot).

The directory that LM Studio ends up creating and maintaining for the weights, works with most of the tooling I come across, except of course Ollama.