Comment by simonw
1 year ago
Don't forget the importance of data privacy. Updating a model with fresh information makes that information available to ALL users of that model. This often isn't what you want - you can run RAG against a user's private email to answer just their queries, without making that email "baked in" to the model.
You don't need to update the whole model for everyone. Fine tuning exists and is even available as a service in openai. The updates are only visible in the specific models you see.
Maintaining a fine-tuned model for every one of your users - even with techniques like LoRA - sounds complicated and expensive to me!
It is, but it's also not that bad. A copy of the weights is X GB of cloud storage, which can be stored as a diff if it helps, and added compute time for loading a custom model and unloading for the next customer. It's not free, but it's an approachable cost for a premium service.