Yeah, but how often is this? Certainly there are weird hobbyist edge cases that don't do well with this, that's true of anything, but a provider is loading weights once every few months
It all depends how many models you are serving from that flash and whether they all fit in there together. If you need to evict and load models, the flash will die a horrible death. This is one resource you should NEVER underprovision.
Yeah, but how often is this? Certainly there are weird hobbyist edge cases that don't do well with this, that's true of anything, but a provider is loading weights once every few months
It all depends how many models you are serving from that flash and whether they all fit in there together. If you need to evict and load models, the flash will die a horrible death. This is one resource you should NEVER underprovision.