Comment by andsoitis 3 days ago A hosted instance serves multiple customers at a time. A local model only one. 1 comment andsoitis Reply toasty228 3 days ago How many though? At 1m context you quickly fill a full gb300's 280gb of memory
How many though? At 1m context you quickly fill a full gb300's 280gb of memory