Comment by benob

What local models shine as local assistants? Is there an effort to evaluate the compromise between compute/memory and local models that can support this use case? What kind of hardware do you need to not feel like playing with a useless shiny toy?