← Back to context

Comment by slowin

9 hours ago

I believe they can currently be used productively for non-coding tasks (classification, light summary)... but they definitely are not even close to SOTA when it comes to software development.

12 comments

slowin

Reply
_hugerobots_  8 hours ago

Defining productivity is a use-case scenario, and a wildly generalized assumption for most people in this argument. Local infrastructure doesn't need to be sota for absolutely every single need for a dev lab, but it absolutely can be delivered with non-api frontier class models.

  • slowin  8 hours ago

    Just to be clear, I'm specifically talking about coding. I think local models can help with productivity today, just not coding.

    I'm also a huge fan of local models and think it's absolutely imperative that they continue to advance so we can move off of the Anthropic/OpenAI hosted models. It's important to accurately asses where we are in that journey though.

    • srcreigh  8 hours ago

      I think the issue is generalization, if you were more specific about which local models aren’t good enough for which tasks compared to which frontier models in your experience, it’d be a lot more informative

      1 reply →

    • _hugerobots_  8 hours ago

      Like the other commenter, I'm confused about the 'just not coding' conclusion. I'm using Qwen 27B on a 5090 at > 100tk/s with 150k context (which isn't enough admittedly), and DeepSeek v4 Flash with 1million context on a gb10/spark. Both of which are performing surface level, and deep needle precision infrastructure architecture. They code 24-7, stupendously.

      1 reply →