← Back to context

Comment by matheusmoreira

5 hours ago

> GLM 5.2 Max = Opus 4.8 Max in thinking behavior

This is insane! I can't wait until technology progresses to the point we can run these things on consumer hardware!

8 comments

matheusmoreira

Reply
chartpath  4 hours ago

Are there any indications that this will be possible? Consumer hardware will continue getting better but I can't see 512GB RAM in a MacBook Pro any time soon. I'm hoping linear attention techniques plus MoE will make breakthroughs in size/compression and throughput.

  • nijave  2 hours ago

    Well, we're probably not going to be running frontier models anytime soon, but I think the general assumption is smaller models will continue to improve until they're sufficiently good frontier models aren't needed.

    There's potentially also augmentation through tools, harnesses and RAG to help boost how well they work without tons of parameters.

  • irishcoffee  39 minutes ago

    There will be a Linux for models. Llama-is-not-upstream-xAI or something if that ilk.

    If the only most is time and money, that isn’t a moat.

muyuu  2 hours ago

you need 8 x 96GB Blackwell or equivalent

so around US$150k which is Small/Medium-Enterprise territory already, but who knows when it will hit "reasonable" home consumer territory

I think there's hope future generations of unified memory machines may get this sort of memory availability when new fabs open in then next couple of years and then ramp up production for a few years afterwards - that makes ~2030s credible at this point, but nobody can really predict the market that far ahead

  • matheusmoreira  1 hour ago

    > I think there's hope future generations of unified memory machines may get this sort of memory availability

    I hope you're right. This is a very exciting idea. The weights are out there. The demand is astronomical. The manufacturers just need to make it happen.

  • sterlind  1 hour ago

    there are cheaper ways to do it. not like, consumer-cheap, but I'm setting up a rig for 80% cheaper than that.

    I'm a tad worried about triggering a run on the particular hardware I'm buying though so I'll leave it vague here, but hit me up on Discord if you're curious.