← Back to context

Comment by kanemcgrath

18 hours ago

I tried it for a bit, and It was not really worth its size. It got swept up in all the other AI news recently, but laguna s 2.1 I think is the best ~100B moe model right now

25 comments

kanemcgrath

Reply
nozzlegear  18 hours ago

I didn't mention it above, but Laguna S is my other favorite model. I use Qwen a lot more, it's smaller and faster, but I like to switch to Laguna when I feel like I need a "heavy hitter" for certain huge or complex tasks.

  • tommica  17 hours ago

    What on earth hardwares do you guys have to be able to run 100gb models locally?! That's crazy! I'm here struggling to even get 27b models to run in somewhat usable way

    • pettijohn  8 hours ago

      Strix Halo, 128GB RAM. I got a refurbished Corsair AI Workstation for a smoking price ($2100) about two months ago. Lucky timing that it was in stock.

      3 replies →

    • nozzlegear  17 hours ago

      Haha I'm on an Mac Studio with an M1 Ultra, 64gb ram. I bought it when it first came out, it just happens to be good for local LLMs. I have to use a smaller quant of Laguna S though (I think 4-bit? Not at my machine to check), as 8-bit and full size definitely don't fit in the 64gb I have.

      5 replies →

    • numpad0  13 hours ago

      Yeah, here I am sitting deeply deeply deeply regretting not buying couple CMP 170HX at $200 or $350, knowing I could just flip them ethically at purchase price if nothing came of it... I could have just casually built a 128GB dual A100 local AI monster

      2 replies →

    • sznio  15 hours ago

      quantized + offload

      I have an RX 6700 XT with 12gb vram and 64gb system ram. running dense models like 27b is difficult, but i can run IQ4/IQ5 qwen 122b-a10b or 35b-a3b at ~20tok/s

      1 reply →

    • cyanydeez  9 hours ago

      Usually theyre quantized. Also, there was a window where AMD 395+ W/128GB was just a high end $2500 hardware with unified gpu memory.

  • ndom91  9 hours ago

    Using qwen 3.6 27b for local coding as well and downloaded Laguna s 2.1 but haven't had time to give it a full spin yet.

    Curious for any more experiences