Comment by azuanrb

I’m building an internal tool for our company, basically an agent to help with on-call and alerts via Slack. I have evals running across a few scenarios, and my favorite models so far are Sol medium and Luna xhigh.

Sol medium has been a nice balance between intelligence and response time. Luna xhigh can achieve similar scores on the evals, but it takes noticeably longer. My impression is that the higher reasoning effort helps compensate for the lower base intelligence.

Cost is definitely a big factor, but latency and intelligence matter too. If I had the budget, I’d take Sol medium over Luna xhigh.

From using both on real scenarios, Sol is noticeably better at navigating around issues, exploring alternatives, and being creative when the obvious approach doesn’t work. That matters quite a bit when you’re investigating live alerts, where the path to the root cause isn’t always straightforward.