Comment by unshavedyak

I gave a fairly complex reverse engineering task to DS-4 xhigh and GPT-5.5 xhigh today.

After about 6 hours, both ultimately failed to fully RE, however, there were some drastic differences:

DS stopped every 30 minutes or so, saying it did full RE and it should all work now, while in fact, it didn't complete even 1% of it. It also looked for shortcuts again and again, despite me prompting heavily that the specific shortcut may not be used. It was a complete and utter failure.

GPT-5.5, on the other hand, blew me away. It just did the right things, didn't jump to next steps until it was sure it completed the initial layers and had a full understanding of what's required. The only time I prompted it during the 6 hours was when I saw it going in the right direction and I could nudge it slightly towards an even better way. I never felt I was fighting it. Okay, maybe a little bit - after compaction, it sometimes would go on a "no I'm not helping you with reverse engineering" tangent, but it would resolve in a clean session.

I cancelled my Claude subscription a month ago, so I haven't tested that, but DeepSeek has reminded me a lot of how I worked with Opus 4.6/4.7. Which perhaps could be a positive sign to some, but GPT-5.5 showed me that the way claude/ds work is just way too annoying.

> I'm concerned since i really want SOTA reasoning

I think you should give other models a try and see how much they differ from SOTA models. I did this and realized, even Qwen-2.5-Max was enough. I am sure even Claude Sonnet 3.5 is enough for things I play around with. I am not really striving for fields medal in Mathematics.

You should definitely stick to the $200 plan, and not try the $10 coding plans with open weight models and higher limits. Anthropic needs your money to stay solvent, and you'll sleep better knowing you're using SOTA.

> i've been wanting to go elsehwere.

There's always the option of using Anthropic's models for some tasks like planning and then just hand over the implementation task to something like DeepSeek. Across different tools, a Markdown plan works pretty okay. That's what I'm planning to do if I go from the 5x Max subscription down to the Pro.

I am also writing a launcher that makes using 3rd party providers with Claude Code easy (https://ccode.kronis.dev) and I already have a local proxy up and running, just not dynamic model switching yet. Though it shouldn't be too hard to add, will probably be there within a week or two, depending on my schedule.

I don't think it's wise to leave Anthropic altogether because their models are great (and a subscription gives you features like Remote Control which I like), but switching tiers and maybe saving a bit of money seems viable! On the other hand, you do need a quality baseline, because I remember using Cerebras with GLM 4.6 way back and there was a bit too much slop.

If you want SOTA reasoning you should be using GPT 5.5 Pro.