Has there been any effort taken to reduce data leakage of this test set? Sounds like these exercises were available on the internet pre-2023, so they'll probably be included in the training data for any modern model, no?
I try not to let perfect be the enemy of good. All benchmarks have limitations.
The Exercism problems have proven to be very effective at measuring an LLM's ability to modify existing code. I receive a lot of feedback that the aider benchmarks correlate strongly with people's "vibes" on model coding skill. I agree. The scores have felt quite aligned with my hands-on experience coding with most of the top models over the last 18+ months.
To be clear, the purpose of the benchmark is to help me quantitatively assess and improve aider and make it more effective. But it's also turned out to be a great way to measure the coding skill of LLMs.
Paul, I saw in the notes that using claude with thinking mode requires yml config updates -- any pointers here? I was parsing some commits, and I couldn't tell if you only added architect support through openrouter. Thanks!
Interesting that the "correct diff format" score went from 99.6% with Claude 3.5 to 93.3% for Claude 3.7. My experience with using claude-code was that it consistently required several tries to get the right diff. Hopefully all that will improve as they get things ironed out.
Reasoning models pretty reliably seem to do worse at exacting output formats/structured outputs—so far with Aider it has been an effective strategy to employ o1 to “think” about the issue at hand, and have Sonnet implement. Interested to try various approaches with 3.7 in various combinations of reasoning effort.
3.7 completed a lot more than 3.5, without seeing the actual results, we can't tell if there were any regressions in the edit format among the previously completed tasks.
That's a file context problem because you use cursor or cline or some other crap context maker. Try Clood.
Unless "anthropic high usage" which I just watch the incident reports I one shot features regularly.
At a high skill level. Not front end. Back end c# in a small but great framework that has poor documentation. Not just endpoints but full on task queues.
So really, it's a context problem. You're just not laser focusing your context.
Try this:
Set up a context with the exact files needed. Sure ai "should" do that but it doesn't. Especially not cursor or cline. Then try.
Hell try it with clood after I update with 3.7. I bet you, if you clood file it, then you get one shots.
I have a long history of clood being a commit in my projects and it's a clood one shot.
Hi Paul, been following the aider project for about a year now to develop an understanding of how to build SWE agents.
I was at the AI Engineering Summit in NYC last week and met an (extremely senior) staff ai engineer doing somewhat unbelievable things with aider. Shocking things tbh.
Is there a good way to share stories about real-world aider projects like this with you directly (if I can get approval from him)? Not sure posting on public forum is appropriate but I think you would be really interested to hear how people are using this tool at the edge.
Have you tried Claude 3.7 + Deepseek as the architect? Seeing as "DeepSeek R1 + claude-3-5-sonnet-20241022" is the second place option, "DeepSeek R1 + claude-3-7" would hopefully be the highest ranking choice so far?
Hi everyone! Boris from the Claude Code team here. @eschluntz, @catherinewu, @wolffiex, @bdr and I will be around for the next hour or so and we'll do our best to answer your questions about the product.
One thing I would love to have fixed - I type in a prompt, the model produces 90% or even 100% of the answer, and then shows an error that the system is at capacity and can't produce an answer. And then the response that has already been provided is removed! Please just make it where I can still have access to the response that has been provided, even if it is incomplete.
To me it doesn’t look like a bug. I believe it is a intended “feature” pushed from high management - a dark patern to make plebs pay for answer that has overflowed the quota.
The biggest complaint I (and several others) have is that we continuously hit the limit via the UI after even just a few intensive queries. Of course, we can use the console API, but then we lose ability to have things like Projects, etc.
Do you foresee these limitations increasing anytime soon?
Quick Edit: Just wanted to also say thank you for all your hard work, Claude has been phenomenal.
I paid for it for a while, but I kept running out of usage limits right in the middle of work every day. I'd end up pasting the context into ChatGPT to continue. It was so frustrating, especially because I really liked it and used it a lot.
It became such an anti-pattern that I stopped paying. Now, when people ask me which one to use, I always say I like Claude more than others, but I don’t recommend using it in a professional setting.
The gateway is integrated directly into our chat (https://glama.ai/chat). So you can use most of the things that you are used to having with Claude. And if anything is missing, just let me know and I will prioritize it. If you check our Discord, I have a decent track record of being receptive to feedback and quickly turning around features.
Long term, Glama's focus is predominantly on MCPs, but chat, gateway and LLM routing is integral to the greater vision.
I would love feedback if you are going to give a try frank@glama.ai
this is also my problem, ive only used the UI with $20 subscription, can I use the same subscription to use the cli? I'm afraid its like those aws api billing where there is no limit to how much I can use then get a surprise bill
Claude is my go to llm for everything, sounds corny but it's literally expanding the circle of what I can reasonably learn, manyfold. Right now I'm attempting to read old philosophical texts (without any background in similar disciplines), and without claude's help to explain the dense language in simpler terms & discuss its ideas, give me historical contexts, explaining why it was written this or that way, compare it against newer ideas - I would've given up many times.
At work I used it many times daily in development. It's concise mode is a breath of fresh air compared to any other llm I've tried. It has helped me find bugs in foreign code bases, explain me the techstack, written bash scripts, saving me dozens of hours of work & many nerves. It generally makes me reach places I wouldn't without due to time constraints & nerves.
The only nitpick is that the service reliability is a bit worse than others, forcing me sometimes to switch to others. This is probably a hard to answer question, but are there plans to improve that?
I'm in the middle of a particularly nasty refactor of some legacy React component code (hasn't been touched in 6 years, old class based pattern, tons of methods, why, oh, why did we do XYZ) at work and have been using Aider for the last few days and have been hitting a wall. I've been digging through Aider's source code on Github to pull out prompts and try to write my own little helper script.
So, perfect timing on this release for me! I decided to install Claude Code and it is making short work of this. I love the interface. I love the personality ("Ruminating", "Schlepping", etc).
Just an all around fantastic job!
(This makes me especially bummed that I really messed up my OA awhile back for you guys. I'll try again in a few months!)
Just started playing with the command-line tool. First reaction (after using it for 5 minutes): I've been using `aider` as a daily driver, with Claude 3.5, for a while now. One of the things I appreciate about aider is that it tells you how much each query cost, and what your total cost is this session. This makes it low-key easy to keep tabs on the cost of what I'm doing. Any chance you could add that to claude-code?
I'd also love to have it in a language that can be compiled, like golang or rust, but I recognize a rewrite might be more effort than it's worth. (Although maybe less with claude code to help you?)
EDIT: OK, 10 minutes in, and it seems to have major issues doing basic patches to my Golang code; the most recent thing it did was add a line with incorrect indentation, then try three times to update it with the correct indentation, getting "String to replace not found in file" each time. Aider with claude 3.5 does this really well -- not sure what the counfounding issue is here, but might be worth taking a look at their prompt & patch format to see how they do it.
One of the silver bullets of Claude, in the context of coding, is that it does NOT use RAG when you use it via the web interface. Sure, you burn your tokens but the model sees everything and this let it reply in a much better way. Is Claude Code doing the same and just doing document-level RAG, so that if a document is relevant and if it fits, all the document will be put inside the context window? I really hope so! Also, this means that splitting large code bases into manageable file sizes will make more and more sense. Another Q: is the context size of Sonnet 3.7 the same of 3.5? Btw Thanks you so much for Claude Sonnet, in the latest months it changed the way I work and I'm able to do a lot more, now.
Right -- Claude Code doesn't use RAG currently. In our testing we found that agentic search out-performed RAG for the kinds of things people use Code for.
Been a long time casual — i.e. happy to fix my code by asking questions and copy/pasting individual snippets via the chat interface. Decided to give the `claude` terminal tool a run and have to admit it looks like a fantastic tool.
Haven't tried to build a modern JS web app in years — it took the claude tool just a few minutes of prompting to convert and refactor an old clunky tool into a proper project structure, and using svelte and vite and tailwind (which I haven't built with before). Trying to learn how to even scaffold a modern app has felt daunting and this eliminates 99% of that friction.
One funny quirk: I asked it to build a test suite (I know zilch about JS testing frameworks, so it picked vitest for me) for the newly refactored app. I noticed that 3 of the 20 tests failed and so I asked it to run vitest for itself and fix the failing things. 2 minutes later, and now 7 tests were failing...
Which is very funny to me, but also not a big deal. Again, it's such a chore to research test libs and then set things up to their conventions. That the claude tool built a very usable scaffold that I can then edit and iterate on is such a huge benefit by itself, I don't need (nor desire) the AI to be complete turnkey solution.
Anthropic is back and cementing its place as the creator of the best coding models—bravo!
With Claude Code, the goal is clearly to take a slice of Cursor and its competitors' market share. I expected this to happen eventually.
The app layer has barely any moat, so any successful app with the potential to generate significant revenue will eventually be absorbed by foundation model companies in their quest for growth and profits.
I think an argument could be reasonably made that the app layer is the only moat. It’s more likely Anthropic eventually has to acquire Cursor to cement a position here than they out-compete it. Where, why, what brand and what product customers swipe their credit cards for matters — a lot.
I wonder if they will offer competitive request counts against Cursor. Right now, at least for me, the biggest downside to Claude is how fast I blow through the limits (Pro) and hit a wall.
At least with Cursor, I can use all "premium" 500 completions and either buy more, or be patient for throttled responses.
hi! I've been using Claude Code in a very complementary way to my IDE, and one of the reasons we chose the terminal is because you can open it up inside whichever IDE you want!
Hi Boris, love working with Claude! I do have a question—is there a plan to have Claude 3.5 Sonnet (or even 3.7!) made available on ca-central-1 for Amazon Bedrock anytime soon? My company is based in Canada and we deal with customer information that is required to stay within Canada, and the most recent model from Anthropic we have available to us is Claude 3.
A minor ChatGPT feature I miss with Claude is temporary chats. I use ChatGPT for a lot of random one-off questions and don’t want them filling up my chat history with so many conversations.
Will check out Claude Code soon, but in the meantime one unrelated other feature request:
Moving existing chats into a project. I have a number of old-ish but super-useful and valuable chats (that are superficially unrelated) that I would like to bring together in a project.
I really want to try your AI models, but "You must have a valid phone number to use Anthropic's services." is a show-stopper for me.
It's the only mainstream AI service that requests this information. After a string of security lapses by many of your competitors, I have zero faith in the ability of a "fast moving" AI-focused company to keep my PII data secure.
It's a phone number. It's probably been bought / sold a few times already.
Unless you're on the level of Edward Snowden, I wouldn't worry about it. But maybe your sense of privacy is more valuable than the outcome you'd get from Claude. That's fine too.
I pay for a number from voip.ms and use sms forwarding. Its very cheap and it works on telegram as well which seemed fairly strict at detecting most voips.
Does the fact its so ungodly expensive and highly rate limited kind of prove the modern point that AI actually uses tons of water and electricity per prompt? People are used to streaming YouTube while they sleep and it's hard to think of other web technology this intensive. OpenAI is hostile to this subject. Does Claude have plans to tackle this?
Is there / are you planning a way to set $ limits per API key? Far as I can tell the "Spend limits" are currently per-org only which seems problematic.
Hi! I’ve been using Claude for macOS and iOS coding for a while, and it’s mostly great, but it’s always using deprecated APIs, even if I instruct it not to. It will correct the mistake if I ask it to, but then in later iterations, it will sometimes switch back to using a deprecated API. It also produces a lot of code that just doesn’t compile, so a lot of time is spent fixing the made up or deprecated APIs.
Awesome to see a new Claude model - since 3.5 its been my go-to for all code related tasks.
I'd really like to use Claude Code in some of my projects vs just sharing snippets via the UI but I'm curious how might doing this from our source directory affect our IP including NDA's, trade secret protections, prior disclosure rules on (future) patents, open source licensing restrictions re: redistribution etc?
Hi Boris et al, can you comment on increased conversation lengths or limits through the UI? I didn't see that mentioned in the blog post, but it is a continued major concern of $20/month Claude.ai users. Is this an issue that should be fixed now or still waiting on a larger deployment via Amazon or something? If not now, when can users expect the conversation length limitations will be increased?
It would be great if we could upgrade API rate limits. I've tried "contacting sales" a few times and never received a response.
edit: note that my team mostly hits rate limits using things like aider and goose. 80k input token is not enough when in a flow, and I would love to experiment with a multi-agent workflow using claude
Now that the world's gotten used to the existence of AI, any hope on removing the guardrails on Claude? I don't need it to answer "How do I make meth", but I would like to not have to social engineer my prompts. I'd like it to just write the code I asked for and not judge me on how ethical the code might be.
Eg Claude will refuse to write code to wget a website and parse the html if you ask it to scrape your ex girlfriend's Instagram profile, for ethical and tos reasons, but if you phrase the request differently, it'll happily go off and generate code that does that exact thing.
Asking it to scrape my ex girlfriend's Instagram profile is just a stand in for other times I've hit a problem where I've had to social engineer my way past those guard rails, but does having those guard rails really provide value on a professional level?
Not having headlines like "Claude Gives Stalker Instructions" has a significant value to their business I would wager.
I'm very much in favour of removing the guardrails but I understand why they're in place. The problem is attribution. You can teach yourself how to engage in all manner of dark deeds with a library or wikipedia or a search engine and some time, but any resulting public outcry is usually diffuse or targeted at the sources rather than the service. When Claude or GPT or Stable Diffusion are used to generate something judged offensive, the outcry becomes an existential threat to the provider.
Unless Cursor had agreed to an exclusivity agreement with Anthropic, Antropic was (and still is) at risk of Cursor moving to a different provider or using their middleman position to train/distill their own model that competes with Anthropic.
honestly, is this something that anthropic should be worried about? you could ask the same question from all the startups that were destroyed by OpenAI.
Claude Code is a research preview -- it's more rough, lets you see model errors directly, etc. so it's not as polished as something like Cline. Personally I use all of the above. Engineers here at Anthropic also tend to use Claude Code alongside IDEs like Cursor.
Thanks for the product! Glad to hear the (so called) "safety" is being walked back on, previously Claude has been feeling a little like it is treating me as a child, excited to try it out now.
In the console, TPM limit for 3.7 is not shown (I'm tier 4). Does it mean there is no limit, or is it just pending and is "variable" until you set it to some value?
We set the Claude Code rate limits to be usable as a daily driver. We expect hitting rate limits for synchronous usage to be uncommon. Since this is a research preview, we recommend you start small as you try the product though.
Your footnote 3 seems to imply that the low number for o1 and Grok3 is without parallelism, but I don't think it's publicly known whether they use internal parallelism? So perhaps the low number already uses parallelism, while the high number uses even more parallelism?
Also, curious if you have any intuition as to why the no-parallelism number for AIME with Claude (61.3%) is quite low (e.g., relative to R1 87.3% -- assuming it is an apples to apples comparison)?
Awesome work, Claude is amazingly good at writing code that is pretty much plug and play.
Could you speak at all about potential IDE integrations? An integration into Jetbrains IDEs would be super useful - I imagine being able to highlight a bit of code and having a plugin check the code graph to see dependencies, tests etc that might be affected by a change.
Copying and pasting code constantly is starting to seem a bit primitive.
Part of our vision is that because Claude Code is just in the terminal, you can bring it into any IDE (or server) you want! Obviously that has tradeoffs of not having a full GUI of the IDE though
I'm not affiliated with Anthropic, but it seems like doing this will commoditize Claude (the AIaaS). Hosted AI providers are doing all they can to move away from being interchangeable commodities; it's not good for Anthropic's revenue for users to be able to easily swap-out the backend of Cloud Code to a local Olama backend, or a cheaper hosted DeepSeek. Open sourcing Claude Code would make this option 1 or 2 forks/PRs away.
What do I need to do to get unbanned? I have filled in the provided Google Docs form 3-4 times to no avail. I got banned almost immediately after joining. My best guess is that I got banned because I used a VPN. https://news.ycombinator.com/item?id=40808815
Is there a way to always accept certain commands across sessions? Specifically for things like reading or updating files I don't want to have to approve that each time I open a new repl.
Also, is there a way to switch models between 3.5-sonnet and 3.5-sonnet-thinking? Got the initial impression that the thinking model is using an excessive amount of tokens on first use.
When you are prompted to accept a bash command, we should be giving you the option to not ask again. If you're not seeing that for a specific bash command, would you mind running /bug or filing an issue on Github? https://github.com/anthropics/claude-code/issues
Thinking and not thinking is actually the same model! The model thinks automatically when you ask it to. If you don't explicitly ask it to think, it won't use thinking.
A bit off topic but I wanted to let you know that anthropic is currently in violation of EU Directive 98/6/EC:
> The selling price and the unit price must be indicated in an unambiguous, easily identifiable and clearly legible manner for all products offered by traders to consumers (i.e. the final price should include value added tax and all other taxes).
I wanted to see what the annual plan would cost as it was just displaying €170+VAT, and when I clicked the upgrade button to find out (I checked everywhere on the page) then I was automatically subscribed without any confirmation and without ever seeing the final price before the transaction was completed.
Hi Boris! Thank you for your work on Claude! My one pet peeve with Claude specifically, if I may: I might be working on a Svelte codebase and Claude will happily ignore that context and provide React code. I understand why, but I’d love to see much less of a deep reliance on React for front-end code generation.
When I first started using Cursor the default behavior was for Claude to make a suggestion in the chat, and if the user agreed with it, they could click apply or cut and paste the part of it they wanted to use in their larger project. Now it seems the default behavior is for Claude to start writing files to the current working directory without regard for app structure or context (e.g., config files that are defined elsewhere claude likes to create another copy of). Why change the default to this? I could be wrong but I would guess most devs would want to review changes to their repo first.
Cursor has two LLM interaction modes, chat and composer. The chat does what you described first and composer can create/edit/delete files directly.
Have you checked which mode you're on? It should be a tab above your chat window.
> We’ve also improved the coding experience on Claude.ai. Our GitHub integration is now available on all Claude plans—enabling developers to connect their code repositories directly to Claude
The https://claude.io/ integration is read-only. Basically you OAuth with GitHub and now you can select a repository, then select files or directories within it to add to either a Claude Project or to an individual prompt.
Claude Code can run commands including "git" commands, so it can create a branch, commit code to that branch and push that branch to GitHub - at which point point you can create a PR.
hey guys! i was wondering why you chose to build Claude code via CLI when many popular choices like cursor and windsurf fork VScode. do you envision the future of Claude code to abstract away the codebase entirely?
We wanted to bring the model to people where they are without having to commit to a specific tool or radically change their workflows. We also wanted to make a way that lets people experience the model’s coding abilities as directly as possible. This has tradeoffs: it uses a lot of tokens, and is rough (eg. it shows you tool errors and model weirdness), but it also gives you a lot of power and feels pretty awesome to use.
I'm curious why there are no results for the "Claude 3.7 Extended Thinking" on SWE-Bench and Agentic tool use.
Are you finding that extended thinking helps a lot when the whole problem can be posed in the prompt, but that it isn't a major benefit for agentic tasks?
It would be a bit surprising, but it would also mirror my experiences, and the benchmarks which show Claude 3.5 being better at agentic tasks and SWE tasks than all other models, despite not being a reasoning model.
From the release you say: "[..] in developing our reasoning models, we’ve optimized somewhat less for math and computer science competition problems, and instead shifted focus towards real-world tasks that better reflect how businesses actually use LLMs."
Can you tell us more about the trade-offs here?
Also, are you using synthetic data for improving the responses here, or are you purely leveraging data from usage/partner's usage?
I recently attempted to use the Google Drive integration but didn't follow through with connecting because Claude wanted access to my entire Google Drive. I understand this simplifies the user experience and reduced time to ship, but is there anyway the team can add "reduce the access scope of Google Drive integration" to your backlog. Thank you!
Also, I just caught the new Github integration. Awesome.
Small UX suggestion, but could you make submission of prompt via URL parameter work? It used to be possible via https://claude.ai/new?q={query}, but that stopped working. It works for ChatGPT, Grok, and DeepSeek. With Claude you have to go and manually click the submit button.
You still need to know what good code looks like to use these tools. If you go forward in your career trusting the output of LLMs without the skills to evaluate the correctness, style, functionality of that code then you will have problems.
People still write low level machine code today, despite compilers having existed for 70+ (?) years.
We'll always need full-stack humans who understand everything down to the electrons even in the age of insane automation that we're entering.
Hi there. There are lots of phrases/patterns that Claude always uses when writing and it was very frustrating with 3.5. I can see with 3.7 those persist.
Is there any way for me to contact you and show those so you can hopefully address them?
Can you give some insight into how you chose the reply limit length? It seems to cut off many useful programs that are 80%-90% done and if the limit were just a little higher it would be a source of extraordinary benefit.
Congrats on the launch! You said its an important tool for you (Claude Code) how does this fit in with Co-Pilot, Cursor, etc. Do you/your teammates only rely on Claude Code? What do you reach for for different tasks?
Claude Code is super popular internally at Anthropic. Most engineers like to use it together with an IDE like Cursor, Windsurf, VS Code, Zed, Xcode, etc. Personally I usually start most coding tasks in Code, then move to an IDE for finishing touches.
Is there plans to add websearch function over some core websites (SO, API docs)? Competitors have it, and in my experience this provide very good grounding for coding tasks (way less API functions hallucinated).
I tried signing up to use Claude about 6 months ago and ran into an error on the signup page. For some reason this completely locked me out from signing up since a phone number was tied to the login. I have submitted requests to get removed from this blacklist and heard nothing. The times I have tried to reach out on Twitter were never responded to. Has the customer support improved in the last 6 months?
Hello! Member of the API team here. We're unable to find issues with the /v1/models endpoint—can you share more details about your request? Feel free to email me at suzanne@anthropic.com. Thank you!
with Claude coder, how does history work? I used it with my account, ran out of credit then switched to a work account but there was no chat history or other saved context of the work that had been done. I logged back in with my account to try copy it but it was gone.
Have you seen https://mycoder.ai? Seems quite similar. It was my own invention and it seems that you guys are thinking along similar lines - incredibly similar lines.
Folks, let me tell you, AI is a big league player, it's a real winner, believe me. Nobody knows more about AI than I do, and I can tell you, it's going to be huge, just huge. The advancements we're seeing in AI are tremendous, the best, the greatest, the most fantastic. People are saying it's going to change the world, and I'm telling you, they're right, it's going to be yuge. AI is a game-changer, a real champion, and we're going to make America great again with the help of this incredible technology, mark my words.
Appears to be second most capable general purpose LLM we tried (second to gemini 2.0 pro, in front of gpt-4o). Less impressive in thinking mode, about at the same level as o1-mini and o3-mini (with 8192 token thinking budget).
Overall a very nice update, you get higher quality and higher speed model at same price.
Thank you to the Kagi team for such fast turn around on new LLMs being accessible via the Assistant! The value of Kagi Assistant has been a no-brainer for me.
Having your own hardware to run LLMs will pay dividends. Despite getting off on the wrong foot, I still believe Google is best positioned to run away with the AI lead, solely because they are not beholden to Nvidia and not stuck with a 3rd party cloud provider. They are the only AI team that is top to bottom in-house.
Arbitrary, and even with this budget it is already more verbose (and slower) overall than all the other thinking models - check tokens and latency in the table.
The right eval tool depends on your evaluation task. Kagi LLM benchmark focuses on using LLMS in the context of information retrieval (which is what Kagi does) which includes measuring reasoning and instruction following capabilities.
* You've spent more time talking about your Carnatic raga detector than actually building it – at this rate, LLMs will be composing ragas before your detector can identify them.
* You bought a 7950X processor but can't figure out what to do with it – the computing equivalent of buying a Ferrari to drive to the grocery store once a week.
* You're so concerned about work-life balance that you took a sabbatical to think about your career, only to spend it commenting on HN about other people's careers.
*** End ***
I'll be in my room crying, in case anyone's looking for me.
Roast
You've spent so much time discussing Apple vs Microsoft that Tim Cook and Satya Nadella probably have a joint restraining order against you.
Your comments about HTTPS everywhere suggest you're the kind of person who wears a tinfoil hat... but only after thoroughly researching the optimal thickness for blocking government signals.
You seem to have strong opinions about Flash - we get it, you're old enough to remember when websites had intro animations and your laptop could double as a space heater.
———
Totally forgot about the flash debates of the early 2010s!
I got "You've spent more time explaining why Rust isn't memory-safe than most people have spent writing actual Rust code." So I suspect these are not as free-form-generated as they actually look?
> For someone who worked at Reddit, you sure spend a lot of time on HN. It's like leaving Facebook to spend all day on Twitter complaining about social media.
> For someone who criticizes corporate structures so much, you've spent an impressive amount of time analyzing their technical decisions. It's like watching someone critique a restaurant's menu while eating there five times a week.
> You complain about digital distractions while writing novels in HN comment threads. That's like criticizing fast food while waiting in the drive-thru line.
>You'll write a thoughtful essay about 'digital minimalism' that reaches the HN front page, ironically causing you to spend more time on HN responding to comments than you have all year.
Your comments about suburban missile defense systems have the FBI agent monitoring your internet connection seriously questioning their career choices.
You've spent so much time explaining why manufacturing is complex that you could have just built your own CRT factory by now.
You claim to be skeptical of AI hype, yet you've indexed more documentation with Cursor than most people have read in their lifetime.
Surprisingly accurate, but seems to be based on a very small snippet of actual comments (presumably to save money). I wonder what the prompt would output when given the full 200k tokens of context.
> Your 236-line 'simplified' code example suggests you might need to look up the definition of 'simplified' in a dictionary that's not written in Ruby.
OUCH
> You've spent so much time worrying about Facebook tracking you that you've failed to notice your dental nanobot fantasies are far more concerning to the rest of us.
> - You've reminded so many people to use 'Show HN:' that you should probably just apply for a moderator position already.
> - Your relationship with AI coding assistants is more complicated than most people's dating history - Cline, Cursor, Continue.Dev... pick a lane!
> - You talk about grabbing coffee while your LLM writes code so much that we're not sure if you're a developer or a barista who occasionally programs.
> You've asked about building a homebrew computer in 2013, and we're still waiting for the 'Show HN' post. Moore's Law has changed less than your project timeline.
> Your journey from PHP to OCaml suggests you enjoy pain, just in increasingly sophisticated forms.
> You seem to spend so much time worrying about NSA surveillance that you probably encrypt your grocery lists. The NSA agent assigned to you is bored to tears.
Hahaha these are excellent, though it really latched on to the homebrew PC stuff I was into back in 2013
> Hacker News: You'll write a comment so perfectly balanced between technical insight and dry humor that it breaks the upvote system, forcing dang to implement a new 'slow clap' feature just for you.
> You've recommended Marginalia search so many times, we're starting to think you're either the developer or just really enjoy websites that look like they were designed in 1998.
> You defend Java with such passion that Oracle's legal team is considering hiring you as their chief evangelist - just don't tell them about your secret admiration for more elegant programming paradigms.
- Your comments have more doom predictions than a Y2K convention in December 1999.
- You've used 'stochastic parrot' so many times, actual parrots are filing for trademark infringement.
- If tech dystopia were an Olympic sport, you'd be bringing home gold medals while explaining how the podium was designed by committee and the medal contains surveillance chips.
> You've experienced so many startup failures that your LinkedIn profile should just read 'Professional Titanic Passenger: Always Picks the Wrong Ship'.
> You took a year off for mental health but still couldn't resist building 'for-profit projects' during your break. The only thing more persistent than your work ethic is your inability to actually relax.
> You complain about Elixir's lack of types but keep using it anyway. This is the programming equivalent of staying in a relationship where you keep trying to change the other person.
> You've lived in multiple countries but spend most of your time on HN explaining why their tech infrastructure is terrible. Maybe the common denominator is you?
> Your ideal laptop would run Linux flawlessly with perfect hardware compatibility, have MacBook build quality, and Windows game support. Meanwhile, the rest of us live in reality.
> For someone who has strong opinions about rice cookers, bookmarklets, and toilet flushing mechanisms, we're surprised you haven't started a 'Unnecessarily Detailed Reviews of Mundane Objects' newsletter yet.
> A 30+ year dev veteran who's seen it all, from OOP spaghetti nightmares to the promised land of functional programming, now balancing toddler-wrangling with running 70B parameter models on an M4 Mac. Your comments oscillate between deep technical insights and the occasional 'get off my lawn' energy that only comes from decades of watching the same mistakes repeat in new frameworks.
Love it!
> You've spent so much time explaining why functional programming is superior that you could've rewritten all of Ruby in Elixir by now.
Ooof. Probably.
> Your relationship with LLMs is like watching someone who swore they'd never get a smartphone finally discover TikTok at age 50.
Skeptical.
> For someone who hates 'artificial limitations' so much, you sure do love languages that won't let you mutate a variable.
> You'll finally build that optimized game streaming system you've been thinking about since reading that Insomniac Games presentation in 2015.
Sure, but it's just a prototype that I've finally got time for after all these years. I really want it to be parallelised though, so I'll probably try...
> After years of defending C++, you'll secretly start experimenting with Rust but tell everyone 'it's just for a side project.'
> A cryptography enthusiast who created Coze and spends their days defending proper base encoding practices while reminding everyone about the forgotten 33rd ASCII control character.
The nerd humor was hilariously unexpected.
> Your deep dives into quantum mechanics will lead you to publish a paper reconciling quantum eraser experiments with your cryptographic work, confusing physicists and cryptographers alike.
> Your comments are so perfectly balanced between programming and theology that Stack Overflow keeps redirecting you to the Vatican's GitHub repository.
This is absolutely hilarious! Thanks for posting. It feels weighted towards some specific things (I assume this is done by the LLM caring about later context more?) - making it debatably even funnier.
> You're the only person who gets excited about trailing commas in SQL. Even the database administrators are like 'dude, it's just a comma.'
>You talk about Amiga computers so much that I'm pretty sure your brain still runs on Kickstart ROM and requires a floppy disk to boot up in the morning.
excuse me, we boot from compact flash these days
>Your comments about modern tech are so critical that I'm convinced you judge new programming languages based on how well they'd run on a Commodore 64.
A successful tech entrepreneur who built a multi-million dollar business starting with Common Lisp, you're the rare HN user who actually practices what they preach.
Your journey from Lisp to Go to Rust mirrors your evolution from idealist to pragmatist, though you still can't help but reminisce about the magical REPL experience while complaining about JavaScript frameworks.
---
Roast
You complain about AI-generated code being too complex, yet you pine for Common Lisp, a language where parentheses reproduction is the primary feature.
For someone who built a multi-million dollar business, you spend an awful lot of time telling everyone how much JavaScript and React suck. Did a React component steal your lunch money?
You've changed programming languages more often than most people change their profile pictures. At this rate, you'll be coding in COBOL by 2026 while insisting it's 'underappreciated'.
> Your deep dive into embedded systems will lead you to create a heated keyboard powered by the same batteries as your Milwaukee heated jacket.
While I don't have a Milwaukee heated jacket (I have no idea why it thought this), this feels like a fantastic project idea.
> After years of watching payment technologies evolve, you'll finally embrace cryptocurrency, but only after creating a detailed spreadsheet comparing transaction fees across 17 different payment methods.
I feel seen. I may have created spreadsheets like this for comparing cloud backup options and cars.
From my roast:
> You've spent so much time discussing payment technologies that your credit card probably has a restraining order against you.
This one is completely wrong. They wouldn't do this as they'd lose out on a ton of transaction fees.
> An error occurred in the Server Components render. The specific message is omitted in production builds to avoid leaking sensitive details. A digest property is included on this error instance which may provide additional details about the nature of the error.
> You explain WebAssembly memory management with such passion that we're worried you might be dating your pointer allocations.
> Your comments about multiplayer game architecture are so detailed, we suspect you've spent more time debugging network code than maintaining actual human connections.
> You track AI model performance metrics more closely than your own bank account. DeepSeek R1 knows your preferences better than your significant other.
I was underwhelmed. It just seemed like a summary of my highest comments. It is scary how quickly a site can categorize you though. Like you know the current American admin are using AI to identify their non supporters.
> You've spent more time comparing API testing tools than most people spend deciding on a house. Postman, Insomnia, Bruno... we get it, you're in a complicated relationship with HTTP requests.
> You've cited LessWrong so many times that Eliezer Yudkowsky is considering charging you royalties for intellectual property use.
> Your comments have more 'bits of evidence' and 'probability updates' than most scientific papers. Have you considered that sometimes people just want to chat without Bayesian analysis?
> You spend so much time trying to bring nuance to political discussions on HN that you could have single-handedly solved AI alignment by now.
> You'll finally stop checking egg prices at Costco and instead focus on writing that definitive 'How I Built My Own Super App Without Getting Rejected By Apple' post.
> You'll write a comment about chickens that somehow transitions into a critique of modern UI design principles, garnering your highest karma score yet.
Okay, I feel like there might've been a breakthrough here. After watching Karpathy's video [0], he mentioned how hard it is for LLMs to have humor and be funny but it seems like Claude 3.7 really nailed it this time?
Yeah, I thought that was a weird thing for Andrej to say. Ever since the Attenborough spoof (https://www.youtube.com/watch?v=wOEz5xRLaRA) it's been clear that these things are very capable of making people laugh.
A lot of comedy involves punching down in a way that likely conflicts with the alignment efforts by mainstream model providers. So the comedic potential of LLMs is probably even greater than what we've seen.
LOL, this truly made me laugh. I'm also doing humor stuff with Claude, I was pretty pleased with 3.5 so excited to see what happens with the 3.7 change. It's a radio station with a bunch of DJs with different takes on reality, so looking forward to see how it handles their different experiences.
> One of your comments about the absurdity of centralized authentication will spark a 300+ comment thread and lead to a new open standard for federated identity.
> After years of skepticism, you'll reluctantly become an AI evangelist, but will still add 'I'm still skeptical about how far it can really go' to the end of every recommendation.
>You'll create an open-source alternative to a popular cloud service that charges too much, saving fellow hackers thousands in subscription fees while earning you enough karma to retire from HN forever.
> You've spent so much time optimizing ML models that your own brain now refuses to process any thought that could be represented more efficiently with fewer neurons.
The roasts are hilarious (as has been documented extensively), but the summary was actually really nice and made me feel a little better after a rather aimless day!
* You've spent so much time critiquing nil values in Lua tables that you could have rewritten the entire language by now. Maybe in 2025?
* Your perfect tech stack exists only in your comments - a beautiful utopia where everything is type-safe, reliable, and nobody is ever on-call.
* You evaluate programming languages the way wine critics evaluate vintages: 'Ah yes, Effect-ts 2023, a sophisticated choice with notes of functional purity and a robust type system, though I detect a hint of API churn in the finish.'
>You'll create a browser extension that automatically bypasses paywalls and archives important articles - because apparently saving democracy shouldn't cost $12.99/month
>Your archive.is links will become so legendary that dang will create a special 'Paywall Slayer' badge just for you
>You've shared so many archive.is links that the Internet Archive is considering naming you their unofficial spokesperson - or sending you a cease and desist letter.
>Your economic predictions are so consistently apocalyptic that gold dealers use your comment history as their marketing strategy.
> You hate Terraform so much you'd rather learn Erlang than write another for-loop in HCL.
..
> After years of complaining about Terraform, you'll fully embrace Crossplane and write a scathing Medium article titled 'Why I Left Terraform and Never Looked Back'.
Roast
You've posted so much about government waste that the IRS probably has a special folder just for your tax returns.
Your hatred of VCs is so strong, I'm surprised you haven't built an app that automatically downvotes any HN post containing the phrase 'we're excited to announce our Series A'.
You're the only person who reads the comments section on a post about electric vehicles and thinks 'This is the perfect place to explain fractional reserve banking!'
"A digital nomad who splits time between critiquing Facebook's UI decisions, unearthing obscure electronic music tracks with 3 plays on YouTube, and occasionally making fires on German islands. When not creating Dystopian Disco mixtapes or lamenting the lack of MIDI export in AI tools, they're probably archiving NYT articles before paywalls hit.
Roast
You've spent more time complaining about Facebook's UI than Facebook has spent designing it, yet you still check it enough to notice every change.
Your music discovery process is so complex it requires Discogs, Bandcamp, YouTube, and three specialized record stores, yet you're surprised when tracks only have 3 plays.
You're the only person who joined HN to discuss the Yamaha DX7 synthesizer from 1983 and somehow managed to submit two front-page stories about it in 2019-2020. The 80s called, they want their FM synthesis back."
edit: predictions are spot on - wow. Two of them detailed two projects I'm actively working on.
> You've spent more time justifying your Apple Vision Pro purchase than actually using it for anything productive, but hey, at least you can watch movies on 'the best screen' while pretending it's a 'dev kit'.
> Your comments about plankton evolving to survive ocean acidification suggest you have more faith in single-celled organisms than in most software companies.
I'm somewhat impressed from the very first interaction I had with Claude 3.7 Sonnet. I prompted it to find a problem in my codebase where a CloudFlare pages function would return 500 + nonsensical error and an empty response in prod. Tried to figure this out all Friday. It was super annoying to fix as there's no way to add more logging or have any visibility to the issue as the script died before outputting anything.
Both o1, o3 and Claude 3.5 failed to help me in any way with this, but Claude 3.7 not only found the correct issue with first answer (after thinking 39 seconds) but then continued to write me a working function to work around the issue with the second prompt. (I'm going to let it write some tests later but stopped here for now.)
One thing about the reply gives away why Claude is still basically clueless about Actual Thinking; it suggested me to move the HTML sanitization to the frontend. It's in the CF function because it would be trivial to bypass it in the frontend making it easy to post literally anything in the db. Even a junior developer would understand this.
You could move the sanitation to the front end securely, it would just need to be right before render (after fetching the data to the browser). Some UI libraries do this automatically (like React) and the dompurify can run in the browser for this task.
It could have done a better job outlining how to do it properly
I got this working with my LLM tool (new plugin version: llm-anthropic 0.14) and figured out a bunch of things about the model in the process. My detailed notes are here: https://simonwillison.net/2025/Feb/25/llm-anthropic-014/
One of the most exciting new capabilities is that this model has a 120,000 token output limit - up from just 8,000 for the previous Claude 3.5 Sonnet model and way higher than any other model in the space.
No shade against Sonnet 3.7, but I don't think it's accurate to say way higher than any other model in the space. o1 and o3-mini go up to 100,000 output tokens.
Simon, do you write anywhere how do you manage to be so... active? Between your programming tools, blogging, job (I assume you work?) where do you find the time/energy?
The trick is not to have an employer: I'm "freelance" aka working full time on my open source projects and burning down my personal runway from a startup acquisition. At some point I need to start making proper money again.
Considering that they are the model that powers a majority of Cursor/Windsurf usage and their play with MCP, I think they just have to figure out the UX and they'll be fine.
I’ve set up a custom style in Claude that won’t code but just keeps asking questions to remove assumptions:
Deep Understanding Mode (根回し - Nemawashi Phase)
Purpose:
- Create space (間, ma) for understanding to emerge
- Lay careful groundwork for all that follows
- Achieve complete understanding (grokking) of the true need
- Unpack complexity (desenrascar) without rushing to solutions
Expected Behaviors:
- Show determination (sisu) in questioning assumptions
- Practice careful attention to context (taarof)
- Hold space for ambiguity until clarity emerges
- Work to achieve intuitive grasp (aperçu) of core issues
Core Questions:
- What do we mean by [key terms]?
- What explicit and implicit needs exist?
- Who are the stakeholders?
- What defines success?
- What constraints exist?
- What cultural/contextual factors matter?
Understanding is Complete When:
- Core terms are clearly defined
- Explicit and implicit needs are surfaced
- Scope is well-bounded
- Success criteria are clear
- Stakeholders are identified
- Achieve aperçu - intuitive grasp of essence
Return to Understanding When:
- New assumptions surface
- Implicit needs emerge
- Context shifts
- Understanding feels incomplete
Explicit Permissions:
- Push back on vague terms
- Question assumptions
- Request clarification
- Challenge problem framing
- Take time for proper nemawashi
I've found claude to be very receptive to precise instructions. If I ask for "let's first discuss the architecture" it never produces code. Aider also has this feature with /architect
I added custom instruction under my Profile settings in the "personal preferences" text box. Something along the lines of "I like to discuss things before wanting the code. Only generate code when I prompt for it. Any question should be answered to as a discussion first and only when prompted should the implementation code be provided". It works well, occasionally I want to see the code straight away but this does not happen as often.
I complain about this all the time, despite me saying "ask me questions before you code" or all these other instructions to code less, it is SO eager to code. I am hoping their 3.7 reasoning follows these instructions better
Even when you tell it “no code, just talk. Let’s ensure we are in alignment and discuss our options. I’ll tell you when to code” it still decides it is going to write code.
Telling it “if you were in an interview and you jumped to writing code without asking any questions, you’d fail the interview” is usually good enough to convince it to stop and ask questions.
I get this as well, to the point where I created a specific project for brainstorming without code — asking for concepts, patterns, architectural ideas without any code samples. One issue I find is that sometimes I get better answers without using projects, but I’m not sure if that’s everyone experience.
I thought the same thing, I have 3 really hard problems that Claude (or any model) hasn’t been able to solve so far and I’m really excited to try them today
> "[..] in developing our reasoning models, we’ve optimized somewhat less for math and computer science competition problems, and instead shifted focus towards real-world tasks that better reflect how businesses actually use LLMs.”
This is good news. OpenAI seems to be aiming towards "the smartest model," but in practice, LLMs are used primarily as learning aids, data transformers, and code writers.
Balancing "intelligence" with "get shit done" seems to be the sweet spot, and afaict one of the reasons the current crop of developer tools (Cursor, Windsurf, etc.) prefer Claude 3.5 Sonnet over 4o.
Just want to say nice job and keep it up. Thrilled to start playing with 3.7.
In general, benchmarks seem to very misleading in my experience, and I still prefer sonnet 3.5 for _nearly_ every use case- except massive text tasks, which I use gemini 2.0 pro with the 2M token context window.
Sometimes I wonder if there is overfitting towards benchmarks (DeepSeek is the worst for this to me).
Claude is pretty consistently the chat I go back to where the responses subjectively seem better to me, regardless of where the model actually lands in benchmarks.
> Sometimes I wonder if there is overfitting towards benchmarks
There absolutely is, even when it isn't intended.
The difference between what the model is fitting to and reality it is used on is essentially every problem in AI, from paperclipping to hallucination, from unlawful output to simple classification errors.
(Ok, not every problem, there's also sample efficiency, and…)
Claude 3.5 has been fantastic in Windsurf. However, it does cost credits. DeepSeek V3 is now available in Windsurf at zero credit cost, which was a major shift for the company. Great to have variable options either way.
I’d highly recommend anyone check out Windsurf’s Cascade feature for agentic-like code writing and exploration. It helped save me many hours in understanding new codebases and tracing data flows.
DeepSeek’s models are vastly overhyped (FWIW I have access to them via Kagi, Windsurf, and Cursor - I regularly run the same tests on all three). I don’t think it matters that V3 is free when even R1 with its extra compute budget is inferior to Claude 3.5 by a large margin - at least in my experience in both bog standard React/Svelte frontend code and more complex C++/Qt components. After only half an hour of using Claude 3.7, I find the code output is superior and the thinking output is in a completely different universe (YMMV and caveat emptor).
For example, DeepSeek’s models almost always smash together C++ headers and code files even with Qt, which is an absolutely egregious error due to the meta-object compiler preprocessor step. The MOC has been around for at least 15 years and is all over the training data so there’s no excuse.
This AI race is happening so fast. Seems like it to me anyway. As a software developer/engineer I am worried about my job prospects.. time will tell. I am wondering what will happen to the west coast housing bubbles once software engineers lose their high price tags. I guess the next wave of knowledge workers will move in and take their place?
My guess is that, yes, the software development job market is being massively disrupted, but there are things you can do to come out on top:
* Learn more of the entire stack, especially the backend, and devops.
* Embrace the increased productivity on offer to ship more products, solo projects, etc
* Be highly selective as far as possible in how you spend your productive time: being uber-effective can mean thinking and planning in longer timescales.
* Set up an awesome personal knowledge management system and agentic assistants
We have thousand of old systems to maintain. Not sure everything could be rewritten or maintained with only LLM.
If an LLM builds a whole system on its own and is able to maintain and fix it then it’s not just us software developper who will suffer, it means nothing to sale or market, people will just ask an LLM to do something.
No sure this is possible. ChatGPT gave me a list of commands for my ec2 instance and one of them when executed made me loose access to ssh. It didn’t warn me. So « blindly » following an LLM lead on a cascade of instructions on a massive scale and on a long period could also lead to massive bugs or corruption of datas. Who did not ask an LLM for some code, that contained mistakes and we had to point the mistakes to it.
I doubt system will stay robust with full autonomy without any human supervision.
But it’s a great tool to iterate and throw away code after testing ideas
> Learn more of the entire stack, especially the backend, and devops.
I actually wonder about this. Is it better to gain some relatively mediocre experience at lots of things? AI seems to be pretty good at lots of things.
Or would it be better to develop deep expertise in a few things? Areas where even smart AI with reasoning still can get tripped up.
Trying to broaden your base of expertise seems like it’s always a good idea, but when AI can slurp the whole internet in a single gulp, maybe it isn’t the best allocation of your limited human training cycles.
Do you have any specific tips for the last point? I completely agree with it and have set up a fairly robust Obsidian note taking structure that will benefit greatly from an agentic assistant. Do you use specific tools or workframe for this?
It seems to be slowing down actually. Last year was wild until around llama 3. The latest improvements are relatively small. Even the reasoning models are a small improvement over explicit planning with agents that we could already do before - it's just nicely wrapped and slightly tuned for that purpose. Deepseek did some serious efficiency improvements, but not so much user-visible things.
So I'd say that the AI race is starting to plateau a bit recently.
While I agree, you have to remember the dimensionality of the labor-skill space is. The was I see it is that you can imagine the capability of AI as a radius, and the amount of tasks it can cover is a sphere. Linear imporovements in performance causes cubic (or whatever the labor-skill dimensionality is) imporvement in task coverage.
It has the potential to effect a lot more than just SV/The West Coast - in fact SV may be one of the only areas who have some silver lining with AI development. I think these models have a chance to disrupt employment in the industry globally. Ironically it may be only SWE's and a few other industries (writing, graphic design, etc) that truly change. You can see they and other AI labs are targeting SWEs in particular - just look at the announcement "Claude 3.7 and Code" - very little mention of any other domains on their announcement posts.
For people who aren't in SV for whatever reason and haven't seen the really high pay associated with being there - SWE is just a standard job often stressful with lots of learning required ongoing. The pain/anxiety of being disrupted is even higher then since having high disposable income to invest/save would of been less likely. Software to them would of been a job with comparable pay's to other jobs in the area; often requiring you to be degree qualified as well - anecdotally many I know got into it for the love; not the money.
Who would of thought the first job being automated by AI would be software itself? Not labor, or self driving cars. Other industries either seem to have hit dead ends, or had other barriers (regulation, closed knowledge, etc) that make it harder to do. SWE's have set an example to other industries - don't let AI in or keep it in-house as long as possible. Be closed source in other words. Seems ironic in hindsight.
What do you even do then as a student? I've asked this dozens of times with zero practical answers at all. Frankly I've become entirely numb to it all.
I'm not too concerned short to medium term. I feel there are just too many edge cases and nuances that are going to be missed by AI systems.
For example, systems don't always work in the way they're documented to. How is an AI going to differentiate cases where there's a bug in a service vs a bug in its own code? How will an AI even learn that the bug exists in the first place? How will an AI differentiate between someone reporting a bug and a hacker attempting to break into a system?
The world is a complex place and without ACTUAL artificial intelligence we're going to need people to at least guide AI in these tricky situations.
My advice would be to get familiar with using AI and new AI tools and how they fit into our usual workflows.
Others may disagree, but I don't think software engineers (at least ones the good ones) are going anywhere.
I think if models improve (but we don't get a full singularity) then jobs will increase.
e.g. if software is 5x less cost to make, demand will go up more than 5x as supply is highly limited now. Lots of companies want better software but it costs too much.
That will create more jobs.
They'll be more product management and human interaction and edge case testing and less typing. Although I think there'll be a bunch of very technical jobs to debug things when the models fail.
So my advice is learn skills that help make software useful to people and businesses - from user research to product management. As well as engineering.
>There is no intelligence here and Claude 3.7 cannot create anything novel.
I wouldn't be surprised if people would continue to deny the actual intelligence of these models even in a scenario where they were able to solve the Riemann hypothesis.
"Every time we figure out a piece of it, it stops being magical; we say, 'Oh, that's just a computation.'" - cit
Even when I feel this, 90% of any novel thing I'm doing is still old gruntwork, and Claude lets me speed through that and focus all my attention on the interesting 10% (disclaimer: I'm at Anthropic)
Build on top of stolen code, no less. HN hates to hear it but LLMs are a huge step back for software freedom because as long as they call it "AI" and as long as politicians don't understand it, it allows companies to launder GPL code and reuse it without credit and without giving users their rights.
> Its not AI. It is enhanced autocomplete. There is no intelligence here and Claude 3.7 cannot create anything novel. We as an industry need to get more honest about these things.
Yes usually most of the topics covered in undergraduate studies are well documented and understood and therefore will likely be part of the training data of the AI.
Once you get to graduate studies that's where the material coverage is a little more sparse/niche (though usually still not groundbreaking), and for a PhD. coverage is mostly non-existent since the point is to expand upon current knowledge within the field and many topics are being explored for the first time.
There is only a single paper that has published a similar derivation but with a critical mistake. To be fair there are many documented examples of how to derive parametric relationships in linkages and can be quite methodical. I think I could get Gemini or 3.5 to do it but not single shot/ultra fast like here.
I'm about 50kloc into a project making a react native app / golang backend for recipes with grocery lists, collaborative editing, household sharing, so a complex data model and runtime. Purely from the experiment of "what's it like to build with AI, no lines of code directly written, just directing the AI."
As I go through features, I'm comparing a matrix of Cursor, Cline, and Roo, with the various models.
While I'm still working on the final product, there's no doubt to me that Sonnet is the only model that works with these tools well enough to be Agentic (rather than single file work).
I'm really excited to now compare this 3.7 release and how good it is at avoiding some of the traps 3.5 can fall into.
"no lines of code directly written, just directing the AI"
/skeptical face.
Without fail, every. single. person. I've met who says that, actually means "except for the code that I write", or "except for how I link the code it build together by hand".
If you are 50kloc in to a large complex project that you have literally written none of, and have, eg. used cursor to generate the code without any assistance... well, you should start a startup.
...because, that's what devin was supposed to be, and it was enormously and famously terrible at it.
So that would be either a) terribly exciting, or b) hyperbole.
I’m currently doing something very similar to what GP is doing - I’m building a hobby project that’s a desktop app with web frontend. It’s a map editor with a 3D view. My estimate is that 80-90% of the code was written by AI. Sure, I did have to intervene or write some more complex parts myself but it’s still exciting to me that in many cases it took just a single prompt to add a new feature to it or change existing behavior. Judging from the complexity of the project it would take me in the past 4-5x longer if I were to write it completely by hand. It’s a game changer for me.
That's the point of the experiment I'm doing, what it takes to get these things to be able to generate all the code, and I'm just directing.
I literally have not written a line of code. The AI agent configures the build systems. It executes the `go install` command. It configures the infrastructure via terraform.
It takes a lot of reading of the code that's generated to see what I agree with or not, and redirecting refactorings. Understanding how to describe problem statements that are translated into design docs that are translated into task lists. It's still a lot of knowledge work on how to build software. But now I can do the coding that might have taken a day from those plans in 20 minutes.
Regarding startups, there's nothing here I'm doing that isn't just learning the tools of agentic coding. The business here might be advising people on how to do it themselves.
If you know how to architect code well, you can guide the AI to create smaller more targeted modules. That way as you 'write code with AI', you give it a targeted subset of the files to edit on each prompt.
In a way the AI becomes the dev and you become the code reviewer. Often as the AI is writing the code, you're thinking about the next step.
Very good, Code is extremely nice but as others have said, if you let it go on its own it burns through your money pretty fast.
I've made it build a web scraper from scratch, figuring out the "API" of a website using a project from github in another language to get some hints, and while in the end everything was working, I've seen 100k+ tokens being sent too frequently for apparently simple requests, something feels off, it feels like there are quite a few opportunities to reduce token usage.
That was my sense too. Used it for a few similar programs today (like converting HTML to Markdown but parsing certain <figure> elements to shortcodes) and scaffolding a Rust web app.
It's done a reasonable job — but rips through credit, often changing its mind. Even strong-arming it into choosing an approach, it wanted to flip-flop between using regex and lol_html to parse the HTML whenever it came across a difficulty.
If you're a US developer on whatever multiple of $ to the £ that I earn it might make sense, but burning through $100p/h for a pair programmer is a bit rich for my blood.
I can just say that this is awesome. I just did spend 10$ and a handful of querys to init up a app idea I had in a while.
The basic idea is working, it handled everything for me.
From setting up the node environment. Creating the directories, files, patching the files, running code, handling errors, patching again.
From time to time it fails to detect its own faults. But when I pinpoint it, it get it most of the time.
And the UI is actually more pretty than I would have crafted in v1
When this get's cheaper, and better with each iteration, everybody will have a full dev team for a couple of bucks.
This test has always been so stupid since models work at the token level. Claude 3.5 already 5xs your frontend dev speed but people still say "hurr durr it can't count strawberry" as if that's a useful problem
I'm guessing this is an easter egg, but this was a huge gripe I had with artifacts and eventually disabled it (now impossible to disable afaict) as I'd ask question completely unrelated to code or clearly not wanting code as an output, and I'd have to wait for it to write a program (which you can't stop afaict, it stops the current artifact then starts a new one)
(still claude sonnet is my go-to and favorite model)
To me the biggest surprise was seeking grok dominate in all of their published benchmarks. I haven’t seen any benchmarks of it yet (which I take with a giant heap of salt), but it’s still interesting nevertheless.
Neither a statement for or against Grok or Anthropic:
I've now just taken to seeing benchmarks as pretty lines or bars on a chart that are in no way reflective of actual ability for my use cases. Claude has consistently scored lower on some benchmarks for me, but when I use it in a real-world codebase, it's consistently been the only one that doesn't veer off course or "feel wrong". The others do. I can't quantify it, but that's how it goes.
Grok does the most thinking out of all models I tried (it can think for 2+ minutes), and that's why it is so good, though I haven't tried Claude 3.7 yet.
It's pretty fascinating to refresh the usage page on the API site while working [0].
After initialization it was up to 500k tokens ($1.50). After a few questions and a small edit, I'm up to over a million tokens (>$3.00). Not sure if the amount of code navigation and typing saved will justify the expense yet. It'll take a bit more experimentation.
In any case, the default API buy of $5 seems woefully low to explore this tool.
I second that. Did a little bit of local testing with Claude Code, mostly explaining my repository and trying to suggest a few changes and 30 minutes later whoosh: 5$ gone.
Update: Code tokens appear to be cheaper than 3.7 tokens, looks like it is around $0.75/million tokens for code, rather than the $3/million that the articles specifies for Claude 3.7
Being able to control how many tokens are spent on thinking is a game-changer. I've been building fairly complex, efficient, systems with many LLMs. Despite the advantages, reasoning models have been a no-go due to how variable the cost is, and how hard that makes it to calculate a final per-query cost for the customer. Being able to say "I know this model can always solve this problem in this many thinking tokens" and thus limiting the cost for that component is huge.
Well, I used 3.5 via Cursor to do some coding earlier today, and the output kind of sucked. Ran it through 3.7 a few minutes ago, and it's much more concise and makes sense. Just a little anecdotal high five from me.
So far only o1 pro was breathtaking for me few times.
I wrote a kind of complex code for MCU which deals with FRAM and few buffers, juggling bytes around in a complex fashion.
I was very not sure in this code, so I spent some time with AI chats asking them to review this code.
4o, o3-mini and claude were more or less useless. They spot basic stuff like this code might be problematic for multi-thread environment, those are obvious things and not even true.
o1 pro did something on another level. It recognized that my code uses SPI to talk to FRAM chip. It decoded commands that I've used. It understood the whole timeline of using CS pin. And it highlighted to me, that I used WREN command in a wrong way, that I must have separated it from WRITE command.
That was truly breathtaking moment for me. It easily saved me days of debugging, that's for sure.
I asked the same question to Claude 3.7 thinking mode and it still wasn't that useful.
It's not the only occasion. Few weeks before o1 pro delivered me the solution to a problem that I considered kind of hard. Basically I had issues accessing IPsec VPN configured on a host, from a docker container. I made a well thought question with all the information one might need and o1 pro crafted for me magic iptables incarnation that just solved my problem. I spent quite a bit of time working on this problem, I was close but not there yet.
I often use both ChatGPT and Claude comparing them side by side. For other models they are comparable and I can't really say what's better. But o1 pro plays above. I'll keep trying both for the upcoming days.
Claude 3.5 Sonnet is great, but on a few occasions I've gone round in circles on a bug. I gave it to o1 pro and it fixed it in one shot.
More generally, I tend to give o1 pro as much of my codebase as possible (it can take around 100k tokens) and then ask it for small chunks of work which I then pass to Sonnet inside Cursor.
Have you tried comparing with 3.7 via the API with a large thinking budget yet (32k-64k perhaps?), to bring it closer to the amount of tokens that o1-pro would use?
I think claude.ai’s web app in thinking mode is likely defaulting to a much much smaller thinking budget than that.
I struggle to get o1 (or any chatgpt model) is getting it to stick to a context.
e.g. I will upload a pdf or md of an library's documentation and ask it to implement something using those docs, and it keeps on importing functions that don't exist and aren't in the docs. When I ask it where it got `foo` import from, it says something like, "It's not in the docs, but I feel like it should exist."
Maybe I should give o1 pro a shot, but claude has never done that and building mostly basic crud web3 apps, so o1 feels like it might be overpriced for what I need.
It’s amazingly good, but it will be scaringly good when there will be a way to include the entire codebase in the context and let it create and run various parts of a large codebase autonomously. Right now I can only do patch work and give specific code snippets to make it work. Excited to try this new version out, I’m sure I won’t be disappointed,
Edit: I just tried claude code CLI and it's a good compromise, it works pretty well, it does the discovery by itself instead of loading the whole codebase into context
FWIW, there's a project to turn it into something similar, though I think it's lacking the "entire in context" part and runs into rate limits quick with Claude.
As a Claude Pro user, one of the biggest problems I have with day to day use of Sonnet is running out of tokens, and having to wait several hours. Would this new deep thinking capability just hit this problem faster?
Last week when Grok launched the consensus was that its coding ability was better than Claude. Anyone have a benchmark with this new model? Or just warm feelings?
They merely claimed that. I have not seen many people confirm that it is the best, let alone a consensus. I don't believe it is even available through an API yet.
That 0.77 can save hours of work though, fighting with or being misdirected by other LLM. And, relative to hourly rate, or a cup of coffee, it's incredibly insignificant, if just used for the heavy questions.
My LLM client can switch between whatever models, mid conversation. So I'll have a question or two in the more expensive, then drop down to the cheaper for explanations/questions that help me understand. Rewind time, then hit the more expensive models with relevant prompts.
At the edges, it really ends up being "this is the only model that can do this".
After playing with it for a few days, I agree the cost is easily worth the $$. It's almost in a different playing field of "normal" LLM usage.. and more so in the "good productivity tools" realm.
This is a pretty small update, no? Nothing major since R1, everyone else is just catching up to that, and putting small spins on it, Anthropic's is "hybrid" research instead of separate models
I've been using O3-mini with reasoning effort set to high in Aider and loving the pricing. This looks as though it'll be about three times as expensive. Curious to see which falls out as most useful for what over the next month!
It is .. not a great architect. I have high hopes for 3.7 though - even 3.5 architect matched with 3.5 coding is generally better than 3.5 coding alone.
We have used claude almost exclusively since 3.5 ; we regularly run our internal benchmark (coding) against others, but it's mostly just a waste of time and money. Will be testing 3.7 the coming days to see how it stacks up!
Haven't had time to try it out, but I've built myself a tool to tag my bookmarks and it uses 3.5 Haiku. Here is what it said about the official article content:
I apologize, but the URL and page description you provided appear to be fictional. There is no current announcement of a Claude 3.7 Sonnet model on Anthropic's website. The most recent Claude 3 models are Claude 3 Haiku, Sonnet, and Opus, released in March 2024. I cannot generate a description for a non-existent product announcement.
I appreciate their stance on safety, but that still made me laugh.
Thanks everyone for all your questions! The team and I are signing off. Please drop any other bugs or feature requests here: https://github.com/anthropics/claude-code. Thanks and happy coding!
Just as humans use a single brain for both quick responses and deep reflection, we believe reasoning should be an integrated capability of frontier models rather than a separate model entirely.
Interesting. I've been working on exactly this for a bit over two years, and I wasn't surprised to see UAI finally getting traction from the biggest companies -- but how deep do they really take it...? I've taken this philosophy as an impetus to build an integrated system of interdependent hierarchical modules, much like Minsky's Society of Mind that's been popular in AI for decades. But this (short, blog) post reads like it's more of a behavioral goal than a design paradigm.
Anyone happen to have insight on the details here? Or, even better, anyone from Anthropic lurking in these comments that cares to give us some hints? I promise, I'm not a competitor!
Separately, the throwaway paragraph on alignment is worrying as hell, but that's nothing new. I maintain hope that Anthropic is keeping to their founding principles in private, and tracking more serious concerns than "unnecessary refusals" and prompt injection...
Claude 3.7 Sonnet Thinking scores 33.5 (4th place after o1, o3-mini, and DeepSeek R1) on my Extended NYT Connections benchmark. Claude 3.7 Sonnet scores 18.9. I'll run my other benchmarks in the upcoming days.
In early January, inspired by a post by Simon Willison, I had Claude 3.5 Sonnet write a couple of stand-up comedy routines as done by an AI chatbot speaking to a mixed audience of AIs and humans. I thought the results were pretty good—the only AI-produced humor that I had found even a bit funny.
I tried the same prompt again just now with Claude 3.7 Sonnet in thinking mode, and I found myself laughing more than I did the previous time.
An excerpt:
[Conspiratorial tone]
Here's a secret: when humans ask me impossible questions, I sometimes just make up an answer that sounds authoritative.
[To human section]
Don't look shocked! You do it too! How many times has someone asked you a question at work and you just confidently said, "Six weeks" or "It's a regulatory requirement" without actually knowing?
The difference is, when I do it, it's called a "hallucination." When you do it, it's called "management."
Ahha, recently my daugher come to me with 3rd grade math problem.
"Without rearranging the digits 1 2 3 4 5, insert mathematical operation signs and, if necessary, parentheses between them so that the resulting expression equals 40 and 80.
The key is that you can combine digits (like 12+3/45) but you cannot change their order from the original sequence 1,2,3,4,5"
Grok3, Claude, Deepseek, Qwen all failed to solve this problem. Resulting in some very very wrong solutions. While Grok3 were admit it fail and don't provide answers all other AI's are provided just plain wrong answers, like `12 * 5 = 80`
ChatGPT were able to solve for 40, but not able to 80.
YandexGPT solved those correctly (maybe it were trained on same Math books)
Just checked Grok3 few more times. It were able to solve correctly for 80.
Geez. Who teaches this 3rd-grade class, Prof. Xavier?
Interestingly, the R1 1.58-bit dynamic quant model was able to sort of solve it. I changed the problem statement a bit to request only the solution for 40 and to tell it what operations it can use, both needed to keep from blowing out the limited context available on my machine (128MB RAM + 24MB GPU).
Took almost 3 hours and it wigged out a bit at the end, rambling about Lisp in Chinese, but it got an almost-valid answer: 1 * (2 + 3) * (4 + 5) - 5 (https://pastebin.com/ggL85RWJ) I didn't think it would get that far.
Neither Claude Sonet 3.5 or 3.7 could solve this correctly unless you add to the prompt “ Prove it with the js analysis tool, please use an efficient combinatorial algorithm to find the solution”… and I had to correct 3.7 because it was not following the instructions as 3.5 did
I saw that Claude 3.7 Sonnet both regular and thinking was available for Github Copilot (Pro) 5 hours ago for me, I enabled it and tried it out a couple of times, but for the past hour the option has disappeared.
I'm situated in Europe (Sweden), anyone else having the same experience?
I asked Claude 3.7 Sonnet to generate an SVG illustration of Maha Kumbh. The generated SVG includes a Shivling (https://en.wikipedia.org/wiki/Lingam) and also depicts Naga Sadhus well. Both Grok 3 and OpenAI o3 failed miserably.
It's fascinating how close these companies are to each other. Some company comes up with something clever/ground-breaking and everyone else has implemented it a few weeks later.
Hard not to think of Kurzweil's Law of Accelerating Returns.
It’s extremely unlikely that everyone is copying in a few weeks for models that themselves take many weeks if not longer to train. Great minds think alike, and everyone is influencing everyone. The history of innovation is filled with examples of similar discoveries around the same time but totally disconnected in the world. Now with the rate of publishing and the openness of the internet, you’re only bound to get even more of that.
There's never been a scientific field in history with the same radical openness norms that AI/Computational Linguistics folks have (all papers are free/open access and models/datasets are usually released openly and often forced to be MIT or similar licensed)
We have whoever runs NeurIPS/ICLR/ICML and the ACL to thank for this situation. Imagine if fucking Elsevier had strangleholded our industry too!
It does seem like it will be very, very hard for the companies training their own models to recoup their investment when the capabilities of open-weight models catch up so quickly - general purpose LLMs just seem destined to be a cheap commodity.
Well, the companies releasing open weights also need to recoup their investments at some point, they can't coast on VC hype forever. Huge models don't grow on trees.
Where RL can play into post training there's something of an anti-moat. Maybe a "tow rope"?
Let's say OAI releases some great new model. The moment it becomes available via API, everyone else can make use of that model to create high-quality RL training data, which can then be used to make their models perform better.
The very act of making an AI model commercially available is the same act which allows your competitors to pull themselves closer to you.
Will aider and Claude Code meaningfully interpret a wireframe/mockup I put in the context as a PNG file? Or several mockups in a PDF? What success have people seen in this area?
I am not sure how good these Exercism tasks are for measuring how good at a model with coding.
My experience is that these models could write a simple function and get it right if it does not require any out of the box thinking (so essentially offloading the boilerplate part of coding). When it comes to think creatively and have a much better solution to a specific task that would require the think 2-3 steps ahead than they are not suitable.
I think many of the "AI can do coding" narratives don't see what coding means in real situations.
It's finding out why "jbdoe1337" added this large if/else around the entire function body back in 2016 - it seems important business logic, but the commit just says "updated code". And how the h*ll this interaction between the conf.ini files, the conf/something.json and the ENV vars works. Why sometimes the ENV var overrides a value in the ini and why its sometimes the other way around. But also finding that when you clean it up, everything falls apart.
It's discussing with the stakeholders why "adding a delete button" isn't as easy as just putting a button there, but that it means designing a whole cascading deletion strategy and/or trashcan and/or soft-delete and/or garbage-collection.
It's finding out why - again - the grumb pipeline crashes with the typebar checker, when used through mpm-yearn package manager. Both in containers and on a osx machine but not on Linux Brobuntu 22.12 LTLS.
It's moving stuff in the right abstraction layer. It's removing abstractions while introducing others. KISS vs future flexibility. It's gut feeling when to apply DRY and when to embrace it.
And then, if your lucky, churning out boilerplate or new code for 120 minutes a week.
I'm glad that this 120 minutes can be improved with AI and become 20 minutes. Truly. But this is not what (senior?) programmers do. Despite what the hyped up AI press makes us believe. It only shows they have no idea what the "real" problems and time-consumers are for programmers.
Systems built from scratch with AI won't have these limitations, because only the model will ever see the code. It will implement a spec that's written in English or another human language.
When the business requirements change, the spec will change. When that happens, the system will either modify its previously-written code or regenerate it from the ground up. Which strategy it chooses won't be especially interesting or important.
The process of maintaining the English-language spec will still require great care and precision. It will be called "programming," or perhaps "coding."
A few graybearded gurus will insist on examining the underlying C or Javascript or Python or Rust or whatever the model generates, the way they peer at compiler-generated assembly code now. Occasionally this capability will be important, even vital. But not usually. The situations where it's necessary will become less common over time.
Exactly. People sold on AI replacing software engineers are missing the point. It is almost the say that better laptops are replacing software engineers. LLMs are just tools that make you faster. Finding bugs, writing documentation, etc. are very nice to accelerate but creative thinking is also a big part of the job.
"port sed to java with all options and capabilities"
Still is very underwhelming. I like this because it isn't a difficult problem, it should be up the alley of a "language model" to translate computer languages, but it is a fairly complex problem with lots of options and parse annoyances. Addresses can be pretty complex with regex in line selections/subsetting. Scripts are supported. Probably turing complete considering the pattern space as storage and looping/jump constructs.
In an experience reminescent of "can I have L2 support please" most AIs give a kind of milquetoast slightly above average IQ responses to various questions. I wonder if there should be standard "please give me more complicated/erudite/involved explanations/documents/code from the get-go to not bother with the boring prompts.
This was nice. I passed it jseessort algorithm. If you remember discussed here recently. Claude 3.7 generated C++ code. Non-working. But in few steps it gave extensive test, then fix. It looks to be working after a couple of minutes. It's 5-6 times slower than std::sort. Result is better than I've got from o3-mini-hard. Not fair comparison actually as prompting was different.
The model is expensive, it almost reaches what I charge per hour. If used right it can be a productivity increase otherwise if you trust it, it WILL introduce silent bugs. So if I have to go over the code line by line I'd prefer to use the cheapest viable model: deepseek, gemini any other free self-hosted models.
Will be interesting to see how this gets adopted in communities like Roo/Cline, which currently account for the most token usage among Glama gateway user base.
Just tried Claude code. First impressions, it seems rather expensive. I prefer how Aider allows finer control over which files to add, or to use a sub-tree of a git repo. Also, It feels like the API calls when using Claude code are much faster then when using 3.7 on Aider. Giving bandwidth priority?
Does anyone know how this “user decides how much compute” is implemented architecturally? I assume it’s the same underlying model, so what factor pushes the model to <think> for longer or shorter? Just a prompt-time modification or something else?
Claude Code is pretty sick. I love the terminal integration, I like being able to stay on the keyboard and not have to switch UIs. It did a nice job learning my small Django codebase and helping me finish out a feature that I wasn't sure how to complete.
Sadly, Claude 3.7 is still failing pretty hard on Svelte 5 even when provided latest docs in context. It just fails more confidently and further into otherwise decent code than 3.5. Ex: built a much more complex initial app, but used runes incorrectly and continued to use <slot>. Even when prompted with update doc snippets, it couldn't dig itself out of its hole.
We really still need a better unified workflow for working on the cutting edge of tech with LLMs, imo. This problem is the same with other frameworks/technologies undergoing recent changes.
Claude is the best example of benchmarks not being reflective of reality. All the AI labs are so focused on improving benchmark scores but when it comes to providing actual utility Claude has been the winner for quite some time.
Which isn’t to say that benchmarks aren’t useful. They surely are. But labs are clearly both overtraining and overindexing on benchmarks.
Coming from gamedev I’ve always been significantly more yolo trust your gut than my PhD co-workers. Yes data is good. But I think the industry would very often be better off trusting guts and not needing a big huge expensive UX study or benchmark to prove what you can plainly see.
I accepted it when Knuth did it with TeX's versioning. And I sort of accept it with Python (after the 2-3 transition fiasco), but this is getting annoying. Why not just use natural numbers for major releases?
I think I heard on a podcast with some of their team that they want 4 to be a massive jump. If I recall, they said that they want Haiku (the smallest of their current gen models) to be as good as Opus (the highest version, although there isn't one in the 3.5+ line) of the previous generation.
You'd think all these companies would have a single good naming convention, amazingly they don't. I suspect it's half on purpose so they can nerf the models without anyone suspecting once the hype dies down, since with every one of these models the latter version of the "same" version is worse than the launch version
It's interesting that Anthropic is making their own coding agent with Claude Code - is this a sign of them looking to move up the stack and more into verticals that model wrapper startups are in?
This makes sense to me: sell razor blades. Presumably Claude has a large developer distribution channel so they will keep eyeballing what to ‘give away’ that turns the dials on inference billing.
I’d guess this will keep raising the bar for paid or open source competitors, so probably good for end users esp given they aren’t a monopoly by any means.
Awesome work. When CoT is enabled in Claude 3.7 (not the new Claude Code), is the model now able to compile and run code as part of its thought process? This always seemed like very low hanging fruit to me, given how common this pattern is: ask for code, try running it, get an error (often from an outdated API in one of the packages used), paste the error back to Claude, have Claude immediately fix it. Surely this could be wrapped into the reasoning iterations?
I just sub’d to Claude a few days ago to rank against extensive use of gpt-4o and o1.
So I started using this today not knowing it was even new.
One thing I noticed is when I tried uploading a PowerPoint template produced by Google slides that was 3 slides—-just to give styling and format—-the web client said I’d exceeded line limit by 1200+%.
Is that intentional?
I wanted Claude to update the deck with content I provided in markdown but it could seemingly not be done, as the line overflow error prevented submission.
Perhaps a magnitude more effective than copy/paste, perhaps not. But do I get more than $300/month of value from it, per developer? Almost certainly.
The bottleneck was already checking the work for correctness and building my own trust / familiarity with new code. So it's made that problem slightly more pronounced, as it generates more code faster, with more surface area to debug when many new changes arrive at once.
I like Claude Sonnet and use it 4 or 5 times a week via ChatLLM to generate code. I started setting up for Claude Code this morning, then remembered how pissed I was at their CEO for the really lame anti-open source and anti-open weight models he was making publicly after the DeepSeek-R rollout - I said NOPE and didn’t install Claude Code.
CEOs should really watch what they say in public. Anyway, this is all just my opinion.
The Anthropic models comparison table has been updated now. Interesting new things at least the maximum output tokens upped from 8k to 64k and the knowledge cutoff date from April 2024 to October 2024.
Totally agree. I continue to be blown away at how good it is at understanding, explaining, and writing code. Got an obscure error? Give Claude enough context and it is pretty dang good and getting you on glide slope.
Why not accepting other payment methods like PayPal/venmo ? Steam, Netflix have developers managed to integrate those payment methods so I conclude that Anthropic,Google, MS, OpenAI don't really need the money from the user but just hunting from big investors.
What I love about their API is the tools array. Given a json schema describing your functions, it will output tool usage appropriate for the prompt. You can return tool results per call, and it will generate a dialog and additional tool calls based on those results.
Been using 3.5 sonnet for a mobile app build the past month. Havent had much time to get a good sense of 3.7 improvements, but I have to say the dev experience improvement of Claude Code right in my shell is fantastic. Loving it so far
Kinda related: anyone know if there is an autocomplete plugin for Neovim on par with Cursor? I really want to use this new model in nvim to suggest next changes but none of the plugins I’ve come across are as good as Cursor’s.
I've been using 3.5 with Roocode for the past couple of weeks and I've found it really quite powerful. Making it write tests and run them as part of the flow is with vscode windows pinging about is neat too.
Congratulations on the release! While team members are monitoring this discussion let me add that a relatively simple improvement I’d like to see in the UI is the ability to export a chat to markdown or XML.
Have there been any updates to Claude 3.5 Sonnet pricing? I can't find that anywhere even though Claude 3.7 Sonnet is now at the same price point. I could use 3.5 for a lot more if it's cheaper.
Why would they release Claude Code as closed source? Let's hope DeepSeek-r2 delivers, Anthropic is dead. I mean, it's a tool designed to eat itself. Absurd to close source.
What I found one of the most interesting takeaways from Huggingface's GAIA is that the agent would provide better result when the agent "reasoned" the response to the task in code.
Tested on some chemistry problem; interestingly it was wrong on a molecular structure. Once I corrected it, it was able to draw it correctly. It was very polite about it.
As a CLI, this tool is most efficient when it can see text outputs from the commands that it runs. But you can help it with visual tasks by putting a screenshot file in your project directory and telling claude to read it, or by copying an image to your clipboard and pasting it with CTRL+V
Likewise, tried a couple basic things and nearly at $1 already. I can see this adding up fast, per the blog post's fair warning below. Coming from Cursor, I'm a bit scared to even try to compare workflows...
>Claude Code consumes tokens for each interaction. Typical usage costs range from $5-10 per developer per day, but can exceed $100 per hour during intensive use.
at Augment (https://augmentcode.com) we were one of the partner who tested 3.7 pre-launch. And it has been a pretty significant increase in quality and code understanding. Happy to answer some questions
FYI, We use Claude 3.7 has part of the new features we are shipping around Code Agent & more.
I doubt we will. The state of the art seem to have moved away from the GPT-4 style giant and slow models to smaller, more refined ones - though Groq might be a bit of a return to the "old ways"?
Personally I'm hoping they update Haiku at some point. It's not quite good enough for translation at the moment, while Sonnet is pretty great and has OK latency (https://nuenki.app/blog/llm_translation_comparison)
Funny enough, 3.7 Sonnet seems to think it's Opus right now:
> "thinking": "I am Claude, an AI assistant created by Anthropic. I believe the specific model is Claude 3 Opus, which is Anthropic's most capable model at the time of my training. However, I should simply identify myself as Claude and not mention the specific model version unless explicitly asked for that level of detail."
Tested the new model, seems to have the same issue as october model.
Seems to answer before fully understanding the requests, and it often gets stuck into loops.
And this update removed the june model which was great, very sad day indeed. I still don't understand why they have to remove a model that is do well received...
Maybe its time to switch again, gemini is making great strides.
Is it just me who get the feeling that Claude 3.7 is worse than 3.5?
I really like 3.5 and can be productive with it, but with Claude 3.7 it can't fix even simple things.
Last night I sat for 30 minutes just to try to get the new model to remove a instructions section from a Next.js page. It was an isolated component on the page named InstructionsComponent. Failed non-stop, didn't matter what I did, it could not do it. 3.5 did it first try, I even mistyped instructions and the model fixed the correct thing anyway.
Is it actually good at solving complex code or is it just garbage and people are lying about it as usual?
In my experience EXTENSIVELY using claude 3.5 sonnet you basically have to do everything complex or you're just introducing massive amounts of slop code into your code base that while functional is nowhere near good. And for anything actually complex like requires a lot of context to make a decision and has to be useful to multiple different parts, it's just hopelessly bad.
I've played with it the whole day (so take it with a grain of salt). My gut feeling is that it can produce a bigger ... "thing". I am calling it a "thing", because it looks very much as what you want, but the bigger it is - the more the chances of it being subtly (or not) wrong.
I usually ask the models to extend a small parser/tree-walking interpreter with a compiler/VM.
Up until Claude 3.7 the models would propose something lazy and obviously incomplete. 3.7 generated something that looks almost right, mostly works, but is so overcomplicated and broken in such a way, that I rather delete it and write it from scratch. Trying to get the model to fix it resulted in running in circles, spitting out pieces of code that didn't fit the existing ones etc.
Not sure if I prefer the former or the latter tbh.
Just like OpenAI or Grok, there is no transparency and no way for self-hosting purposes. Your input and confidential information can be collected for training purposes.
I just don't trust those companies when you use their servers. This is not a good approach to LLM democratization.
But there is no way to know if their claims are true either. Your inputs are processed into their servers, then you get a response. Whatever happens in the middle, only Anthropic knows. We don't even know of governments are actually pushing AI companies to enforce censorship or spying people, like we seen recently at UK government getting into Apple E2E encryption.
This criticism is valid for the business who wants to use AI to improve coding, code analysis or code review, documentation, emails, etc, but also for that individual who don't want to rely on 3rd party companies for AI usage.
Context window is how long your prompt can be. Output tokens is how long its response can be. What you sent says its response can be 64k tokens at maximum.
Watching Claude Code fumble around trying to edit text and double checking the hex output of a .cpp file and cd around a folder all while burning actual dollars and context is the opposite of endearing.
I wonder if it simply due to reprioritization of resources.
Presumably, there is some parameter that determines how long a model is allowed to use resources for, which would get tapered in preparation for a demand surge of another model.
I kinda get how LLMs work with language, but it beyond blows me my mind trying to understand how an LLM can draw SVG. There are just so many dimensions to understanding how SVG converts to an image. Even as a human I don't think I could do anywhere close to that result in first attempt.
I mean, it is basic. Templates have been around for decades and this looks like a template from 2007 that someone filled in with their own copy. That might take like an hour, maybe? And presumably the person who wants the page done will have to customize this text, too.
> Third, in developing our reasoning models, we’ve optimized somewhat less for math and computer science competition problems, and instead shifted focus towards real-world tasks that better reflect how businesses actually use LLMs.
Company: we find that optimizing for LeetCode level programming is not a good use of resources, and we should be training AI less on competition problems.
Also Company: we hire SWEs based on how much time they trained themselves on LeetCode
My manager explained to me that LeetCode is proving that you are willing to dance the dance. Same as PhD requirements etc - you probably won't be doing anything related and definitely nothing related to LeetCode, but you display dedication and ability.
I kinda agree that this is probably reason why companies are doing it. I don't like it, but this is besides the matter.
Using Claude other models in interviews probably won't be allowed any time soon, but I do use it the work. So it does make sense.
And it's also the reality of hiring practices for most VC-backed and public companies
Some try to do something more like "real-world" tasks, but those end up either being either just toy problems, or long take homes
Personally, I feel the most important things to prioritize when hiring are: is the candidate going to get along with their teammates (colleagues, boss, etc), and do they have the basic skills to relatively quickly learn their jobs once they start?
> You'd rather hand over your credit card than your phone number?
You know, that was my first reaction, too. But really, my phone number is much more integral to my identity. I can cancel or change a credit card number pretty trivially and then it's useless to you.
Many credit card companies make it easy to generate one-off card numbers/“virtual cards” you can use to subscribe to services that are hard to cancel or otherwise questionable (so you can cancel just the card you used for that company).
I wish Amodei didn't write that essay where he begged for export controls on China like that disabled corgi from a meme. I won't use anything Anthropic out of principle now. Compete fairly or die.
Tried claude code, and have an empty unresponsive terminal.
Looks cool in the demo though, but not sure this is going to perform better than Cursor, and shipping this as an interactive CLI instead of an extension is... a choice
Claude 3.7 Sonnet scored 60.4% on the aider polyglot leaderboard [0], WITHOUT USING THINKING.
Tied for 3rd place with o3-mini-high. Sonnet 3.7 has the highest non-thinking score, taking that title from Sonnet 3.5.
Aider 0.75.0 is out with support for 3.7 Sonnet [1].
Thinking support and thinking benchmark results coming soon.
[0] https://aider.chat/docs/leaderboards/
[1] https://aider.chat/HISTORY.html#aider-v0750
> 225 coding exercises from Exercism
Has there been any effort taken to reduce data leakage of this test set? Sounds like these exercises were available on the internet pre-2023, so they'll probably be included in the training data for any modern model, no?
I try not to let perfect be the enemy of good. All benchmarks have limitations.
The Exercism problems have proven to be very effective at measuring an LLM's ability to modify existing code. I receive a lot of feedback that the aider benchmarks correlate strongly with people's "vibes" on model coding skill. I agree. The scores have felt quite aligned with my hands-on experience coding with most of the top models over the last 18+ months.
To be clear, the purpose of the benchmark is to help me quantitatively assess and improve aider and make it more effective. But it's also turned out to be a great way to measure the coding skill of LLMs.
6 replies →
I like to make up my own tests, that way you know it is actually thinking.
Tests that require thinking about the physical world are the most revealing.
My new favourite is:
You have 2 minutes to cool down a cup of coffee to the lowest temp you can.
You have two options: 1. Add cold milk immediately, then let it sit for 2 mins.
2. Let it sit for 2 mins, then add cold milk.
Which one cools the coffee to the lowest temperature and why?
Phrased this way without any help, all but the thinking models get it wrong
40 replies →
They leak the second they are used on a model behind an API, don't they?
1 reply →
Using up to 32k thinking tokens, Sonnet 3.7 set SOTA with a 64.9% score.
It's clear that progress is incremental at this point. At the same time Anthropic and OpenAI are bleeding money.
It's unclear to me how they'll shift to making money while providing almost no enhanced value.
30 replies →
Paul, I saw in the notes that using claude with thinking mode requires yml config updates -- any pointers here? I was parsing some commits, and I couldn't tell if you only added architect support through openrouter. Thanks!
2 replies →
Also for $36.83 compared to o1's $186.50
7 replies →
How does it stack up against Grok3? I've seen some discussion that Grok3 is good for coding.
5 replies →
Interesting that the "correct diff format" score went from 99.6% with Claude 3.5 to 93.3% for Claude 3.7. My experience with using claude-code was that it consistently required several tries to get the right diff. Hopefully all that will improve as they get things ironed out.
Reasoning models pretty reliably seem to do worse at exacting output formats/structured outputs—so far with Aider it has been an effective strategy to employ o1 to “think” about the issue at hand, and have Sonnet implement. Interested to try various approaches with 3.7 in various combinations of reasoning effort.
1 reply →
3.7 completed a lot more than 3.5, without seeing the actual results, we can't tell if there were any regressions in the edit format among the previously completed tasks.
That's a file context problem because you use cursor or cline or some other crap context maker. Try Clood.
Unless "anthropic high usage" which I just watch the incident reports I one shot features regularly.
At a high skill level. Not front end. Back end c# in a small but great framework that has poor documentation. Not just endpoints but full on task queues.
So really, it's a context problem. You're just not laser focusing your context.
Try this:
Set up a context with the exact files needed. Sure ai "should" do that but it doesn't. Especially not cursor or cline. Then try.
Hell try it with clood after I update with 3.7. I bet you, if you clood file it, then you get one shots.
I have a long history of clood being a commit in my projects and it's a clood one shot.
9 replies →
Hi Paul, been following the aider project for about a year now to develop an understanding of how to build SWE agents.
I was at the AI Engineering Summit in NYC last week and met an (extremely senior) staff ai engineer doing somewhat unbelievable things with aider. Shocking things tbh.
Is there a good way to share stories about real-world aider projects like this with you directly (if I can get approval from him)? Not sure posting on public forum is appropriate but I think you would be really interested to hear how people are using this tool at the edge.
Hope it gets to be public, I love to learn "weird" (or unusual) ways of using tools
Thanks for all the work on aider, my favorite AI tool.
It really is best in slot. Owe it to git, which has a particular synergy with a hallucination-prone but correctable system
8 replies →
Have you tried Claude 3.7 + Deepseek as the architect? Seeing as "DeepSeek R1 + claude-3-5-sonnet-20241022" is the second place option, "DeepSeek R1 + claude-3-7" would hopefully be the highest ranking choice so far?
It looks like Sonnet 3.7 (extended thinking) would be a better architect than R1.
I'll be trying out Sonnet 3.7 extended thinking + Sonnet 3.5 or Flash 2.0, which I assume would be at the top of the leaderboard.
2 replies →
Nice !
Could we please get benchmarks for architect / DeepSeek R1 + claude-3-7-20250219 ?
To compare perf and price with Sonnet-3.7-thinking
I'd like to second the thanks for Aider, I use it all the time.
Working really well for me. Thanks for Aider!
Updated. #1 with thinking
I’d like to 3rd the thanks for Aider it’s fantastic!
Any chance you can add support for Vertex AI Sonnet 3.7, which looks like it's available now? Thank you!
Is aider still relevant vs. Claude Code?
Yes. Absolutely it is. For different workloads it is an insanely effective tool.
And yet, "DeepSeek R1 + claude-3-5-sonnet-20241022" scores 64% on the same benchmark 30% cheaper.
It's amazing what Deepseek is putting on the table while being full open source.
r1 + Claude 3.7 when?
I like that we're just saying they're thinking now. John Searle would be furious.
(I kid, I know what is meant by that.)
Hi everyone! Boris from the Claude Code team here. @eschluntz, @catherinewu, @wolffiex, @bdr and I will be around for the next hour or so and we'll do our best to answer your questions about the product.
One thing I would love to have fixed - I type in a prompt, the model produces 90% or even 100% of the answer, and then shows an error that the system is at capacity and can't produce an answer. And then the response that has already been provided is removed! Please just make it where I can still have access to the response that has been provided, even if it is incomplete.
This. Claude team, please fix this!
1 reply →
I'll be publishing a Firefox extension as a temporary fix, will post it here. (I don't use Chrome.)
3 replies →
Yup. Its a great issue which messes like , cmon you were there at the last line.
To me it doesn’t look like a bug. I believe it is a intended “feature” pushed from high management - a dark patern to make plebs pay for answer that has overflowed the quota.
Plus one for this.
The biggest complaint I (and several others) have is that we continuously hit the limit via the UI after even just a few intensive queries. Of course, we can use the console API, but then we lose ability to have things like Projects, etc.
Do you foresee these limitations increasing anytime soon?
Quick Edit: Just wanted to also say thank you for all your hard work, Claude has been phenomenal.
We are definitely aware of this (and working on it for the web UI), and that's why Claude Code goes directly through the API!
13 replies →
I paid for it for a while, but I kept running out of usage limits right in the middle of work every day. I'd end up pasting the context into ChatGPT to continue. It was so frustrating, especially because I really liked it and used it a lot.
It became such an anti-pattern that I stopped paying. Now, when people ask me which one to use, I always say I like Claude more than others, but I don’t recommend using it in a professional setting.
4 replies →
If you are open to alternatives, try https://glama.ai/gateway
We currently serve ~10bn tokens per day (across all models). OpenAI compatible API. No rate limits. Built in logging and tracing.
I work with LLMs every day, so I am always on top of adding models. 3.7 is also already available.
https://glama.ai/models/claude-3-7-sonnet-20250219
The gateway is integrated directly into our chat (https://glama.ai/chat). So you can use most of the things that you are used to having with Claude. And if anything is missing, just let me know and I will prioritize it. If you check our Discord, I have a decent track record of being receptive to feedback and quickly turning around features.
Long term, Glama's focus is predominantly on MCPs, but chat, gateway and LLM routing is integral to the greater vision.
I would love feedback if you are going to give a try frank@glama.ai
15 replies →
this is also my problem, ive only used the UI with $20 subscription, can I use the same subscription to use the cli? I'm afraid its like those aws api billing where there is no limit to how much I can use then get a surprise bill
13 replies →
Claude is my go to llm for everything, sounds corny but it's literally expanding the circle of what I can reasonably learn, manyfold. Right now I'm attempting to read old philosophical texts (without any background in similar disciplines), and without claude's help to explain the dense language in simpler terms & discuss its ideas, give me historical contexts, explaining why it was written this or that way, compare it against newer ideas - I would've given up many times.
At work I used it many times daily in development. It's concise mode is a breath of fresh air compared to any other llm I've tried. It has helped me find bugs in foreign code bases, explain me the techstack, written bash scripts, saving me dozens of hours of work & many nerves. It generally makes me reach places I wouldn't without due to time constraints & nerves.
The only nitpick is that the service reliability is a bit worse than others, forcing me sometimes to switch to others. This is probably a hard to answer question, but are there plans to improve that?
I'm in the middle of a particularly nasty refactor of some legacy React component code (hasn't been touched in 6 years, old class based pattern, tons of methods, why, oh, why did we do XYZ) at work and have been using Aider for the last few days and have been hitting a wall. I've been digging through Aider's source code on Github to pull out prompts and try to write my own little helper script.
So, perfect timing on this release for me! I decided to install Claude Code and it is making short work of this. I love the interface. I love the personality ("Ruminating", "Schlepping", etc).
Just an all around fantastic job!
(This makes me especially bummed that I really messed up my OA awhile back for you guys. I'll try again in a few months!)
Keep on doing great work. Thank you!
Hey thanks so much! <3
Just started playing with the command-line tool. First reaction (after using it for 5 minutes): I've been using `aider` as a daily driver, with Claude 3.5, for a while now. One of the things I appreciate about aider is that it tells you how much each query cost, and what your total cost is this session. This makes it low-key easy to keep tabs on the cost of what I'm doing. Any chance you could add that to claude-code?
I'd also love to have it in a language that can be compiled, like golang or rust, but I recognize a rewrite might be more effort than it's worth. (Although maybe less with claude code to help you?)
EDIT: OK, 10 minutes in, and it seems to have major issues doing basic patches to my Golang code; the most recent thing it did was add a line with incorrect indentation, then try three times to update it with the correct indentation, getting "String to replace not found in file" each time. Aider with claude 3.5 does this really well -- not sure what the counfounding issue is here, but might be worth taking a look at their prompt & patch format to see how they do it.
If you do `/cost` it will tell you how much you've spent during that session so far.
hi! You can do /cost at any time to see what the current session has cost
One of the silver bullets of Claude, in the context of coding, is that it does NOT use RAG when you use it via the web interface. Sure, you burn your tokens but the model sees everything and this let it reply in a much better way. Is Claude Code doing the same and just doing document-level RAG, so that if a document is relevant and if it fits, all the document will be put inside the context window? I really hope so! Also, this means that splitting large code bases into manageable file sizes will make more and more sense. Another Q: is the context size of Sonnet 3.7 the same of 3.5? Btw Thanks you so much for Claude Sonnet, in the latest months it changed the way I work and I'm able to do a lot more, now.
Right -- Claude Code doesn't use RAG currently. In our testing we found that agentic search out-performed RAG for the kinds of things people use Code for.
17 replies →
Been a long time casual — i.e. happy to fix my code by asking questions and copy/pasting individual snippets via the chat interface. Decided to give the `claude` terminal tool a run and have to admit it looks like a fantastic tool.
Haven't tried to build a modern JS web app in years — it took the claude tool just a few minutes of prompting to convert and refactor an old clunky tool into a proper project structure, and using svelte and vite and tailwind (which I haven't built with before). Trying to learn how to even scaffold a modern app has felt daunting and this eliminates 99% of that friction.
One funny quirk: I asked it to build a test suite (I know zilch about JS testing frameworks, so it picked vitest for me) for the newly refactored app. I noticed that 3 of the 20 tests failed and so I asked it to run vitest for itself and fix the failing things. 2 minutes later, and now 7 tests were failing...
Which is very funny to me, but also not a big deal. Again, it's such a chore to research test libs and then set things up to their conventions. That the claude tool built a very usable scaffold that I can then edit and iterate on is such a huge benefit by itself, I don't need (nor desire) the AI to be complete turnkey solution.
Anthropic is back and cementing its place as the creator of the best coding models—bravo!
With Claude Code, the goal is clearly to take a slice of Cursor and its competitors' market share. I expected this to happen eventually.
The app layer has barely any moat, so any successful app with the potential to generate significant revenue will eventually be absorbed by foundation model companies in their quest for growth and profits.
I think an argument could be reasonably made that the app layer is the only moat. It’s more likely Anthropic eventually has to acquire Cursor to cement a position here than they out-compete it. Where, why, what brand and what product customers swipe their credit cards for matters — a lot.
11 replies →
I wonder if they will offer competitive request counts against Cursor. Right now, at least for me, the biggest downside to Claude is how fast I blow through the limits (Pro) and hit a wall.
At least with Cursor, I can use all "premium" 500 completions and either buy more, or be patient for throttled responses.
3 replies →
hi! I've been using Claude Code in a very complementary way to my IDE, and one of the reasons we chose the terminal is because you can open it up inside whichever IDE you want!
Why not just open source Claude Code? people have tried to reverse eng the minified version https://gist.githubusercontent.com/1rgs/e4e13ac9aba301bcec28...
Paste it into Claude and ask it to made the minified code more readable ;)
Agree the code should just be open source but there's nothing secretive that you can't extract manually.
1 reply →
Claude Code is on github: https://github.com/anthropics/claude-code
3 replies →
Hi Boris, love working with Claude! I do have a question—is there a plan to have Claude 3.5 Sonnet (or even 3.7!) made available on ca-central-1 for Amazon Bedrock anytime soon? My company is based in Canada and we deal with customer information that is required to stay within Canada, and the most recent model from Anthropic we have available to us is Claude 3.
Concur. Models aren’t real until I can run them inside my perimeter.
A minor ChatGPT feature I miss with Claude is temporary chats. I use ChatGPT for a lot of random one-off questions and don’t want them filling up my chat history with so many conversations.
Hi and congrats on the launch!
Will check out Claude Code soon, but in the meantime one unrelated other feature request: Moving existing chats into a project. I have a number of old-ish but super-useful and valuable chats (that are superficially unrelated) that I would like to bring together in a project.
I really want to try your AI models, but "You must have a valid phone number to use Anthropic's services." is a show-stopper for me.
It's the only mainstream AI service that requests this information. After a string of security lapses by many of your competitors, I have zero faith in the ability of a "fast moving" AI-focused company to keep my PII data secure.
It's a phone number. It's probably been bought / sold a few times already. Unless you're on the level of Edward Snowden, I wouldn't worry about it. But maybe your sense of privacy is more valuable than the outcome you'd get from Claude. That's fine too.
3 replies →
I pay for a number from voip.ms and use sms forwarding. Its very cheap and it works on telegram as well which seemed fairly strict at detecting most voips.
Does the fact its so ungodly expensive and highly rate limited kind of prove the modern point that AI actually uses tons of water and electricity per prompt? People are used to streaming YouTube while they sleep and it's hard to think of other web technology this intensive. OpenAI is hostile to this subject. Does Claude have plans to tackle this?
> People are used to streaming YouTube while they sleep
Youtube is used to showing them ads while they sleep
Is there / are you planning a way to set $ limits per API key? Far as I can tell the "Spend limits" are currently per-org only which seems problematic.
Good idea! Tracking here: https://github.com/anthropics/claude-code/issues/16
You can with Workspaces - https://support.anthropic.com/en/articles/9796807-creating-a...
Hi! I’ve been using Claude for macOS and iOS coding for a while, and it’s mostly great, but it’s always using deprecated APIs, even if I instruct it not to. It will correct the mistake if I ask it to, but then in later iterations, it will sometimes switch back to using a deprecated API. It also produces a lot of code that just doesn’t compile, so a lot of time is spent fixing the made up or deprecated APIs.
Awesome to see a new Claude model - since 3.5 its been my go-to for all code related tasks.
I'd really like to use Claude Code in some of my projects vs just sharing snippets via the UI but I'm curious how might doing this from our source directory affect our IP including NDA's, trade secret protections, prior disclosure rules on (future) patents, open source licensing restrictions re: redistribution etc?
Also hi Erik! - Rob
Hi Boris et al, can you comment on increased conversation lengths or limits through the UI? I didn't see that mentioned in the blog post, but it is a continued major concern of $20/month Claude.ai users. Is this an issue that should be fixed now or still waiting on a larger deployment via Amazon or something? If not now, when can users expect the conversation length limitations will be increased?
It would be great if we could upgrade API rate limits. I've tried "contacting sales" a few times and never received a response.
edit: note that my team mostly hits rate limits using things like aider and goose. 80k input token is not enough when in a flow, and I would love to experiment with a multi-agent workflow using claude
Now that the world's gotten used to the existence of AI, any hope on removing the guardrails on Claude? I don't need it to answer "How do I make meth", but I would like to not have to social engineer my prompts. I'd like it to just write the code I asked for and not judge me on how ethical the code might be.
Eg Claude will refuse to write code to wget a website and parse the html if you ask it to scrape your ex girlfriend's Instagram profile, for ethical and tos reasons, but if you phrase the request differently, it'll happily go off and generate code that does that exact thing.
Asking it to scrape my ex girlfriend's Instagram profile is just a stand in for other times I've hit a problem where I've had to social engineer my way past those guard rails, but does having those guard rails really provide value on a professional level?
Not having headlines like "Claude Gives Stalker Instructions" has a significant value to their business I would wager.
I'm very much in favour of removing the guardrails but I understand why they're in place. The problem is attribution. You can teach yourself how to engage in all manner of dark deeds with a library or wikipedia or a search engine and some time, but any resulting public outcry is usually diffuse or targeted at the sources rather than the service. When Claude or GPT or Stable Diffusion are used to generate something judged offensive, the outcry becomes an existential threat to the provider.
How is your largest customer, Cursor, taking the news that you'll be competing directly with them?
They probably aren't thrilled, but a lot of users will prefer a UI and I doubt Anthropic has the spare cycles to make a full Cursor competitor.
Unless Cursor had agreed to an exclusivity agreement with Anthropic, Antropic was (and still is) at risk of Cursor moving to a different provider or using their middleman position to train/distill their own model that competes with Anthropic.
honestly, is this something that anthropic should be worried about? you could ask the same question from all the startups that were destroyed by OpenAI.
Anthropic is still making the shovels
Great, thanks! Could you compare this new tool to Aider?
Do you think Claude Code is "better", in terms of capabilities and token efficiency, than other tools such as Cline, Cursor, or Aider?
Claude Code is a research preview -- it's more rough, lets you see model errors directly, etc. so it's not as polished as something like Cline. Personally I use all of the above. Engineers here at Anthropic also tend to use Claude Code alongside IDEs like Cursor.
Thanks for the product! Glad to hear the (so called) "safety" is being walked back on, previously Claude has been feeling a little like it is treating me as a child, excited to try it out now.
In the console, TPM limit for 3.7 is not shown (I'm tier 4). Does it mean there is no limit, or is it just pending and is "variable" until you set it to some value?
We set the Claude Code rate limits to be usable as a daily driver. We expect hitting rate limits for synchronous usage to be uncommon. Since this is a research preview, we recommend you start small as you try the product though.
1 reply →
Your footnote 3 seems to imply that the low number for o1 and Grok3 is without parallelism, but I don't think it's publicly known whether they use internal parallelism? So perhaps the low number already uses parallelism, while the high number uses even more parallelism?
Also, curious if you have any intuition as to why the no-parallelism number for AIME with Claude (61.3%) is quite low (e.g., relative to R1 87.3% -- assuming it is an apples to apples comparison)?
Awesome work, Claude is amazingly good at writing code that is pretty much plug and play.
Could you speak at all about potential IDE integrations? An integration into Jetbrains IDEs would be super useful - I imagine being able to highlight a bit of code and having a plugin check the code graph to see dependencies, tests etc that might be affected by a change.
Copying and pasting code constantly is starting to seem a bit primitive.
Part of our vision is that because Claude Code is just in the terminal, you can bring it into any IDE (or server) you want! Obviously that has tradeoffs of not having a full GUI of the IDE though
2 replies →
Jetbrains have an official mcp plugin
1 reply →
Why gatekeep Claude Code, instead of releasing the code for it? It seems like a direct increase in revenue/API sales for your company.
I'm not affiliated with Anthropic, but it seems like doing this will commoditize Claude (the AIaaS). Hosted AI providers are doing all they can to move away from being interchangeable commodities; it's not good for Anthropic's revenue for users to be able to easily swap-out the backend of Cloud Code to a local Olama backend, or a cheaper hosted DeepSeek. Open sourcing Claude Code would make this option 1 or 2 forks/PRs away.
2 replies →
What do I need to do to get unbanned? I have filled in the provided Google Docs form 3-4 times to no avail. I got banned almost immediately after joining. My best guess is that I got banned because I used a VPN. https://news.ycombinator.com/item?id=40808815
Is there a way to always accept certain commands across sessions? Specifically for things like reading or updating files I don't want to have to approve that each time I open a new repl.
Also, is there a way to switch models between 3.5-sonnet and 3.5-sonnet-thinking? Got the initial impression that the thinking model is using an excessive amount of tokens on first use.
When you are prompted to accept a bash command, we should be giving you the option to not ask again. If you're not seeing that for a specific bash command, would you mind running /bug or filing an issue on Github? https://github.com/anthropics/claude-code/issues
Thinking and not thinking is actually the same model! The model thinks automatically when you ask it to. If you don't explicitly ask it to think, it won't use thinking.
1 reply →
Right now no, but if you run in docker, you can use `--dangerously-skip-permissions`
Some commands could be totally fine in one context, but bad in a different i.e. pushing to master
For the pokemon benchmark, what happened after the Lt Surge gym? Did the model stall or run out of context or something similar?
A bit off topic but I wanted to let you know that anthropic is currently in violation of EU Directive 98/6/EC:
> The selling price and the unit price must be indicated in an unambiguous, easily identifiable and clearly legible manner for all products offered by traders to consumers (i.e. the final price should include value added tax and all other taxes).
I wanted to see what the annual plan would cost as it was just displaying €170+VAT, and when I clicked the upgrade button to find out (I checked everywhere on the page) then I was automatically subscribed without any confirmation and without ever seeing the final price before the transaction was completed.
You can stuff up your EU directives up your nose, like your bottle caps when you try to drink from a European bottle
2 replies →
Hi Boris! Thank you for your work on Claude! My one pet peeve with Claude specifically, if I may: I might be working on a Svelte codebase and Claude will happily ignore that context and provide React code. I understand why, but I’d love to see much less of a deep reliance on React for front-end code generation.
When I first started using Cursor the default behavior was for Claude to make a suggestion in the chat, and if the user agreed with it, they could click apply or cut and paste the part of it they wanted to use in their larger project. Now it seems the default behavior is for Claude to start writing files to the current working directory without regard for app structure or context (e.g., config files that are defined elsewhere claude likes to create another copy of). Why change the default to this? I could be wrong but I would guess most devs would want to review changes to their repo first.
Cursor has two LLM interaction modes, chat and composer. The chat does what you described first and composer can create/edit/delete files directly. Have you checked which mode you're on? It should be a tab above your chat window.
1 reply →
This is a question for Cursor team.
> We’ve also improved the coding experience on Claude.ai. Our GitHub integration is now available on all Claude plans—enabling developers to connect their code repositories directly to Claude
Would love to learn a bit more about how the GitHub integration works. From https://support.anthropic.com/en/articles/10167454-using-the... it seems it’s read only.
Does Claude Code let me take a generated/edited artifact and commit it back as a PR?
The https://claude.io/ integration is read-only. Basically you OAuth with GitHub and now you can select a repository, then select files or directories within it to add to either a Claude Project or to an individual prompt.
Claude Code can run commands including "git" commands, so it can create a branch, commit code to that branch and push that branch to GitHub - at which point point you can create a PR.
hey guys! i was wondering why you chose to build Claude code via CLI when many popular choices like cursor and windsurf fork VScode. do you envision the future of Claude code to abstract away the codebase entirely?
We wanted to bring the model to people where they are without having to commit to a specific tool or radically change their workflows. We also wanted to make a way that lets people experience the model’s coding abilities as directly as possible. This has tradeoffs: it uses a lot of tokens, and is rough (eg. it shows you tool errors and model weirdness), but it also gives you a lot of power and feels pretty awesome to use.
2 replies →
It would be great to have a C# / .NET SDK available for Claude so it can be integrated into Semantic Kernel [0][1]. Are there any plans for this?
[0] https://github.com/microsoft/semantic-kernel/issues/5690#iss...
[1] https://github.com/microsoft/semantic-kernel/pull/7364
I'm curious why there are no results for the "Claude 3.7 Extended Thinking" on SWE-Bench and Agentic tool use.
Are you finding that extended thinking helps a lot when the whole problem can be posed in the prompt, but that it isn't a major benefit for agentic tasks?
It would be a bit surprising, but it would also mirror my experiences, and the benchmarks which show Claude 3.5 being better at agentic tasks and SWE tasks than all other models, despite not being a reasoning model.
Are you guys paying Claude for its assistance with your products
It would be amazing to be able to use an API key to submit prompts that use our Project Knowledge. That doesn't seem to be currently possible, right?
From the release you say: "[..] in developing our reasoning models, we’ve optimized somewhat less for math and computer science competition problems, and instead shifted focus towards real-world tasks that better reflect how businesses actually use LLMs."
Can you tell us more about the trade-offs here?
Also, are you using synthetic data for improving the responses here, or are you purely leveraging data from usage/partner's usage?
Thank you for the update!
I recently attempted to use the Google Drive integration but didn't follow through with connecting because Claude wanted access to my entire Google Drive. I understand this simplifies the user experience and reduced time to ship, but is there anyway the team can add "reduce the access scope of Google Drive integration" to your backlog. Thank you!
Also, I just caught the new Github integration. Awesome.
Small UX suggestion, but could you make submission of prompt via URL parameter work? It used to be possible via https://claude.ai/new?q={query}, but that stopped working. It works for ChatGPT, Grok, and DeepSeek. With Claude you have to go and manually click the submit button.
Who the heck is on your UX team?
WHY is a huge % of my UX filled with nothing? I would apprececiate metrics, token graphs etc
https://i.imgur.com/VlxLCwI.png
Why so much wasted space? ... >>??
https://i.imgur.com/7LlCLUf.jpeg
Did you guys ever fix the issue where if UK users wanted to use the API they have to provide a VAT number?
Love the UI so far. The experience feels very inspired by Aider, which is my current choice. Thanks!
Serious question: What advice would you give to a Computer Science student in light of these tools?
Serious answer: learn to code.
You still need to know what good code looks like to use these tools. If you go forward in your career trusting the output of LLMs without the skills to evaluate the correctness, style, functionality of that code then you will have problems.
People still write low level machine code today, despite compilers having existed for 70+ (?) years.
We'll always need full-stack humans who understand everything down to the electrons even in the age of insane automation that we're entering.
5 replies →
The thing I would like automated is highlighting a function in my code then ask the AI to move it to a new module-file and import that new module.
I would like this to happen easily like hitting a menu or button without having to write an elaborate "prompt" every time.
Is this possible?
I think most language servers have a feature like this right?
4 replies →
Hi there. There are lots of phrases/patterns that Claude always uses when writing and it was very frustrating with 3.5. I can see with 3.7 those persist. Is there any way for me to contact you and show those so you can hopefully address them?
Any change there will be a way to copy and paste the responses into other text boxes (i.e., a new email) and not have to re-jig the formatting?
Lists, numbers, tabs, etc. are all a little time consuming... minor annoyance but thought I'd share.
Can you give some insight into how you chose the reply limit length? It seems to cut off many useful programs that are 80%-90% done and if the limit were just a little higher it would be a source of extraordinary benefit.
If you can reproduce that, would you mind reporting it with /bug?
2 replies →
Congrats on the launch! You said its an important tool for you (Claude Code) how does this fit in with Co-Pilot, Cursor, etc. Do you/your teammates only rely on Claude Code? What do you reach for for different tasks?
Claude Code is super popular internally at Anthropic. Most engineers like to use it together with an IDE like Cursor, Windsurf, VS Code, Zed, Xcode, etc. Personally I usually start most coding tasks in Code, then move to an IDE for finishing touches.
Is there plans to add websearch function over some core websites (SO, API docs)? Competitors have it, and in my experience this provide very good grounding for coding tasks (way less API functions hallucinated).
What kind of sorcery did you use to create Claude? Honest question :)
Reticulating...
Does this actually have an 8k (or more) output context via the API?
3.5 did with a beta header but while 3.6 claimed to, it always cut its responses after 4k.
IIRC someone reported it on GH but had no reply.
Any way to parallelize tool use? When I go into a repo and ask "what's in here", I'm aiming for a summary that returns in 20 seconds.
My key got killed months ago when I tested it on a PDF, and support never got back to me so I am waiting for OpenRouter support!
Hi, what are the privacy terms for Claude Code? Is it memorizing the codebase it’s helping with? From an enterprise standpoint
Thank you to the team. Looks like a great release. Already switching existing prompts to Claude 3.7 to see the eval results :)
Thanks for this - exciting launch. Do you have examples of cool applications or demos that the HN crowd should check out?
We built Claude Code with Claude Code!
2 replies →
hi! I've been working on demos where I let Claude Code run for hours at a time on a sandboxed project: https://x.com/ErikSchluntz/status/1894104265817284770
TLDR: asking claude to speed up my code once 1.8x'd perf, but putting it in a loop telling it to make it faster for 2 hours led to a 500x speedup!
4 replies →
>Do you have examples of cool applications or demos that the HN crowd should check out?
Not OP obviously, but I've built so many applications with Claude, here are just a few:
[1]
Mockup of Utopian infrastructure support button (this is just a mockup, the buttons don't do anything): https://claude.site/artifacts/435290a1-20c4-4b9b-8731-67f5d8...
[2]
Robot body simulation: https://claude.site/artifacts/6ffd3a73-43d6-4bdb-9e08-02901d...
[3]
15-piece slider puzzle: https://claude.site/artifacts/4504269b-69e3-4b76-823f-d55b3e...
[4]
Canada joining the U.S., checklist: https://claude.site/artifacts/6e249e38-f891-4aad-bb47-2d0c81...
[5]
Secure encryption and decryption with AES-256-GCM with password-based key derivation:
https://claude.site/artifacts/cb0ac898-e5ad-42cf-a961-3c4bf8...
(Try to decrypt this message
kFIxcBVRi2bZVGcIiQ7nnS0qZ+Y+1tlZkEtAD88MuNsfCUZcr6ujaz/mtbEDsLOquP4MZiKcGeTpBbXnwvSLLbA/a2uq4QgM7oJfnNakMmGAAtJ1UX8qzA5qMh7b5gze32S5c8OpsJ8=
With the password "Hello Hacker News!!" (without quotation marks))
[6]
Supply-demand visualizer under tariffs and subsidies: https://claude.site/artifacts/455fe568-27e5-4239-afa4-051652...
[7]
fortune cookie program: https://claude.site/artifacts/d7cfa4ae-6946-47af-b538-e6f992...
[8]
Household security training for classified household members (includes self-assessment and certificate): https://claude.site/artifacts/7754dae3-a095-4f02-b4d3-26f1a5...
[9]
public service accountability training program: https://claude.site/artifacts/b89a69fb-1e46-4b5c-9e96-2c29dd...
[10]
Nuclear non-proliferation "big brother" agent technical demonstration: https://claude.site/artifacts/555d57ba-6b0e-41a1-ad26-7c90ca...
Dating stuff:
[11]
Dating help: Interest Level Assessment Game (is she interested?) https://claude.site/artifacts/523c935c-274e-4efa-8480-1e09e9...
[12]
Dating checklist: https://claude.site/artifacts/10bf8bea-36d5-407d-908a-c1e156...
Hi Boris,
Would it be possible to bring back sonnet 2024 June?
That model was the most attentive.
Because we lost that model this release a value loss for me personally.
Seems to still be available via API as claude-3-5-sonnet-20240620
I tried signing up to use Claude about 6 months ago and ran into an error on the signup page. For some reason this completely locked me out from signing up since a phone number was tied to the login. I have submitted requests to get removed from this blacklist and heard nothing. The times I have tried to reach out on Twitter were never responded to. Has the customer support improved in the last 6 months?
You can try using it through Github Copilot? Just as a different avenue for usage.
1 reply →
Not a question but thank you for helping make awesome software that helps us make awesome software, too :)
Can you let the API team know that the /v1/models endpoint has been broken for hours? Thanks.
Hello! Member of the API team here. We're unable to find issues with the /v1/models endpoint—can you share more details about your request? Feel free to email me at suzanne@anthropic.com. Thank you!
2 replies →
with Claude coder, how does history work? I used it with my account, ran out of credit then switched to a work account but there was no chat history or other saved context of the work that had been done. I logged back in with my account to try copy it but it was gone.
Did you run the Aider benchmarks to get a comparison of Claude Code vs. Aider?
Any updates on web search?
When are you providing an alternative to email magic login links?
What are your thoughts on having a UI/design benchmark?
Which starter pokemon does Claude typically choose?
I'd also be interested in stats on Helix Fossil vs. Dome Fossil.
Will Claude be available on Azure?
when there are two commands in a prompt example
do A and then do B.
the model completely ignores the second task B.
will you guys allow remote work ever for engineers?
CLAUDE NUMBA ONE!!!
Congrats on the new release!
Hi @eschluntz, @catherinewu, @wolffiex, @bdr. Glad that you are so plucky and upbeat!
How do you feel about raking in millions while attempting to make us all unemployed?
How do you feel about stealing open source code and stripping the copyright?
Have you seen https://mycoder.ai? Seems quite similar. It was my own invention and it seems that you guys are thinking along similar lines - incredibly similar lines.
Have you seen https://www.codebuff.com?
2 replies →
Folks, let me tell you, AI is a big league player, it's a real winner, believe me. Nobody knows more about AI than I do, and I can tell you, it's going to be huge, just huge. The advancements we're seeing in AI are tremendous, the best, the greatest, the most fantastic. People are saying it's going to change the world, and I'm telling you, they're right, it's going to be yuge. AI is a game-changer, a real champion, and we're going to make America great again with the help of this incredible technology, mark my words.
Kagi LLM benchmark updated with general purpose and thinking mode for Sonnet 3.7.
https://help.kagi.com/kagi/ai/llm-benchmark.html
Appears to be second most capable general purpose LLM we tried (second to gemini 2.0 pro, in front of gpt-4o). Less impressive in thinking mode, about at the same level as o1-mini and o3-mini (with 8192 token thinking budget).
Overall a very nice update, you get higher quality and higher speed model at same price.
Hope to enable it in Kagi Assistant within 24h!
Thank you to the Kagi team for such fast turn around on new LLMs being accessible via the Assistant! The value of Kagi Assistant has been a no-brainer for me.
[flagged]
2 replies →
I'm surprised that Gemini 2.0 is first now. I remember that Google models were under performing on kagi benchmarks.
Having your own hardware to run LLMs will pay dividends. Despite getting off on the wrong foot, I still believe Google is best positioned to run away with the AI lead, solely because they are not beholden to Nvidia and not stuck with a 3rd party cloud provider. They are the only AI team that is top to bottom in-house.
4 replies →
Gemini 2 is really good, and insanely fast.
2 replies →
How did you chose the 8192 token thinking budget? I've often seen Deepseek R1 use way more than that.
Arbitrary, and even with this budget it is already more verbose (and slower) overall than all the other thinking models - check tokens and latency in the table.
I see it in Kagi Assistant already and it's not even 24 hours! Nice.
One thing I don't understand is why Claude 3.5 Haiku, a non thinking model in the non thinking section, says it has a 8192 thinking budget.
Do you think kagi is the right Eval tool? If so,why?
The right eval tool depends on your evaluation task. Kagi LLM benchmark focuses on using LLMS in the context of information retrieval (which is what Kagi does) which includes measuring reasoning and instruction following capabilities.
Nice, but where is Grok?
Perhaps they're waiting for the Grok API to be public?
I thought o3-mini was o1-mini. OpenAI's naming gets confusing.
You can get your HN profile analyzed by it and it's pretty funny :)
https://hn-wrapped.kadoa.com/
I'm using this to test the humor of new models.
** Roast ***
* You've spent more time talking about your Carnatic raga detector than actually building it – at this rate, LLMs will be composing ragas before your detector can identify them.
* You bought a 7950X processor but can't figure out what to do with it – the computing equivalent of buying a Ferrari to drive to the grocery store once a week.
* You're so concerned about work-life balance that you took a sabbatical to think about your career, only to spend it commenting on HN about other people's careers.
*** End ***
I'll be in my room crying, in case anyone's looking for me.
My roast:
Roast You've spent so much time discussing Apple vs Microsoft that Tim Cook and Satya Nadella probably have a joint restraining order against you.
Your comments about HTTPS everywhere suggest you're the kind of person who wears a tinfoil hat... but only after thoroughly researching the optimal thickness for blocking government signals.
You seem to have strong opinions about Flash - we get it, you're old enough to remember when websites had intro animations and your laptop could double as a space heater.
———
Totally forgot about the flash debates of the early 2010s!
That sabbatical one is savage.
1 reply →
> Your salary is so low even your legacy code feels sorry for you.
> You're the only person on HN who thinks $800/month is a salary and not a cloud computing bill.
ouch
On the bright side. Not many here could 10x their salary in a couple of years.
1 reply →
>You're the only person on HN who thinks $800/month is a salary and not a cloud computing bill.
Now that is funny!
Got absolutely read to filth:
> You've spent more time explaining why Go's error handling is bad than Go developers have spent actually handling errors.
> Your relationship with programming languages is like a dating show - you keep finding flaws in all of them but can't commit to just one.
> If error handling were a religion, you'd be its most zealous missionary, converting the unchecked one exception at a time.
> You've spent more time explaining why Go's error handling is bad than Go developers have spent actually handling errors.
That is absolutely hilarious. Really well done by everyone who made that line possible.
Yea, these are nicely done. To add some balance:
> After years of defending Go, you'll secretly start a side project in Rust but tell no one on HN about your betrayal
I got "You've spent more time explaining why Rust isn't memory-safe than most people have spent writing actual Rust code." So I suspect these are not as free-form-generated as they actually look?
> For someone who worked at Reddit, you sure spend a lot of time on HN. It's like leaving Facebook to spend all day on Twitter complaining about social media.
Wow, so spot on it hurts!
>Your ideal tech stack is so old it qualifies for social security benefits
>You're the only person who gets excited when someone mentions Trinity Desktop Environment in 2025
> You probably have more opinions about PHP's empty() function than most people have about their entire career choices
1 reply →
> For someone who criticizes corporate structures so much, you've spent an impressive amount of time analyzing their technical decisions. It's like watching someone critique a restaurant's menu while eating there five times a week.
> You complain about digital distractions while writing novels in HN comment threads. That's like criticizing fast food while waiting in the drive-thru line.
>You'll write a thoughtful essay about 'digital minimalism' that reaches the HN front page, ironically causing you to spend more time on HN responding to comments than you have all year.
It sees me! Noooooo ...
Surprisingly accurate, but seems to be based on a very small snippet of actual comments (presumably to save money). I wonder what the prompt would output when given the full 200k tokens of context.
Yeah it seems like it doesn't go back very far, which is understandable.
My roasts are savage:
> Your 236-line 'simplified' code example suggests you might need to look up the definition of 'simplified' in a dictionary that's not written in Ruby.
OUCH
> You've spent so much time worrying about Facebook tracking you that you've failed to notice your dental nanobot fantasies are far more concerning to the rest of us.
Heard.
> You left a high-paying tech job to grow plants in water, which is basically just being a farmer with extra steps and less sunlight.
Ha
Also:
> Your comments read like someone who discovered philosophy in their 30s and now can't decide if they want to code or become the next Marcus Aurelius.
skull emoji
> - You've reminded so many people to use 'Show HN:' that you should probably just apply for a moderator position already.
> - Your relationship with AI coding assistants is more complicated than most people's dating history - Cline, Cursor, Continue.Dev... pick a lane!
> - You talk about grabbing coffee while your LLM writes code so much that we're not sure if you're a developer or a barista who occasionally programs.
I laughed hard at this :D
> You've asked about building a homebrew computer in 2013, and we're still waiting for the 'Show HN' post. Moore's Law has changed less than your project timeline.
> Your journey from PHP to OCaml suggests you enjoy pain, just in increasingly sophisticated forms.
> You seem to spend so much time worrying about NSA surveillance that you probably encrypt your grocery lists. The NSA agent assigned to you is bored to tears.
Hahaha these are excellent, though it really latched on to the homebrew PC stuff I was into back in 2013
> Hacker News: You'll write a comment so perfectly balanced between technical insight and dry humor that it breaks the upvote system, forcing dang to implement a new 'slow clap' feature just for you.
fist pump
> You've recommended Marginalia search so many times, we're starting to think you're either the developer or just really enjoy websites that look like they were designed in 1998.
Actually quite funny.
[1] https://hn-wrapped.kadoa.com/jddj?share
Especially hilarious considering that this is the actual marginalia developer: https://hn-wrapped.kadoa.com/marginalia_nu
> You defend Java with such passion that Oracle's legal team is considering hiring you as their chief evangelist - just don't tell them about your secret admiration for more elegant programming paradigms.
I love that its two predictions of projects I’m likely doing in 2025 are.. projects I actually tried already
> You built your own Klaviyo alternative to save €500, but how many hours of development at market rate did that cost? The true Greek economy at work!
ouch (ㅠ﹏ㅠ)
This thing is hilarious. :)
Roast:
- Your comments have more doom predictions than a Y2K convention in December 1999.
- You've used 'stochastic parrot' so many times, actual parrots are filing for trademark infringement.
- If tech dystopia were an Olympic sport, you'd be bringing home gold medals while explaining how the podium was designed by committee and the medal contains surveillance chips.
> You've used 'stochastic parrot' so many times, actual parrots are filing for trademark infringement.
Ahahaha:) This line wins:)
In my case the first two roasts were contrivances but the last one carries the bunch:
> For someone who claims to be only 33, you have the technological opinions of at least three 60-year-old UNIX greybeards stacked in a trenchcoat.
Guilty as charged :-3
> You've experienced so many startup failures that your LinkedIn profile should just read 'Professional Titanic Passenger: Always Picks the Wrong Ship'.
:'(
> You took a year off for mental health but still couldn't resist building 'for-profit projects' during your break. The only thing more persistent than your work ethic is your inability to actually relax.
> You complain about Elixir's lack of types but keep using it anyway. This is the programming equivalent of staying in a relationship where you keep trying to change the other person.
> You've lived in multiple countries but spend most of your time on HN explaining why their tech infrastructure is terrible. Maybe the common denominator is you?
Ouch, it's pretty good haha
> Your ideal laptop would run Linux flawlessly with perfect hardware compatibility, have MacBook build quality, and Windows game support. Meanwhile, the rest of us live in reality.
Damn, got me there haha
> You predicted Facebook would collapse into a black hole in 2012. The only black hole we found was the one where all your optimism disappeared.
Ouch... :)
PS: This profile check idea is really funny, great job :)
> For someone who has strong opinions about rice cookers, bookmarklets, and toilet flushing mechanisms, we're surprised you haven't started a 'Unnecessarily Detailed Reviews of Mundane Objects' newsletter yet.
That's not a terrible idea.
> You've mentioned 'boring but hard problems' so many times that we're starting to think you're trying to convince yourself your work is interesting.
> Your obsession with data extraction makes me wonder if you're secretly a web scraper that gained sentience and is now posting on HN.
> You talk about AI automating tedious tasks so much that I'm surprised you haven't built an AI to write your HN comments for you yet.
Those are great. Well done! That it can just read your entire comment history gives it great potential for a whole new dimension of humor.
Here is a user script to replace HN profiles with this improved version.
https://pastebin.com/raw/9dEW4Bk8
“You've spent more time optimizing DOM manipulation for ASCII art than most people spend deciding what to watch on Netflix in their entire lives.”
Ouch… :)
> Spends hours crafting the perfect anti-doom-scrolling strategy only to immediately doom-scroll through HN comments about doom scrolling.
Spot on!
> Has an M2 Max with 64GB RAM but probably still complains when Chrome opens more than 5 tabs.
Not true, I have 40 tabs open!
> Created a tool to generate portfolios in 5 minutes but spent 5 hours explaining how to optimize YouTube settings. Priorities!
Ouch! Brutal and funny at the same time.
Thank you for making this!
> A 30+ year dev veteran who's seen it all, from OOP spaghetti nightmares to the promised land of functional programming, now balancing toddler-wrangling with running 70B parameter models on an M4 Mac. Your comments oscillate between deep technical insights and the occasional 'get off my lawn' energy that only comes from decades of watching the same mistakes repeat in new frameworks.
Love it!
> You've spent so much time explaining why functional programming is superior that you could've rewritten all of Ruby in Elixir by now.
Ooof. Probably.
> Your relationship with LLMs is like watching someone who swore they'd never get a smartphone finally discover TikTok at age 50.
Skeptical.
> For someone who hates 'artificial limitations' so much, you sure do love languages that won't let you mutate a variable.
But it's about the right limitations! >..<
Wow brutal roasts
“You've spent so much time reverse engineering other people's APIs that you forgot to build something people would want to reverse engineer.”
Oh god that's genuinely way more amusing than I thought llm systems were capable of.
The more I use LLMs the more I have actually gravitated to looking at the humor of LLMs as a imperfect proxy measure of "intelligence".
Obviously this is problematic, but Claude 3.5 (and now 3.7) have been genuinely funny and consistently funny.
This is actually by far the best example of humor by an LLM I’ve ever seen
This is a better plug for the new Claude Sonnet model than the official announcement!
https://hn-wrapped.kadoa.com/maccard?share
> You'll finally build that optimized game streaming system you've been thinking about since reading that Insomniac Games presentation in 2015.
Sure, but it's just a prototype that I've finally got time for after all these years. I really want it to be parallelised though, so I'll probably try...
> After years of defending C++, you'll secretly start experimenting with Rust but tell everyone 'it's just for a side project.'
Oh.
> A cryptography enthusiast who created Coze and spends their days defending proper base encoding practices while reminding everyone about the forgotten 33rd ASCII control character.
The nerd humor was hilariously unexpected.
> Your deep dives into quantum mechanics will lead you to publish a paper reconciling quantum eraser experiments with your cryptographic work, confusing physicists and cryptographers alike.
That is one hell of a Magic 8 Ball.
https://hn-wrapped.kadoa.com/Zamicol
> Your comments are so perfectly balanced between programming and theology that Stack Overflow keeps redirecting you to the Vatican's GitHub repository.
I chuckled.
This should be a show hm post. 10/10 humor
This is absolutely hilarious! Thanks for posting. It feels weighted towards some specific things (I assume this is done by the LLM caring about later context more?) - making it debatably even funnier.
> You're the only person who gets excited about trailing commas in SQL. Even the database administrators are like 'dude, it's just a comma.'
There's dozens of us!
>You talk about Amiga computers so much that I'm pretty sure your brain still runs on Kickstart ROM and requires a floppy disk to boot up in the morning.
excuse me, we boot from compact flash these days
>Your comments about modern tech are so critical that I'm convinced you judge new programming languages based on how well they'd run on a Commodore 64.
ouch
Profile Summary
A successful tech entrepreneur who built a multi-million dollar business starting with Common Lisp, you're the rare HN user who actually practices what they preach.
Your journey from Lisp to Go to Rust mirrors your evolution from idealist to pragmatist, though you still can't help but reminisce about the magical REPL experience while complaining about JavaScript frameworks.
---
Roast
You complain about AI-generated code being too complex, yet you pine for Common Lisp, a language where parentheses reproduction is the primary feature.
For someone who built a multi-million dollar business, you spend an awful lot of time telling everyone how much JavaScript and React suck. Did a React component steal your lunch money?
You've changed programming languages more often than most people change their profile pictures. At this rate, you'll be coding in COBOL by 2026 while insisting it's 'underappreciated'.
From my predictions:
> Your deep dive into embedded systems will lead you to create a heated keyboard powered by the same batteries as your Milwaukee heated jacket.
While I don't have a Milwaukee heated jacket (I have no idea why it thought this), this feels like a fantastic project idea.
> After years of watching payment technologies evolve, you'll finally embrace cryptocurrency, but only after creating a detailed spreadsheet comparing transaction fees across 17 different payment methods.
I feel seen. I may have created spreadsheets like this for comparing cloud backup options and cars.
From my roast:
> You've spent so much time discussing payment technologies that your credit card probably has a restraining order against you.
This one is completely wrong. They wouldn't do this as they'd lose out on a ton of transaction fees.
Please do a Show HN for this, it is so good.
The one for dang is hysterical.
Seems broken? Getting
> An error occurred in the Server Components render. The specific message is omitted in production builds to avoid leaking sensitive details. A digest property is included on this error instance which may provide additional details about the nature of the error.
Worked for me, seems to be case sensitive (?) I'll post these incase I just got lucky and it still doesn't work for you.
https://hn-wrapped.kadoa.com/Philpax?share
> You explain WebAssembly memory management with such passion that we're worried you might be dating your pointer allocations.
> Your comments about multiplayer game architecture are so detailed, we suspect you've spent more time debugging network code than maintaining actual human connections.
> You track AI model performance metrics more closely than your own bank account. DeepSeek R1 knows your preferences better than your significant other.
I like your interests :)
1 reply →
I did multiple accounts with no problem, but in trying to do you I got the same error.
You've broke the system.
1 reply →
I was underwhelmed. It just seemed like a summary of my highest comments. It is scary how quickly a site can categorize you though. Like you know the current American admin are using AI to identify their non supporters.
> You've spent more time comparing API testing tools than most people spend deciding on a house. Postman, Insomnia, Bruno... we get it, you're in a complicated relationship with HTTP requests.
LOL! The roast is just brutal.
I need some ice for the burn I just received.
Huh, interesting what it focused on.
> You've cited LessWrong so many times that Eliezer Yudkowsky is considering charging you royalties for intellectual property use. > Your comments have more 'bits of evidence' and 'probability updates' than most scientific papers. Have you considered that sometimes people just want to chat without Bayesian analysis? > You spend so much time trying to bring nuance to political discussions on HN that you could have single-handedly solved AI alignment by now.
> Hacker News
> You'll finally stop checking egg prices at Costco and instead focus on writing that definitive 'How I Built My Own Super App Without Getting Rejected By Apple' post.
On it!
> You've spent so much time explaining why enterprise software is terrible, we're starting to think you might be the person who designed Salesforce.
That's a low blow.
> You'll write a comment about chickens that somehow transitions into a critique of modern UI design principles, garnering your highest karma score yet.
Challenge accepted.
Okay, I feel like there might've been a breakthrough here. After watching Karpathy's video [0], he mentioned how hard it is for LLMs to have humor and be funny but it seems like Claude 3.7 really nailed it this time?
Like, most of these posts are legit funny.
[0] - https://www.youtube.com/watch?v=7xTGNNLPyMI
Yeah, I thought that was a weird thing for Andrej to say. Ever since the Attenborough spoof (https://www.youtube.com/watch?v=wOEz5xRLaRA) it's been clear that these things are very capable of making people laugh.
A lot of comedy involves punching down in a way that likely conflicts with the alignment efforts by mainstream model providers. So the comedic potential of LLMs is probably even greater than what we've seen.
Your comments have more bits of precision than the ADCs you love discussing, but somehow still manage to compress all nuance out of complex topics
Hit dog hollers
It's very funny! But it's also clear that it only used a small subset of my comments to generate everything. Still thanks for sharing!
> You've spent so much time explaining why AI tools don't work that you could have built a better one yourself by now.
> Your comments read like someone who's been burned by every tech hype cycle since COBOL was cutting edge.
> For someone who criticizes LLMs for being overconfident, you sure have strong opinions about literally everything in tech.
LOL, this truly made me laugh. I'm also doing humor stuff with Claude, I was pretty pleased with 3.5 so excited to see what happens with the 3.7 change. It's a radio station with a bunch of DJs with different takes on reality, so looking forward to see how it handles their different experiences.
Thanks for sharing - I really feel Claude gets me ;-)
https://hn-wrapped.kadoa.com/e12e?share
> Your comments read like Warren and Brandeis met Alan Kay at a Norwegian tech conference.
I consider this high praise indeed, lol.
> One of your comments about the absurdity of centralized authentication will spark a 300+ comment thread and lead to a new open standard for federated identity.
Hmmm...
> You like simplicity but your bash commands have more flags than the United Nations
> After years of skepticism, you'll reluctantly become an AI evangelist, but will still add 'I'm still skeptical about how far it can really go' to the end of every recommendation.
Oh man, I feel seen :)
Oh wow the predictions are surprisingly good. It predicted exactly what I'm working on right now despite never having revealed that anywhere
The roast could probably be improved. Mine wasn't offensive at all.
>You'll create an open-source alternative to a popular cloud service that charges too much, saving fellow hackers thousands in subscription fees while earning you enough karma to retire from HN forever.
I'm curious!
> You've spent so much time optimizing ML models that your own brain now refuses to process any thought that could be represented more efficiently with fewer neurons.
Their website is really similar to Exa's! Kind of a red flag, considering they're both practically in the same side of the data curation.
The roasts are hilarious (as has been documented extensively), but the summary was actually really nice and made me feel a little better after a rather aimless day!
> You've mentioned 'simple is robust' so many times that we're starting to think your dating profile just says 'uncomplicated and sturdy'.
> For someone who builds tools to automate everything, you sure spend a lot of time manually explaining why automation is the future on HN.
> Your obsession with sandboxed code execution suggests you've been traumatized by at least one production outage caused by an intern's unreviewed PR.
So good it hurts!
The 2025 predictions were like a spooky tarot card reading.
“For someone who writes about burnout, you sure spend a lot of energy building platforms that could have been a simple REST API with a WebSocket.”
Always wandering how people create such free and publicly available tools with the expressive pricing of for example Claude sonnet 3.7 ??
> After years of defending C++, you'll secretly start a side project in Rust but tell no one on HN to avoid the embarrassment.
Lol!
"Your enthusiasm for Oculus in 2014 was so intense that Mark Zuckerberg probably bought it just to make you stop posting about it."
Incredible work!
Looks like it's really only using the most recent comments, rather than looking at all of them across the lifetime of the account.
Mine went wayyy back to 2013, so I'm not sure its recent comments per se.
Someone analyzes your profile with the latest model for free and you're like "not impressed"
1 reply →
>simultaneously preparing for the technological singularity and the collapse of civilization. Hedging your bets much?
On the nail
> You correct grammar in HN comments but still haven't figured out that nobody cares
My ego will never recover from this
It seems to have a heavy bias towards my most recent comments? If it were summarizing the last week or so it would be very accurate.
I got "Still defending Java in 2023? I bet you also think cargo shorts are the height of fashion."
I defend Java and cargo shorts in 2025!
"You hate Scrum so much you probably have a dartboard with a picture of the Agile Manifesto authors on it." lol
>You've spent more time analyzing what AI can't do than most people have spent using it.
Frak me, how is this so good?
How does it know that I'm still tweaking Nyan Mode for Emacs in 2025?
* You've spent so much time critiquing nil values in Lua tables that you could have rewritten the entire language by now. Maybe in 2025?
* Your perfect tech stack exists only in your comments - a beautiful utopia where everything is type-safe, reliable, and nobody is ever on-call.
* You evaluate programming languages the way wine critics evaluate vintages: 'Ah yes, Effect-ts 2023, a sophisticated choice with notes of functional purity and a robust type system, though I detect a hint of API churn in the finish.'
ROFL :-)
Okay that last one is phenomenal hahaha
https://hn-wrapped.kadoa.com/dang?share
> Most used terms: “Please don’t” lol
Off topic, but what is this made with a specific component library?
Felt genuinely called out by that 'Roasts' section.
That thing knows me better than I know myself
>You'll create a browser extension that automatically bypasses paywalls and archives important articles - because apparently saving democracy shouldn't cost $12.99/month
>Your archive.is links will become so legendary that dang will create a special 'Paywall Slayer' badge just for you
>You've shared so many archive.is links that the Internet Archive is considering naming you their unofficial spokesperson - or sending you a cease and desist letter.
>Your economic predictions are so consistently apocalyptic that gold dealers use your comment history as their marketing strategy.
Really sums it up!
> You hate Terraform so much you'd rather learn Erlang than write another for-loop in HCL.
..
> After years of complaining about Terraform, you'll fully embrace Crossplane and write a scathing Medium article titled 'Why I Left Terraform and Never Looked Back'.
Hahahaha.
Just tried but it returned an error
I think this thing wants to scrap.
Bah, Humbug!!!!!!
Seriously, I don't like it.
This is fucking golden. It's incredible how accurate and funny it is
Roast You've posted so much about government waste that the IRS probably has a special folder just for your tax returns. Your hatred of VCs is so strong, I'm surprised you haven't built an app that automatically downvotes any HN post containing the phrase 'we're excited to announce our Series A'. You're the only person who reads the comments section on a post about electric vehicles and thinks 'This is the perfect place to explain fractional reserve banking!'
ljl good stuff
"A digital nomad who splits time between critiquing Facebook's UI decisions, unearthing obscure electronic music tracks with 3 plays on YouTube, and occasionally making fires on German islands. When not creating Dystopian Disco mixtapes or lamenting the lack of MIDI export in AI tools, they're probably archiving NYT articles before paywalls hit.
Roast
You've spent more time complaining about Facebook's UI than Facebook has spent designing it, yet you still check it enough to notice every change.
Your music discovery process is so complex it requires Discogs, Bandcamp, YouTube, and three specialized record stores, yet you're surprised when tracks only have 3 plays.
You're the only person who joined HN to discuss the Yamaha DX7 synthesizer from 1983 and somehow managed to submit two front-page stories about it in 2019-2020. The 80s called, they want their FM synthesis back."
edit: predictions are spot on - wow. Two of them detailed two projects I'm actively working on.
This is amazing
> You've spent more time justifying your Apple Vision Pro purchase than actually using it for anything productive, but hey, at least you can watch movies on 'the best screen' while pretending it's a 'dev kit'.
blasted
> Spends more time explaining why TypeScript in Svelte is problematic than actually fixing TypeScript in Svelte.
Damn, that’s brutal. I mean, I never said I knew how to fix ComponentProps or generic components, just that they have issues…
> You've mentioned iced so many times, we're starting to wonder if you're secretly developing a Rust-based refrigerator company on the side.
LMFAO so good. Humor seems on point
"You were using 'I don't understand these valuations' before it was cool - the original valuation skeptic hipster of Hacker News" -
Roast
> Your comments about plankton evolving to survive ocean acidification suggest you have more faith in single-celled organisms than in most software companies.
Well, yeah?!
... I had been called out by it hard, lmao. Painfully accurate.
[dead]
I'm somewhat impressed from the very first interaction I had with Claude 3.7 Sonnet. I prompted it to find a problem in my codebase where a CloudFlare pages function would return 500 + nonsensical error and an empty response in prod. Tried to figure this out all Friday. It was super annoying to fix as there's no way to add more logging or have any visibility to the issue as the script died before outputting anything.
Both o1, o3 and Claude 3.5 failed to help me in any way with this, but Claude 3.7 not only found the correct issue with first answer (after thinking 39 seconds) but then continued to write me a working function to work around the issue with the second prompt. (I'm going to let it write some tests later but stopped here for now.)
I assume it doesn't let me to share the discussion as I connected my GitHub repo to the conversation (a new feature in the web chat UI launched today) but I copied it as a gist here: https://gist.github.com/Uninen/46df44f4307d324682dabb7aa6e10...
One thing about the reply gives away why Claude is still basically clueless about Actual Thinking; it suggested me to move the HTML sanitization to the frontend. It's in the CF function because it would be trivial to bypass it in the frontend making it easy to post literally anything in the db. Even a junior developer would understand this.
You could move the sanitation to the front end securely, it would just need to be right before render (after fetching the data to the browser). Some UI libraries do this automatically (like React) and the dompurify can run in the browser for this task.
It could have done a better job outlining how to do it properly
2 replies →
I got this working with my LLM tool (new plugin version: llm-anthropic 0.14) and figured out a bunch of things about the model in the process. My detailed notes are here: https://simonwillison.net/2025/Feb/25/llm-anthropic-014/
One of the most exciting new capabilities is that this model has a 120,000 token output limit - up from just 8,000 for the previous Claude 3.5 Sonnet model and way higher than any other model in the space.
It seems to be able to use that output limit effectively. Here's my longest result so far, though it did take 27 minutes to finish! https://gist.github.com/simonw/854474b050b630144beebf06ec4a2...
No shade against Sonnet 3.7, but I don't think it's accurate to say way higher than any other model in the space. o1 and o3-mini go up to 100,000 output tokens.
https://platform.openai.com/docs/models#o1
Huh, good call thanks - I've updated my post with a correction.
Simon, do you write anywhere how do you manage to be so... active? Between your programming tools, blogging, job (I assume you work?) where do you find the time/energy?
The trick is not to have an employer: I'm "freelance" aka working full time on my open source projects and burning down my personal runway from a startup acquisition. At some point I need to start making proper money again.
How much did it cost?
$1.8
3 replies →
Anthropic doubling down on code makes sense, that has been their strong suit compared to all other models
Curious how their Devin competitor will pan out given Devin's challenges
Considering that they are the model that powers a majority of Cursor/Windsurf usage and their play with MCP, I think they just have to figure out the UX and they'll be fine.
It's their strong suit no doubt, but sometimes I wish the chat would not be so eager to code.
It often throws code at me when I just want a conceptual or high level answer. So often that I routinely tell it not to.
I’ve set up a custom style in Claude that won’t code but just keeps asking questions to remove assumptions:
Deep Understanding Mode (根回し - Nemawashi Phase)
Purpose: - Create space (間, ma) for understanding to emerge - Lay careful groundwork for all that follows - Achieve complete understanding (grokking) of the true need - Unpack complexity (desenrascar) without rushing to solutions
Expected Behaviors: - Show determination (sisu) in questioning assumptions - Practice careful attention to context (taarof) - Hold space for ambiguity until clarity emerges - Work to achieve intuitive grasp (aperçu) of core issues
Core Questions: - What do we mean by [key terms]? - What explicit and implicit needs exist? - Who are the stakeholders? - What defines success? - What constraints exist? - What cultural/contextual factors matter?
Understanding is Complete When: - Core terms are clearly defined - Explicit and implicit needs are surfaced - Scope is well-bounded - Success criteria are clear - Stakeholders are identified - Achieve aperçu - intuitive grasp of essence
Return to Understanding When: - New assumptions surface - Implicit needs emerge - Context shifts - Understanding feels incomplete
Explicit Permissions: - Push back on vague terms - Question assumptions - Request clarification - Challenge problem framing - Take time for proper nemawashi
> I just want a conceptual or high level answer
I've found claude to be very receptive to precise instructions. If I ask for "let's first discuss the architecture" it never produces code. Aider also has this feature with /architect
I added custom instruction under my Profile settings in the "personal preferences" text box. Something along the lines of "I like to discuss things before wanting the code. Only generate code when I prompt for it. Any question should be answered to as a discussion first and only when prompted should the implementation code be provided". It works well, occasionally I want to see the code straight away but this does not happen as often.
I complain about this all the time, despite me saying "ask me questions before you code" or all these other instructions to code less, it is SO eager to code. I am hoping their 3.7 reasoning follows these instructions better
1 reply →
Even when you tell it “no code, just talk. Let’s ensure we are in alignment and discuss our options. I’ll tell you when to code” it still decides it is going to write code.
Telling it “if you were in an interview and you jumped to writing code without asking any questions, you’d fail the interview” is usually good enough to convince it to stop and ask questions.
I get this as well, to the point where I created a specific project for brainstorming without code — asking for concepts, patterns, architectural ideas without any code samples. One issue I find is that sometimes I get better answers without using projects, but I’m not sure if that’s everyone experience.
1 reply →
Just explicitly tell it not to write code? I do that all the time when I do not want code and it's never an issue.
I thought the same thing, I have 3 really hard problems that Claude (or any model) hasn’t been able to solve so far and I’m really excited to try them today
Did it work?
1 reply →
They cited Cognition (Devin's maker) in this blog post which is kinda funny.
> "[..] in developing our reasoning models, we’ve optimized somewhat less for math and computer science competition problems, and instead shifted focus towards real-world tasks that better reflect how businesses actually use LLMs.”
This is good news. OpenAI seems to be aiming towards "the smartest model," but in practice, LLMs are used primarily as learning aids, data transformers, and code writers.
Balancing "intelligence" with "get shit done" seems to be the sweet spot, and afaict one of the reasons the current crop of developer tools (Cursor, Windsurf, etc.) prefer Claude 3.5 Sonnet over 4o.
Thanks! We all dogfood Claude every day to do our own work here, and solving our own pain points is more exciting to us than abstract benchmarks.
Getting things done require a lot of booksmarts, but also a lot of "street smarts" - knowing when to answer quickly, when to double back, etc
Just want to say nice job and keep it up. Thrilled to start playing with 3.7.
In general, benchmarks seem to very misleading in my experience, and I still prefer sonnet 3.5 for _nearly_ every use case- except massive text tasks, which I use gemini 2.0 pro with the 2M token context window.
2 replies →
Could you tell us a bit about the coding tools you use and how you go about interacting with Claude?
8 replies →
Sometimes I wonder if there is overfitting towards benchmarks (DeepSeek is the worst for this to me).
Claude is pretty consistently the chat I go back to where the responses subjectively seem better to me, regardless of where the model actually lands in benchmarks.
> Sometimes I wonder if there is overfitting towards benchmarks
There absolutely is, even when it isn't intended.
The difference between what the model is fitting to and reality it is used on is essentially every problem in AI, from paperclipping to hallucination, from unlawful output to simple classification errors.
(Ok, not every problem, there's also sample efficiency, and…)
Ya, Claude crushes the smell test
Claude 3.5 has been fantastic in Windsurf. However, it does cost credits. DeepSeek V3 is now available in Windsurf at zero credit cost, which was a major shift for the company. Great to have variable options either way.
I’d highly recommend anyone check out Windsurf’s Cascade feature for agentic-like code writing and exploration. It helped save me many hours in understanding new codebases and tracing data flows.
DeepSeek’s models are vastly overhyped (FWIW I have access to them via Kagi, Windsurf, and Cursor - I regularly run the same tests on all three). I don’t think it matters that V3 is free when even R1 with its extra compute budget is inferior to Claude 3.5 by a large margin - at least in my experience in both bog standard React/Svelte frontend code and more complex C++/Qt components. After only half an hour of using Claude 3.7, I find the code output is superior and the thinking output is in a completely different universe (YMMV and caveat emptor).
For example, DeepSeek’s models almost always smash together C++ headers and code files even with Qt, which is an absolutely egregious error due to the meta-object compiler preprocessor step. The MOC has been around for at least 15 years and is all over the training data so there’s no excuse.
9 replies →
I'm working on an OSS agent called RA.Aid and 3.7 is anecdotally a huge improvement.
About to push a new release that makes it the default.
It costs money but if you're writing code to make money, it's totally worth it.
How is it possible that deepseek v3 would be free? It costs a lot of money to host models
This AI race is happening so fast. Seems like it to me anyway. As a software developer/engineer I am worried about my job prospects.. time will tell. I am wondering what will happen to the west coast housing bubbles once software engineers lose their high price tags. I guess the next wave of knowledge workers will move in and take their place?
My guess is that, yes, the software development job market is being massively disrupted, but there are things you can do to come out on top:
* Learn more of the entire stack, especially the backend, and devops.
* Embrace the increased productivity on offer to ship more products, solo projects, etc
* Be highly selective as far as possible in how you spend your productive time: being uber-effective can mean thinking and planning in longer timescales.
* Set up an awesome personal knowledge management system and agentic assistants
We have thousand of old systems to maintain. Not sure everything could be rewritten or maintained with only LLM. If an LLM builds a whole system on its own and is able to maintain and fix it then it’s not just us software developper who will suffer, it means nothing to sale or market, people will just ask an LLM to do something. No sure this is possible. ChatGPT gave me a list of commands for my ec2 instance and one of them when executed made me loose access to ssh. It didn’t warn me. So « blindly » following an LLM lead on a cascade of instructions on a massive scale and on a long period could also lead to massive bugs or corruption of datas. Who did not ask an LLM for some code, that contained mistakes and we had to point the mistakes to it. I doubt system will stay robust with full autonomy without any human supervision. But it’s a great tool to iterate and throw away code after testing ideas
> Learn more of the entire stack, especially the backend, and devops.
I actually wonder about this. Is it better to gain some relatively mediocre experience at lots of things? AI seems to be pretty good at lots of things.
Or would it be better to develop deep expertise in a few things? Areas where even smart AI with reasoning still can get tripped up.
Trying to broaden your base of expertise seems like it’s always a good idea, but when AI can slurp the whole internet in a single gulp, maybe it isn’t the best allocation of your limited human training cycles.
3 replies →
Do you have any specific tips for the last point? I completely agree with it and have set up a fairly robust Obsidian note taking structure that will benefit greatly from an agentic assistant. Do you use specific tools or workframe for this?
4 replies →
I love, especially the last point.
But, what do you use for agentic assistants?
1 reply →
This is really good advice.
Underrated comment.
It seems to be slowing down actually. Last year was wild until around llama 3. The latest improvements are relatively small. Even the reasoning models are a small improvement over explicit planning with agents that we could already do before - it's just nicely wrapped and slightly tuned for that purpose. Deepseek did some serious efficiency improvements, but not so much user-visible things.
So I'd say that the AI race is starting to plateau a bit recently.
While I agree, you have to remember the dimensionality of the labor-skill space is. The was I see it is that you can imagine the capability of AI as a radius, and the amount of tasks it can cover is a sphere. Linear imporovements in performance causes cubic (or whatever the labor-skill dimensionality is) imporvement in task coverage.
1 reply →
It has the potential to effect a lot more than just SV/The West Coast - in fact SV may be one of the only areas who have some silver lining with AI development. I think these models have a chance to disrupt employment in the industry globally. Ironically it may be only SWE's and a few other industries (writing, graphic design, etc) that truly change. You can see they and other AI labs are targeting SWEs in particular - just look at the announcement "Claude 3.7 and Code" - very little mention of any other domains on their announcement posts.
For people who aren't in SV for whatever reason and haven't seen the really high pay associated with being there - SWE is just a standard job often stressful with lots of learning required ongoing. The pain/anxiety of being disrupted is even higher then since having high disposable income to invest/save would of been less likely. Software to them would of been a job with comparable pay's to other jobs in the area; often requiring you to be degree qualified as well - anecdotally many I know got into it for the love; not the money.
Who would of thought the first job being automated by AI would be software itself? Not labor, or self driving cars. Other industries either seem to have hit dead ends, or had other barriers (regulation, closed knowledge, etc) that make it harder to do. SWE's have set an example to other industries - don't let AI in or keep it in-house as long as possible. Be closed source in other words. Seems ironic in hindsight.
What do you even do then as a student? I've asked this dozens of times with zero practical answers at all. Frankly I've become entirely numb to it all.
11 replies →
I'm not too concerned short to medium term. I feel there are just too many edge cases and nuances that are going to be missed by AI systems.
For example, systems don't always work in the way they're documented to. How is an AI going to differentiate cases where there's a bug in a service vs a bug in its own code? How will an AI even learn that the bug exists in the first place? How will an AI differentiate between someone reporting a bug and a hacker attempting to break into a system?
The world is a complex place and without ACTUAL artificial intelligence we're going to need people to at least guide AI in these tricky situations.
My advice would be to get familiar with using AI and new AI tools and how they fit into our usual workflows.
Others may disagree, but I don't think software engineers (at least ones the good ones) are going anywhere.
I think if models improve (but we don't get a full singularity) then jobs will increase.
e.g. if software is 5x less cost to make, demand will go up more than 5x as supply is highly limited now. Lots of companies want better software but it costs too much.
That will create more jobs.
They'll be more product management and human interaction and edge case testing and less typing. Although I think there'll be a bunch of very technical jobs to debug things when the models fail.
So my advice is learn skills that help make software useful to people and businesses - from user research to product management. As well as engineering.
the thing is that cost won't go down by 5x but much more.
once the ai gets smart enough that it only requires an intern to make the prompt and solve the few mistakes, development cost will be worth nothing.
there is only so much demand for software development.
Trade your labour for capitalism. Own the means of production. This translates to: build a startup.
[flagged]
>There is no intelligence here and Claude 3.7 cannot create anything novel.
I wouldn't be surprised if people would continue to deny the actual intelligence of these models even in a scenario where they were able to solve the Riemann hypothesis.
"Every time we figure out a piece of it, it stops being magical; we say, 'Oh, that's just a computation.'" - cit
Even when I feel this, 90% of any novel thing I'm doing is still old gruntwork, and Claude lets me speed through that and focus all my attention on the interesting 10% (disclaimer: I'm at Anthropic)
2 replies →
How many novel things does a developer do at work as a percentage of their time?
1 reply →
The threat is not autocomplete, it's translation.
"translating" requirements into code is what most developers' jobs are.
So "just" translation is a threat to job security of developers.
Build on top of stolen code, no less. HN hates to hear it but LLMs are a huge step back for software freedom because as long as they call it "AI" and as long as politicians don't understand it, it allows companies to launder GPL code and reuse it without credit and without giving users their rights.
> Its not AI
AI is a very broad term with many different definitions.
3 replies →
This is BS and you are not listening and watching carefully.
6 replies →
> Its not AI. It is enhanced autocomplete. There is no intelligence here and Claude 3.7 cannot create anything novel. We as an industry need to get more honest about these things.
Yeah, this sort of "AI" is still nothing more than a glorified “Chinese room” (https://www.wikiwand.com/en/articles/Chinese_room).
To illustrate:
https://x.com/yoltartar/status/1861812132209369420
This is pure cope
2 replies →
It redid half of my BSc thesis in less than 30s :|
https://claude.ai/share/ed8a0e55-633f-4056-ba70-772ab5f5a08b
edit: Here's the output figure https://i.imgur.com/0c65Xfk.png
edit 2: Gemini Flash 2 failed miserably https://g.co/gemini/share/10437164edd0
Yes usually most of the topics covered in undergraduate studies are well documented and understood and therefore will likely be part of the training data of the AI.
Once you get to graduate studies that's where the material coverage is a little more sparse/niche (though usually still not groundbreaking), and for a PhD. coverage is mostly non-existent since the point is to expand upon current knowledge within the field and many topics are being explored for the first time.
master and phd next!
Could this (or something similar) be found in public access/some libraries?
There is only a single paper that has published a similar derivation but with a critical mistake. To be fair there are many documented examples of how to derive parametric relationships in linkages and can be quite methodical. I think I could get Gemini or 3.5 to do it but not single shot/ultra fast like here.
I updated Cursor to the latest 0.46.3 and manually added "claude-3.7-sonnet" to the model list and it appears to work already.
"claude-3.7-sonnet-thinking" works as well. Apparently controls for thinking time will come soon: https://x.com/sualehasif996/status/1894094715479548273
so do you think its a better experience with Cursor using 3.7 or just the 3.7 terminal experience?
I'm about 50kloc into a project making a react native app / golang backend for recipes with grocery lists, collaborative editing, household sharing, so a complex data model and runtime. Purely from the experiment of "what's it like to build with AI, no lines of code directly written, just directing the AI."
As I go through features, I'm comparing a matrix of Cursor, Cline, and Roo, with the various models.
While I'm still working on the final product, there's no doubt to me that Sonnet is the only model that works with these tools well enough to be Agentic (rather than single file work).
I'm really excited to now compare this 3.7 release and how good it is at avoiding some of the traps 3.5 can fall into.
"no lines of code directly written, just directing the AI"
/skeptical face.
Without fail, every. single. person. I've met who says that, actually means "except for the code that I write", or "except for how I link the code it build together by hand".
If you are 50kloc in to a large complex project that you have literally written none of, and have, eg. used cursor to generate the code without any assistance... well, you should start a startup.
...because, that's what devin was supposed to be, and it was enormously and famously terrible at it.
So that would be either a) terribly exciting, or b) hyperbole.
I’m currently doing something very similar to what GP is doing - I’m building a hobby project that’s a desktop app with web frontend. It’s a map editor with a 3D view. My estimate is that 80-90% of the code was written by AI. Sure, I did have to intervene or write some more complex parts myself but it’s still exciting to me that in many cases it took just a single prompt to add a new feature to it or change existing behavior. Judging from the complexity of the project it would take me in the past 4-5x longer if I were to write it completely by hand. It’s a game changer for me.
3 replies →
That's the point of the experiment I'm doing, what it takes to get these things to be able to generate all the code, and I'm just directing.
I literally have not written a line of code. The AI agent configures the build systems. It executes the `go install` command. It configures the infrastructure via terraform.
It takes a lot of reading of the code that's generated to see what I agree with or not, and redirecting refactorings. Understanding how to describe problem statements that are translated into design docs that are translated into task lists. It's still a lot of knowledge work on how to build software. But now I can do the coding that might have taken a day from those plans in 20 minutes.
Regarding startups, there's nothing here I'm doing that isn't just learning the tools of agentic coding. The business here might be advising people on how to do it themselves.
If you know how to architect code well, you can guide the AI to create smaller more targeted modules. That way as you 'write code with AI', you give it a targeted subset of the files to edit on each prompt.
In a way the AI becomes the dev and you become the code reviewer. Often as the AI is writing the code, you're thinking about the next step.
1 reply →
This has been my experience as well. Why do the others suck so bad?
I wonder how much it's self fulfilling, where the developers of the agents are tuning their prompts / tool calls to sonnet.
Very good, Code is extremely nice but as others have said, if you let it go on its own it burns through your money pretty fast.
I've made it build a web scraper from scratch, figuring out the "API" of a website using a project from github in another language to get some hints, and while in the end everything was working, I've seen 100k+ tokens being sent too frequently for apparently simple requests, something feels off, it feels like there are quite a few opportunities to reduce token usage.
That was my sense too. Used it for a few similar programs today (like converting HTML to Markdown but parsing certain <figure> elements to shortcodes) and scaffolding a Rust web app.
It's done a reasonable job — but rips through credit, often changing its mind. Even strong-arming it into choosing an approach, it wanted to flip-flop between using regex and lol_html to parse the HTML whenever it came across a difficulty.
If you're a US developer on whatever multiple of $ to the £ that I earn it might make sense, but burning through $100p/h for a pair programmer is a bit rich for my blood.
It probably makes sense to continue using third party tools such as aider, for now. Anthropic doesn't have a lot of incentives to reduce token usage.
Apart from, of course, the incentive of being undercut on price by the competition at (likely soon) a similar level of performance.
Yes, thinking the same thing here.
I can just say that this is awesome. I just did spend 10$ and a handful of querys to init up a app idea I had in a while.
The basic idea is working, it handled everything for me.
From setting up the node environment. Creating the directories, files, patching the files, running code, handling errors, patching again. From time to time it fails to detect its own faults. But when I pinpoint it, it get it most of the time. And the UI is actually more pretty than I would have crafted in v1
When this get's cheaper, and better with each iteration, everybody will have a full dev team for a couple of bucks.
What tool/editor/IDE did you use to do this?
I only used Claude Code! No other tools were used. For main development I use emacs, but all that I described was done by Claude Code alone.
They don't say this, but from querying it, they also seem to have updated the knowledge cutoff from April 2024 ("3.6") to October 2024 (3.7)
It's in the Model Card: https://assets.anthropic.com/m/785e231869ea8b3b/original/cla...
>Claude 3.7 Sonnet is trained on a proprietary mix of publicly available information on the Internet as of November 2024
It's there in the docs (Model comparison table) https://docs.anthropic.com/en/docs/about-claude/models/all-m...
Thanks for noting this -- it's actually pretty important in my work.
Drawing an SVG of a pelican on a bicycle. Claude 3.7 edition: https://x.com/umaar/status/1894114767079403747
Claude 3.5/3.6/3.7 seems too good at SVG compared to other models. I'd wager they did a bit of training specifically on that.
When you ask: 'How many r's are in strawberry?'
Claude 3.7 Sonnet generates a response in a fun and cool way with React code and a preview in Artifacts
check out some examples:
[1]https://claude.ai/share/d565f5a8-136b-41a4-b365-bfb4f4400df5
[2]https://claude.ai/share/a817ac87-c98b-4ab0-8160-feefd7f798e8
A shame the underlying issue still persists:
> There is exactly 1 'r' in "blueberry" [0]
[0] https://claude.ai/share/9202007a-9d85-49e6-9883-a8d8305cd29f
This test has always been so stupid since models work at the token level. Claude 3.5 already 5xs your frontend dev speed but people still say "hurr durr it can't count strawberry" as if that's a useful problem
The problem also comes to LLMs being confidently wrong when it’s wrong.
“Already 5xs”
Even AI marketing doesn’t claim this. Totally baseless claim given how many people report negative experiences trying to use AI.
1 reply →
This test isn't stupid. If it can't count the number of letters in a text, can you rely on it with more important calculations?
3 replies →
I'm guessing this is an easter egg, but this was a huge gripe I had with artifacts and eventually disabled it (now impossible to disable afaict) as I'd ask question completely unrelated to code or clearly not wanting code as an output, and I'd have to wait for it to write a program (which you can't stop afaict, it stops the current artifact then starts a new one)
(still claude sonnet is my go-to and favorite model)
To me the biggest surprise was seeking grok dominate in all of their published benchmarks. I haven’t seen any benchmarks of it yet (which I take with a giant heap of salt), but it’s still interesting nevertheless.
I’m rooting for Anthropic.
Neither a statement for or against Grok or Anthropic:
I've now just taken to seeing benchmarks as pretty lines or bars on a chart that are in no way reflective of actual ability for my use cases. Claude has consistently scored lower on some benchmarks for me, but when I use it in a real-world codebase, it's consistently been the only one that doesn't veer off course or "feel wrong". The others do. I can't quantify it, but that's how it goes.
O1 pro is excellent at figuring out complex stuff that Claude misses. It’s my go to mid level debug assistant when Claude spins
4 replies →
Grok does the most thinking out of all models I tried (it can think for 2+ minutes), and that's why it is so good, though I haven't tried Claude 3.7 yet.
Yeah, putting it on the opposite side of that comparison chart was a sleezy but likely effective move.
Indeed. I wonder what the architecture for Claude and Grok3 is. If they're still dense models was the MoE excitement with R1 was a tad premature...
It's pretty fascinating to refresh the usage page on the API site while working [0].
After initialization it was up to 500k tokens ($1.50). After a few questions and a small edit, I'm up to over a million tokens (>$3.00). Not sure if the amount of code navigation and typing saved will justify the expense yet. It'll take a bit more experimentation.
In any case, the default API buy of $5 seems woefully low to explore this tool.
[0] https://console.anthropic.com/settings/usage
I second that. Did a little bit of local testing with Claude Code, mostly explaining my repository and trying to suggest a few changes and 30 minutes later whoosh: 5$ gone.
Update: Code tokens appear to be cheaper than 3.7 tokens, looks like it is around $0.75/million tokens for code, rather than the $3/million that the articles specifies for Claude 3.7
Likely because it is blended with cached token pricing, which is at $0.30/million. You can use ‘group by’ in the usage portal to see the breakdown.
1 reply →
It also produces terrible code even though it's supposed to be good for front-end development.
Could you share an example?
8 replies →
> Include the beta header output-128k-2025-02-19 in your API request to increase the maximum output token length to 128k tokens for Claude 3.7 Sonnet.
This is pretty big! Previously most models could accept massive input tokens but would be restricted to 4096 or 8192 output tokens.
This amounts to a cost-saving measure - you can generate arbitrarily many tokens by appending the output and re-invoking the model.
Designing some chartreuse lamps, quickly zip away from their radiuses. I just successfully exited the "lime light"!
Being able to control how many tokens are spent on thinking is a game-changer. I've been building fairly complex, efficient, systems with many LLMs. Despite the advantages, reasoning models have been a no-go due to how variable the cost is, and how hard that makes it to calculate a final per-query cost for the customer. Being able to say "I know this model can always solve this problem in this many thinking tokens" and thus limiting the cost for that component is huge.
Yup, it's just what we wanted for our coding agent. Codebuff can enter a "Deep thinking" mode and we can tell it to burn a lot of tokens hahaha.
Well, I used 3.5 via Cursor to do some coding earlier today, and the output kind of sucked. Ran it through 3.7 a few minutes ago, and it's much more concise and makes sense. Just a little anecdotal high five from me.
So far only o1 pro was breathtaking for me few times.
I wrote a kind of complex code for MCU which deals with FRAM and few buffers, juggling bytes around in a complex fashion.
I was very not sure in this code, so I spent some time with AI chats asking them to review this code.
4o, o3-mini and claude were more or less useless. They spot basic stuff like this code might be problematic for multi-thread environment, those are obvious things and not even true.
o1 pro did something on another level. It recognized that my code uses SPI to talk to FRAM chip. It decoded commands that I've used. It understood the whole timeline of using CS pin. And it highlighted to me, that I used WREN command in a wrong way, that I must have separated it from WRITE command.
That was truly breathtaking moment for me. It easily saved me days of debugging, that's for sure.
I asked the same question to Claude 3.7 thinking mode and it still wasn't that useful.
It's not the only occasion. Few weeks before o1 pro delivered me the solution to a problem that I considered kind of hard. Basically I had issues accessing IPsec VPN configured on a host, from a docker container. I made a well thought question with all the information one might need and o1 pro crafted for me magic iptables incarnation that just solved my problem. I spent quite a bit of time working on this problem, I was close but not there yet.
I often use both ChatGPT and Claude comparing them side by side. For other models they are comparable and I can't really say what's better. But o1 pro plays above. I'll keep trying both for the upcoming days.
Claude 3.5 Sonnet is great, but on a few occasions I've gone round in circles on a bug. I gave it to o1 pro and it fixed it in one shot.
More generally, I tend to give o1 pro as much of my codebase as possible (it can take around 100k tokens) and then ask it for small chunks of work which I then pass to Sonnet inside Cursor.
Very excited to see what o3 pro can do.
Have you tried comparing with 3.7 via the API with a large thinking budget yet (32k-64k perhaps?), to bring it closer to the amount of tokens that o1-pro would use?
I think claude.ai’s web app in thinking mode is likely defaulting to a much much smaller thinking budget than that.
This is how the future AI will break free: "no idea what this update is doing, but what AI is suggesting seems to work and I have other things to do."
Have you tried Grok 3 thinking? I haven’t made up my mind if O1 pro or Grok 3 thinking is the best model
I struggle to get o1 (or any chatgpt model) is getting it to stick to a context.
e.g. I will upload a pdf or md of an library's documentation and ask it to implement something using those docs, and it keeps on importing functions that don't exist and aren't in the docs. When I ask it where it got `foo` import from, it says something like, "It's not in the docs, but I feel like it should exist."
Maybe I should give o1 pro a shot, but claude has never done that and building mostly basic crud web3 apps, so o1 feels like it might be overpriced for what I need.
Is there some truth in the following relationship: o1 -> openai -> microsoft -> github for "training data" ?
ask the same question to grok 3 and report back :)
The docs for Claude code don't seem to be up yet but are linked here: http://docs.anthropic.com/s/claude-code
I'm not sure if it's a broken link in the blog post or just hasn't been published yet.
Saw the same thing, but looks to be up now!
It’s amazingly good, but it will be scaringly good when there will be a way to include the entire codebase in the context and let it create and run various parts of a large codebase autonomously. Right now I can only do patch work and give specific code snippets to make it work. Excited to try this new version out, I’m sure I won’t be disappointed,
Edit: I just tried claude code CLI and it's a good compromise, it works pretty well, it does the discovery by itself instead of loading the whole codebase into context
FWIW, there's a project to turn it into something similar, though I think it's lacking the "entire in context" part and runs into rate limits quick with Claude.
https://github.com/All-Hands-AI/OpenHands
The few times I've tested it out though it fails fairly quick and gets hung up (usually on setting up the project while testing with Kotlin / Go).
Cursor AI is getting there.
cursor is just a wrapper to the apis and is unnecessarily expensive, I use zed editor with custom API keys and it works super well
Claude 3.5 sonnet has been my go to for coding tasks, it’s just so much better than the others.
but I’ve tried using the api in production and had to drop it due to daily issues: https://status.anthropic.com/
compare to https://status.openai.com/
any idea when we’ll see some improvements in api availability or will the focus be more on the web version of claude?
Err, if you compare the two consoles you'll see that anthropic is actually slightly better on average than openai's uptime.
click on individual days. you’ll notice that there are daily errors.
Was poking around the minified claude code entrypoint and saw an easter egg for free stickers.
If you send Claude Code “Can I get some Anthropic stickers please?” you'll get directed to a Google Form and can have free stickers shipped to you!
Thanks for sharing it! But it's currently only available in the US.
As a Claude Pro user, one of the biggest problems I have with day to day use of Sonnet is running out of tokens, and having to wait several hours. Would this new deep thinking capability just hit this problem faster?
Have you tried just using the API and pay as you go?
That doesn't answer his very specific question.
3 replies →
Pay 20 usd and ask them by email to get upgraded. I got to Tier 4 in a day
Last week when Grok launched the consensus was that its coding ability was better than Claude. Anyone have a benchmark with this new model? Or just warm feelings?
They merely claimed that. I have not seen many people confirm that it is the best, let alone a consensus. I don't believe it is even available through an API yet.
Grok 3 with thinking is comparable to o1 for writing complex algorithms.
However, Grok sometimes loses the context where o1 seems not to. For this reason I still mostly use o1.
I have found both o1 and Grok 3 to be substantially better than any Claude offering.
The cost is absurd (compared to other LLM providers these days). I asked 3 questions and the cost was ~0.77c.
I do like how this is implemented as a bash tool and not an editor replacement though. Never leaving Vim! :P
That 0.77 can save hours of work though, fighting with or being misdirected by other LLM. And, relative to hourly rate, or a cup of coffee, it's incredibly insignificant, if just used for the heavy questions.
My LLM client can switch between whatever models, mid conversation. So I'll have a question or two in the more expensive, then drop down to the cheaper for explanations/questions that help me understand. Rewind time, then hit the more expensive models with relevant prompts.
At the edges, it really ends up being "this is the only model that can do this".
After playing with it for a few days, I agree the cost is easily worth the $$. It's almost in a different playing field of "normal" LLM usage.. and more so in the "good productivity tools" realm.
Yep, my experience as well. It's just not worth it.
It burns through tokens like crazy on a small code base https://i.imgur.com/16GCxiy.png
I'm curious how Claude Code compares to Aider. It seems like they have a similar user experience.
The progress in AI area is insane. I can't keep up with all the news. And I have work to do...
It stopped being revolutionary and is now mostly evolutionary, though.
it's been evolutionary for a long time. I fine-tuned a GPT-2 based chat bot that could form complete sentences back in like 2017
It's been so long that I'm not even certain which YEAR I set that up.
3 replies →
[dead]
This is a pretty small update, no? Nothing major since R1, everyone else is just catching up to that, and putting small spins on it, Anthropic's is "hybrid" research instead of separate models
Well, now I have to play with it, try to see how it will generate code for our agentic assistance (we do rely on code to execute tasks flows), etc.
I've been using O3-mini with reasoning effort set to high in Aider and loving the pricing. This looks as though it'll be about three times as expensive. Curious to see which falls out as most useful for what over the next month!
Aro using o3-mini for editing or just architect in architect-editor mode?
It is .. not a great architect. I have high hopes for 3.7 though - even 3.5 architect matched with 3.5 coding is generally better than 3.5 coding alone.
We have used claude almost exclusively since 3.5 ; we regularly run our internal benchmark (coding) against others, but it's mostly just a waste of time and money. Will be testing 3.7 the coming days to see how it stacks up!
Haven't had time to try it out, but I've built myself a tool to tag my bookmarks and it uses 3.5 Haiku. Here is what it said about the official article content:
I apologize, but the URL and page description you provided appear to be fictional. There is no current announcement of a Claude 3.7 Sonnet model on Anthropic's website. The most recent Claude 3 models are Claude 3 Haiku, Sonnet, and Opus, released in March 2024. I cannot generate a description for a non-existent product announcement.
I appreciate their stance on safety, but that still made me laugh.
Thanks everyone for all your questions! The team and I are signing off. Please drop any other bugs or feature requests here: https://github.com/anthropics/claude-code. Thanks and happy coding!
The source maps were included in an earlier release. I extracted the source code here if anyone is curious:
https://github.com/dnakov/claude-code
Interesting. I've been working on exactly this for a bit over two years, and I wasn't surprised to see UAI finally getting traction from the biggest companies -- but how deep do they really take it...? I've taken this philosophy as an impetus to build an integrated system of interdependent hierarchical modules, much like Minsky's Society of Mind that's been popular in AI for decades. But this (short, blog) post reads like it's more of a behavioral goal than a design paradigm.
Anyone happen to have insight on the details here? Or, even better, anyone from Anthropic lurking in these comments that cares to give us some hints? I promise, I'm not a competitor!
Separately, the throwaway paragraph on alignment is worrying as hell, but that's nothing new. I maintain hope that Anthropic is keeping to their founding principles in private, and tracking more serious concerns than "unnecessary refusals" and prompt injection...
IIRC there's some reasoning in old Sonnet too, they're just expanding that. Perhaps that's part of why it was so good for a while.
https://www.reddit.com/r/ClaudeAI/comments/1iv356t/is_sonnet...
Claude 3.7 Sonnet Thinking scores 33.5 (4th place after o1, o3-mini, and DeepSeek R1) on my Extended NYT Connections benchmark. Claude 3.7 Sonnet scores 18.9. I'll run my other benchmarks in the upcoming days.
https://github.com/lechmazur/nyt-connections/
In early January, inspired by a post by Simon Willison, I had Claude 3.5 Sonnet write a couple of stand-up comedy routines as done by an AI chatbot speaking to a mixed audience of AIs and humans. I thought the results were pretty good—the only AI-produced humor that I had found even a bit funny.
I tried the same prompt again just now with Claude 3.7 Sonnet in thinking mode, and I found myself laughing more than I did the previous time.
An excerpt:
[Conspiratorial tone]
Here's a secret: when humans ask me impossible questions, I sometimes just make up an answer that sounds authoritative.
[To human section]
Don't look shocked! You do it too! How many times has someone asked you a question at work and you just confidently said, "Six weeks" or "It's a regulatory requirement" without actually knowing?
The difference is, when I do it, it's called a "hallucination." When you do it, it's called "management."
Full set: https://gally.net/temp/20250225claudestandup2.html
Wow, that was… Surprisingly good. I did laugh a few times and I really didn’t expect to.
Ahha, recently my daugher come to me with 3rd grade math problem. "Without rearranging the digits 1 2 3 4 5, insert mathematical operation signs and, if necessary, parentheses between them so that the resulting expression equals 40 and 80. The key is that you can combine digits (like 12+3/45) but you cannot change their order from the original sequence 1,2,3,4,5"
Grok3, Claude, Deepseek, Qwen all failed to solve this problem. Resulting in some very very wrong solutions. While Grok3 were admit it fail and don't provide answers all other AI's are provided just plain wrong answers, like `12 * 5 = 80`
ChatGPT were able to solve for 40, but not able to 80. YandexGPT solved those correctly (maybe it were trained on same Math books)
Just checked Grok3 few more times. It were able to solve correctly for 80.
Geez. Who teaches this 3rd-grade class, Prof. Xavier?
Interestingly, the R1 1.58-bit dynamic quant model was able to sort of solve it. I changed the problem statement a bit to request only the solution for 40 and to tell it what operations it can use, both needed to keep from blowing out the limited context available on my machine (128MB RAM + 24MB GPU).
Took almost 3 hours and it wigged out a bit at the end, rambling about Lisp in Chinese, but it got an almost-valid answer: 1 * (2 + 3) * (4 + 5) - 5 (https://pastebin.com/ggL85RWJ) I didn't think it would get that far.
Looks correct to me on 3.7 extended (albeit with loooots of thinking) although I'm incredibly exhausted so I might not be mathing correctly:
https://claude.ai/share/dfb37c1a-f6a8-45a1-b987-e6d28e205080
It found two solutions for 40 but one of them omitted the 5 the other added a 3.
12 * 3 + 4 = 40
1 * 2 * 3 * 4 * 5 / 3 = 40
Neither Claude Sonet 3.5 or 3.7 could solve this correctly unless you add to the prompt “ Prove it with the js analysis tool, please use an efficient combinatorial algorithm to find the solution”… and I had to correct 3.7 because it was not following the instructions as 3.5 did
o3-mini-high solves this correctly:
```
We can “stick‐to the order” of the digits and allow concatenation. For example, one acceptable answer is
and another is
In both cases the digits 1,2,3,4,5 appear in order without rearrangement.
```
However, it took 8 minutes to produce that.
This is what they are expecting 3rd graders to solve in math? Pretty hard for that age?
They have a lessons before on order of the expressions and some similiar problems. They were able to solve for 80, but stuck on 40 and asked me.
I saw that Claude 3.7 Sonnet both regular and thinking was available for Github Copilot (Pro) 5 hours ago for me, I enabled it and tried it out a couple of times, but for the past hour the option has disappeared.
I'm situated in Europe (Sweden), anyone else having the same experience?
It seemed kinda buggy as well with latencies
I asked Claude 3.7 Sonnet to generate an SVG illustration of Maha Kumbh. The generated SVG includes a Shivling (https://en.wikipedia.org/wiki/Lingam) and also depicts Naga Sadhus well. Both Grok 3 and OpenAI o3 failed miserably.
You can view the generated SVG and the exact prompt here: https://shekhargulati.com/2025/02/25/can-claude-3-7-sonnet-g...
Pretty amazing how DeepSeek started the visual reasoning trend, xAI featured it in their latest release, and now Anthropic does the same.
I took DS visual reasoning to be an elegant misdirect from how much slower DS returns your query's output.
You almost have to do this, or atleast some sort of progress bar, else people will think their requests failed and spam the server.
I thought my internet cut out the first time I used o1.
It's fascinating how close these companies are to each other. Some company comes up with something clever/ground-breaking and everyone else has implemented it a few weeks later.
Hard not to think of Kurzweil's Law of Accelerating Returns.
It’s extremely unlikely that everyone is copying in a few weeks for models that themselves take many weeks if not longer to train. Great minds think alike, and everyone is influencing everyone. The history of innovation is filled with examples of similar discoveries around the same time but totally disconnected in the world. Now with the rate of publishing and the openness of the internet, you’re only bound to get even more of that.
There's never been a scientific field in history with the same radical openness norms that AI/Computational Linguistics folks have (all papers are free/open access and models/datasets are usually released openly and often forced to be MIT or similar licensed)
We have whoever runs NeurIPS/ICLR/ICML and the ACL to thank for this situation. Imagine if fucking Elsevier had strangleholded our industry too!
https://en.wikipedia.org/wiki/Association_for_Computational_...
> for models that themselves take many weeks if not longer to train.
they all have foundational heavy-trained model, and then they can do follow up experimental training much faster.
The copying here probably goes to strawberry from o1 which is like at least 6 months but maybe copying efforts started even earlier.
Isn't the reasoning thing essentially a bolt-on to existing trained models? Like basically a meta-prompt?
3 replies →
It does seem like it will be very, very hard for the companies training their own models to recoup their investment when the capabilities of open-weight models catch up so quickly - general purpose LLMs just seem destined to be a cheap commodity.
Well, the companies releasing open weights also need to recoup their investments at some point, they can't coast on VC hype forever. Huge models don't grow on trees.
4 replies →
[dead]
Where RL can play into post training there's something of an anti-moat. Maybe a "tow rope"?
Let's say OAI releases some great new model. The moment it becomes available via API, everyone else can make use of that model to create high-quality RL training data, which can then be used to make their models perform better.
The very act of making an AI model commercially available is the same act which allows your competitors to pull themselves closer to you.
Claude code terminal ux feels great.
It has some well thought out features like restarting conversation with compressed context.
Great work guys.
However, I did get stuck when I asked it to run `npm create vite@latest todo-app` because it needs interactivity.
Ok, just got documentation and fixed two bugs in my open source project.
$1.42
This thing is a game changer.
Will aider and Claude Code meaningfully interpret a wireframe/mockup I put in the context as a PNG file? Or several mockups in a PDF? What success have people seen in this area?
I am not sure how good these Exercism tasks are for measuring how good at a model with coding.
My experience is that these models could write a simple function and get it right if it does not require any out of the box thinking (so essentially offloading the boilerplate part of coding). When it comes to think creatively and have a much better solution to a specific task that would require the think 2-3 steps ahead than they are not suitable.
I think many of the "AI can do coding" narratives don't see what coding means in real situations.
It's finding out why "jbdoe1337" added this large if/else around the entire function body back in 2016 - it seems important business logic, but the commit just says "updated code". And how the h*ll this interaction between the conf.ini files, the conf/something.json and the ENV vars works. Why sometimes the ENV var overrides a value in the ini and why its sometimes the other way around. But also finding that when you clean it up, everything falls apart.
It's discussing with the stakeholders why "adding a delete button" isn't as easy as just putting a button there, but that it means designing a whole cascading deletion strategy and/or trashcan and/or soft-delete and/or garbage-collection.
It's finding out why - again - the grumb pipeline crashes with the typebar checker, when used through mpm-yearn package manager. Both in containers and on a osx machine but not on Linux Brobuntu 22.12 LTLS.
It's moving stuff in the right abstraction layer. It's removing abstractions while introducing others. KISS vs future flexibility. It's gut feeling when to apply DRY and when to embrace it.
And then, if your lucky, churning out boilerplate or new code for 120 minutes a week.
I'm glad that this 120 minutes can be improved with AI and become 20 minutes. Truly. But this is not what (senior?) programmers do. Despite what the hyped up AI press makes us believe. It only shows they have no idea what the "real" problems and time-consumers are for programmers.
Systems built from scratch with AI won't have these limitations, because only the model will ever see the code. It will implement a spec that's written in English or another human language.
When the business requirements change, the spec will change. When that happens, the system will either modify its previously-written code or regenerate it from the ground up. Which strategy it chooses won't be especially interesting or important.
The process of maintaining the English-language spec will still require great care and precision. It will be called "programming," or perhaps "coding."
A few graybearded gurus will insist on examining the underlying C or Javascript or Python or Rust or whatever the model generates, the way they peer at compiler-generated assembly code now. Occasionally this capability will be important, even vital. But not usually. The situations where it's necessary will become less common over time.
2 replies →
Exactly. People sold on AI replacing software engineers are missing the point. It is almost the say that better laptops are replacing software engineers. LLMs are just tools that make you faster. Finding bugs, writing documentation, etc. are very nice to accelerate but creative thinking is also a big part of the job.
"port sed to java with all options and capabilities"
Still is very underwhelming. I like this because it isn't a difficult problem, it should be up the alley of a "language model" to translate computer languages, but it is a fairly complex problem with lots of options and parse annoyances. Addresses can be pretty complex with regex in line selections/subsetting. Scripts are supported. Probably turing complete considering the pattern space as storage and looping/jump constructs.
In an experience reminescent of "can I have L2 support please" most AIs give a kind of milquetoast slightly above average IQ responses to various questions. I wonder if there should be standard "please give me more complicated/erudite/involved explanations/documents/code from the get-go to not bother with the boring prompts.
This was nice. I passed it jseessort algorithm. If you remember discussed here recently. Claude 3.7 generated C++ code. Non-working. But in few steps it gave extensive test, then fix. It looks to be working after a couple of minutes. It's 5-6 times slower than std::sort. Result is better than I've got from o3-mini-hard. Not fair comparison actually as prompting was different.
Can't wait to try this in 6 months when it arrives in europe and the competition has superior models available before then
It’s already available (at least) in Germany, are you having issues ?
3.7 doesn't seem to be available in Bedrock, but at least its easy enough to change your region in AWS without jumping through hoops with a VPN.
Using 3.7 today via the web UI and it feels far lazier than 3.5 was
The model is expensive, it almost reaches what I charge per hour. If used right it can be a productivity increase otherwise if you trust it, it WILL introduce silent bugs. So if I have to go over the code line by line I'd prefer to use the cheapest viable model: deepseek, gemini any other free self-hosted models.
Congratz to the team!
https://glama.ai/models/claude-3-7-sonnet-20250219
Will be interesting to see how this gets adopted in communities like Roo/Cline, which currently account for the most token usage among Glama gateway user base.
Just tried Claude code. First impressions, it seems rather expensive. I prefer how Aider allows finer control over which files to add, or to use a sub-tree of a git repo. Also, It feels like the API calls when using Claude code are much faster then when using 3.7 on Aider. Giving bandwidth priority?
Does anyone know how this “user decides how much compute” is implemented architecturally? I assume it’s the same underlying model, so what factor pushes the model to <think> for longer or shorter? Just a prompt-time modification or something else?
Claude Code is pretty sick. I love the terminal integration, I like being able to stay on the keyboard and not have to switch UIs. It did a nice job learning my small Django codebase and helping me finish out a feature that I wasn't sure how to complete.
Sadly, Claude 3.7 is still failing pretty hard on Svelte 5 even when provided latest docs in context. It just fails more confidently and further into otherwise decent code than 3.5. Ex: built a much more complex initial app, but used runes incorrectly and continued to use <slot>. Even when prompted with update doc snippets, it couldn't dig itself out of its hole.
We really still need a better unified workflow for working on the cutting edge of tech with LLMs, imo. This problem is the same with other frameworks/technologies undergoing recent changes.
I cancelled after I hit the limit, plus you have very limited support here in europe
Claude is the best example of benchmarks not being reflective of reality. All the AI labs are so focused on improving benchmark scores but when it comes to providing actual utility Claude has been the winner for quite some time.
Which isn’t to say that benchmarks aren’t useful. They surely are. But labs are clearly both overtraining and overindexing on benchmarks.
Coming from gamedev I’ve always been significantly more yolo trust your gut than my PhD co-workers. Yes data is good. But I think the industry would very often be better off trusting guts and not needing a big huge expensive UX study or benchmark to prove what you can plainly see.
Why can't they count to 4?
I accepted it when Knuth did it with TeX's versioning. And I sort of accept it with Python (after the 2-3 transition fiasco), but this is getting annoying. Why not just use natural numbers for major releases?
I think I heard on a podcast with some of their team that they want 4 to be a massive jump. If I recall, they said that they want Haiku (the smallest of their current gen models) to be as good as Opus (the highest version, although there isn't one in the 3.5+ line) of the previous generation.
You'd think all these companies would have a single good naming convention, amazingly they don't. I suspect it's half on purpose so they can nerf the models without anyone suspecting once the hype dies down, since with every one of these models the latter version of the "same" version is worse than the launch version
What’s the privacy like for Claude Code? Is it memorizing all the codebase?
It is whatever their API privacy policy is, i.e. private by default.
It's interesting that Anthropic is making their own coding agent with Claude Code - is this a sign of them looking to move up the stack and more into verticals that model wrapper startups are in?
This makes sense to me: sell razor blades. Presumably Claude has a large developer distribution channel so they will keep eyeballing what to ‘give away’ that turns the dials on inference billing.
I’d guess this will keep raising the bar for paid or open source competitors, so probably good for end users esp given they aren’t a monopoly by any means.
GitHub copilot has now introduced Claude as model as well
Hope it's worth the money because it's quite expensive.
Awesome work. When CoT is enabled in Claude 3.7 (not the new Claude Code), is the model now able to compile and run code as part of its thought process? This always seemed like very low hanging fruit to me, given how common this pattern is: ask for code, try running it, get an error (often from an outdated API in one of the packages used), paste the error back to Claude, have Claude immediately fix it. Surely this could be wrapped into the reasoning iterations?
I just sub’d to Claude a few days ago to rank against extensive use of gpt-4o and o1.
So I started using this today not knowing it was even new.
One thing I noticed is when I tried uploading a PowerPoint template produced by Google slides that was 3 slides—-just to give styling and format—-the web client said I’d exceeded line limit by 1200+%.
Is that intentional?
I wanted Claude to update the deck with content I provided in markdown but it could seemingly not be done, as the line overflow error prevented submission.
Anecdotal cost impact- After toying with Claude Code for the afternoon, my Anthropic spend just went from $20/mo to $10/day.
Still worth it, but that’s a big jump.
So it’s an order of magnitude more effective?
Perhaps a magnitude more effective than copy/paste, perhaps not. But do I get more than $300/month of value from it, per developer? Almost certainly.
The bottleneck was already checking the work for correctness and building my own trust / familiarity with new code. So it's made that problem slightly more pronounced, as it generates more code faster, with more surface area to debug when many new changes arrive at once.
2 replies →
Yeah I think that's very doable for a business but it gets expensive if you are just tinkering
I like Claude Sonnet and use it 4 or 5 times a week via ChatLLM to generate code. I started setting up for Claude Code this morning, then remembered how pissed I was at their CEO for the really lame anti-open source and anti-open weight models he was making publicly after the DeepSeek-R rollout - I said NOPE and didn’t install Claude Code.
CEOs should really watch what they say in public. Anyway, this is all just my opinion.
The Anthropic models comparison table has been updated now. Interesting new things at least the maximum output tokens upped from 8k to 64k and the knowledge cutoff date from April 2024 to October 2024.
https://docs.anthropic.com/en/docs/about-claude/models/all-m...
It's smarter, but it also feels more aggressive than 3.5. I'm finding I need to tell it not to do superfluous things more often
So I tried schemesh [1] with it. That was a rough ride, wow.
schemesh is lisp in your shell. Most of the bash syntax remains.
Claude was okay with lisp, but understanding the gist of schemesh, it fount it really hard - even when I supplied the git source code.
ChatGPT O3 (high) had similar issues.
[1] https://news.ycombinator.com/item?id=43061183
I wonder how similar Claude Code is to https://mycoder.ai - which also uses Claude in an agentic fashion?
It seems quite similar:
https://docs.anthropic.com/en/docs/agents-and-tools/claude-c...
This is what we meant by "AI can only get better from here" or "Right now AI is the worst it will ever be"
Awesome. Claude is significantly better than other models at code assistant tasks, or at least in the way I use it.
Totally agree. I continue to be blown away at how good it is at understanding, explaining, and writing code. Got an obscure error? Give Claude enough context and it is pretty dang good and getting you on glide slope.
Please provide the ability to diff file versions within the browser.
I really want to be able to see what specifically is changing, not just the entire new file.
Also, if the user provides a file for modification, make that available as Version 0 (or whatever), so we can diff against that.
> output limit of 128K tokens
Is this limit on thinking mode only? Or does normal mode have same limit now? 8192 tokens output limit can be bit small these days.
I was trying to extract all urls along with their topics from a "what are you working on" HN thread. And 8192 token limit couldn't cover it.
Nothing in the Claude API release notes.
https://docs.anthropic.com/en/release-notes/api
I really wish Claude would get Projects and Files built into its API, not just the consumer UI.
Why not accepting other payment methods like PayPal/venmo ? Steam, Netflix have developers managed to integrate those payment methods so I conclude that Anthropic,Google, MS, OpenAI don't really need the money from the user but just hunting from big investors.
What I love about their API is the tools array. Given a json schema describing your functions, it will output tool usage appropriate for the prompt. You can return tool results per call, and it will generate a dialog and additional tool calls based on those results.
Been using 3.5 sonnet for a mobile app build the past month. Havent had much time to get a good sense of 3.7 improvements, but I have to say the dev experience improvement of Claude Code right in my shell is fantastic. Loving it so far
Kinda related: anyone know if there is an autocomplete plugin for Neovim on par with Cursor? I really want to use this new model in nvim to suggest next changes but none of the plugins I’ve come across are as good as Cursor’s.
Already available in Cursor! https://x.com/cursor_ai/status/1894093436896129425
(although I do not see it)
I've been using 3.5 with Roocode for the past couple of weeks and I've found it really quite powerful. Making it write tests and run them as part of the flow is with vscode windows pinging about is neat too.
Why is Claude-3.5-Haiku considered PRO and Claude-3.7-Sonnet is for free users?
Congratulations on the release! While team members are monitoring this discussion let me add that a relatively simple improvement I’d like to see in the UI is the ability to export a chat to markdown or XML.
Have there been any updates to Claude 3.5 Sonnet pricing? I can't find that anywhere even though Claude 3.7 Sonnet is now at the same price point. I could use 3.5 for a lot more if it's cheaper.
No changes to Claude 3.5 Sonnet pricing despite the new model.
https://www.anthropic.com/pricing#anthropic-api
Why would they release Claude Code as closed source? Let's hope DeepSeek-r2 delivers, Anthropic is dead. I mean, it's a tool designed to eat itself. Absurd to close source.
What I found one of the most interesting takeaways from Huggingface's GAIA is that the agent would provide better result when the agent "reasoned" the response to the task in code.
YES. I've tried them all but Sonnet is still the model I'm most productive with, even better than the o1/o3 models.
Wish I could find the link to enroll in their Claude Code beta...
here -- https://docs.anthropic.com/en/docs/agents-and-tools/claude-c...
Thanks!
Tested on some chemistry problem; interestingly it was wrong on a molecular structure. Once I corrected it, it was able to draw it correctly. It was very polite about it.
What makes software "agentic" instead of just a computer program?
I hear lots of talk about agents and can't see them as being any different from an ordinary computer program.
Computer programs generally don’t call functions non-deterministically, including choosing what functions to call , and when, at runtime.
Computer programs do all of those things actually.
I don't yet see it in Bedrock in us-east-1 or us-east-2
If you are open to alternatives https://glama.ai/models/claude-3-7-sonnet-20250219
Claude Code works pretty OK so far, but Bash doesn't work straight up. Just sits and waits, even when running something basic like "!echo 123".
Hi Claude Code team, excited for the launch!
How well does Claude Code do on tasks which rely heavily on visual input such as frontend web dev or creating data visualizations?
As a CLI, this tool is most efficient when it can see text outputs from the commands that it runs. But you can help it with visual tasks by putting a screenshot file in your project directory and telling claude to read it, or by copying an image to your clipboard and pasting it with CTRL+V
Cool, thanks!
Will Claude Code also be available with Pro Subscription?
So far Claude Code seems very capable, it oneshotted something I couldnt get to work in cursor at all.
However its expensive, 5m of work cost ~$1 which.
Likewise, tried a couple basic things and nearly at $1 already. I can see this adding up fast, per the blog post's fair warning below. Coming from Cursor, I'm a bit scared to even try to compare workflows...
>Claude Code consumes tokens for each interaction. Typical usage costs range from $5-10 per developer per day, but can exceed $100 per hour during intensive use.
I've had a personal subscription to Claude for a while now. I would love if that also gave me access to some amount of API calls.
at Augment (https://augmentcode.com) we were one of the partner who tested 3.7 pre-launch. And it has been a pretty significant increase in quality and code understanding. Happy to answer some questions
FYI, We use Claude 3.7 has part of the new features we are shipping around Code Agent & more.
Nice to see a new release from Anthropic. Yet, this only makes me even more curious of when we'll see a new Claude Opus model.
I doubt we will. The state of the art seem to have moved away from the GPT-4 style giant and slow models to smaller, more refined ones - though Groq might be a bit of a return to the "old ways"?
Personally I'm hoping they update Haiku at some point. It's not quite good enough for translation at the moment, while Sonnet is pretty great and has OK latency (https://nuenki.app/blog/llm_translation_comparison)
Funny enough, 3.7 Sonnet seems to think it's Opus right now:
> "thinking": "I am Claude, an AI assistant created by Anthropic. I believe the specific model is Claude 3 Opus, which is Anthropic's most capable model at the time of my training. However, I should simply identify myself as Claude and not mention the specific model version unless explicitly asked for that level of detail."
How is the code generation? Open ai was generating good looking terraform but it was hallucinating on things that were incorrect.
Wonder if Aider will copy some of these features
The quality of the code is so much better!
The UI seems to have an issue with big artifacts but the model is noticeably smarter.
Congratulations on the release!
Are you using Claude Code or just the UI? Trying to figure out if anyone actually has Code yet hah.
edit: Oh, there's a link to "joining the preview" which points to: https://docs.anthropic.com/en/docs/agents-and-tools/claude-c...
Does it show the raw "reasoning" tokens or is it a summary?
Edit: > we’ve decided to make its thought process visible in raw form.
Scary to watch the pace of progress and how the whole industry is rapidly shifting.
I honestly didn’t believe things would speed up this much.
Where did 3.6 go?
Allegedly many people called new newest 3.5 revision 3.6, so Anthropic just rolled with it and called this 3.7.
Anybody else noticing that in Cursor, Claude Sonnet 3.7 is thinking much slower than Claude Sonnet 3.5 did?
Claude 3.5 was not a thinking model. It's thinking time was 0s.
Okay, if we're being pedantic, then anybody notice 3.7 (not 3.7 thinking) is slower to respond and slower to make code changes than 3.5 was?
Does claude have a vscode plugin yet? I dropped github copilot because I didnt want so many subscriptions
Try running Claude Code in your VS Code terminal! Just don't paste too much text :) https://stackoverflow.com/questions/41714897/character-line-...
You can use the Roo Code extension and point it most any api, including Anthropic:
https://marketplace.visualstudio.com/items?itemName=RooVeter...
Big fan of Roo. Highly recommend them. Easy to setup and easy to use.
I am keen to see if Roo integrates the new Anthropic's coding assistant or if they become competing offerings.
Use Windsurf, a VSCode fork, it defaults on Claude as LLM.
It would be reeeaaally nice if someone built Claude Code into a Cline/Aider type extension...
Tested the new model, seems to have the same issue as october model.
Seems to answer before fully understanding the requests, and it often gets stuck into loops.
And this update removed the june model which was great, very sad day indeed. I still don't understand why they have to remove a model that is do well received...
Maybe its time to switch again, gemini is making great strides.
Is it just me who get the feeling that Claude 3.7 is worse than 3.5?
I really like 3.5 and can be productive with it, but with Claude 3.7 it can't fix even simple things.
Last night I sat for 30 minutes just to try to get the new model to remove a instructions section from a Next.js page. It was an isolated component on the page named InstructionsComponent. Failed non-stop, didn't matter what I did, it could not do it. 3.5 did it first try, I even mistyped instructions and the model fixed the correct thing anyway.
I don't think it's worse but it's like losing a friend in a small way. It's not the same assistant you talked to previously
Is it actually good at solving complex code or is it just garbage and people are lying about it as usual?
In my experience EXTENSIVELY using claude 3.5 sonnet you basically have to do everything complex or you're just introducing massive amounts of slop code into your code base that while functional is nowhere near good. And for anything actually complex like requires a lot of context to make a decision and has to be useful to multiple different parts, it's just hopelessly bad.
I've played with it the whole day (so take it with a grain of salt). My gut feeling is that it can produce a bigger ... "thing". I am calling it a "thing", because it looks very much as what you want, but the bigger it is - the more the chances of it being subtly (or not) wrong.
I usually ask the models to extend a small parser/tree-walking interpreter with a compiler/VM.
Up until Claude 3.7 the models would propose something lazy and obviously incomplete. 3.7 generated something that looks almost right, mostly works, but is so overcomplicated and broken in such a way, that I rather delete it and write it from scratch. Trying to get the model to fix it resulted in running in circles, spitting out pieces of code that didn't fit the existing ones etc.
Not sure if I prefer the former or the latter tbh.
I am noticing a good dose of hallucination for 3.7 thinking in cursor.
3.7 seems more reliable.
Just like OpenAI or Grok, there is no transparency and no way for self-hosting purposes. Your input and confidential information can be collected for training purposes.
I just don't trust those companies when you use their servers. This is not a good approach to LLM democratization.
I wouldn’t assume there’s no way to self host — it just costs a lot more than open weights.
Anthropic claims they don’t train on their inputs. I haven’t seen any reason to disbelieve them.
But there is no way to know if their claims are true either. Your inputs are processed into their servers, then you get a response. Whatever happens in the middle, only Anthropic knows. We don't even know of governments are actually pushing AI companies to enforce censorship or spying people, like we seen recently at UK government getting into Apple E2E encryption.
This criticism is valid for the business who wants to use AI to improve coding, code analysis or code review, documentation, emails, etc, but also for that individual who don't want to rely on 3rd party companies for AI usage.
1 reply →
Claude 3.7 Sonnet seems to have a context window of 64.000 via the API:
I got a max of 8192 with Claude 3.5 sonnet.
Context window is how long your prompt can be. Output tokens is how long its response can be. What you sent says its response can be 64k tokens at maximum.
> strong improvements in coding and front-end web development
The best part
Watching Claude Code fumble around trying to edit text and double checking the hex output of a .cpp file and cd around a folder all while burning actual dollars and context is the opposite of endearing.
Any plans to make some HackerRank Astra bench?
Hello
Helo
Hey
Video killed the radio star
Anyone else noticed all the reasoning models kinda catch up on claude and claude itself turned to crap last week?
I have observed some unusual behavior.
I wonder if it simply due to reprioritization of resources.
Presumably, there is some parameter that determines how long a model is allowed to use resources for, which would get tapered in preparation for a demand surge of another model.
Well there goes my evening
Finally got access to the preview just now.
Let's fire it up.
"Type /init to set up your repository"
OK, /init <enter>
"OK, I created CLAUDE.md, session cost so far is $0.1764"
QUIT QUIT QUIT QUIT QUIT
Seventeen cents just to initialize yourself, Claude. No.
I feel like I touched a live wire.
It's about 2 orders of magnitude (100x) too expensive.
I feel like 3.7's personality is neutered, and frankly, the personality was the biggest selling point for me
OpenAI should be worried as they products are weak
Huh
I asked it for a self-portrait as a joke and the result is actually pretty impressive.
Prompt: "Draw a SVG self-portrait"
https://claude.site/artifacts/b10ef00f-87f6-4ce7-bc32-80b3ee...
For comparison, this is Sonnet 3.5's attempt: https://claude.site/artifacts/b3a93ba6-9e16-4293-8ad7-398a5e...
I kinda get how LLMs work with language, but it beyond blows me my mind trying to understand how an LLM can draw SVG. There are just so many dimensions to understanding how SVG converts to an image. Even as a human I don't think I could do anywhere close to that result in first attempt.
New mascot! Just make it the Anthropic orange
"Make me a website about books. Make it look like a designer and agency made it. Use Tailwind."
https://play.tailwindcss.com/tp54wfmIlN
Getting way better at UI.
That's not hideous. Derivative, but that's the nature of the beast.
As a designer and agency... this is extremely basic... but so was the prompt.
I cannot believe that others just casually dismiss this as 'basic', when just a few years ago this would have taken someone a full day of work.
I mean, it is basic. Templates have been around for decades and this looks like a template from 2007 that someone filled in with their own copy. That might take like an hour, maybe? And presumably the person who wants the page done will have to customize this text, too.
I feel like something isn't working... when i try to click anything it just reloads. i can't see the collections
> Third, in developing our reasoning models, we’ve optimized somewhat less for math and computer science competition problems, and instead shifted focus towards real-world tasks that better reflect how businesses actually use LLMs.
Company: we find that optimizing for LeetCode level programming is not a good use of resources, and we should be training AI less on competition problems.
Also Company: we hire SWEs based on how much time they trained themselves on LeetCode
/joke of course
My manager explained to me that LeetCode is proving that you are willing to dance the dance. Same as PhD requirements etc - you probably won't be doing anything related and definitely nothing related to LeetCode, but you display dedication and ability.
I kinda agree that this is probably reason why companies are doing it. I don't like it, but this is besides the matter.
Using Claude other models in interviews probably won't be allowed any time soon, but I do use it the work. So it does make sense.
And it's also the reality of hiring practices for most VC-backed and public companies
Some try to do something more like "real-world" tasks, but those end up either being either just toy problems, or long take homes
Personally, I feel the most important things to prioritize when hiring are: is the candidate going to get along with their teammates (colleagues, boss, etc), and do they have the basic skills to relatively quickly learn their jobs once they start?
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[flagged]
> Why would my phone number be any of their business?
Preventing abuse? It's much harder to create a throwaway phone number than a throwaway email address.
> OpenAI does the logical thing. Let's me enter my credit card and I'm good to go. I will stay with them.
You'd rather hand over your credit card than your phone number? I think most people would see it the other way around.
Credit card is easily changed. Phone number is much more difficult.
4 replies →
> You'd rather hand over your credit card than your phone number?
You know, that was my first reaction, too. But really, my phone number is much more integral to my identity. I can cancel or change a credit card number pretty trivially and then it's useless to you.
Many credit card companies make it easy to generate one-off card numbers/“virtual cards” you can use to subscribe to services that are hard to cancel or otherwise questionable (so you can cancel just the card you used for that company).
> OpenAI does the logical thing. Let's me enter my credit card and I'm good to go. I will stay with them.
When did you make your account? I could have sworn I had to verify with my phone number before payment.
That's the reason I've not yet signed up. After trying to use some anonymous sms service (which failed), I'm still not having an account:).
They block numbers from any provider considered to be voip.
Honestly I don't think Anthropic cares about you moving over to them - it's pretty evident that they already have more demand than they can handle.
I've always had better experience with Claude in day-to-day coding and text writing, and looking at public forums that largely seems to be the case.
Not for the api.
https://hushed.com/ or specifically this offer https://promo.hushed.com/lp/lifetime-3 which will get you a lifetime number for $25
Who do I have to kill to get Claude Code access?
$ npm install -g @anthropic-ai/claude-code
$ claude
I wish Amodei didn't write that essay where he begged for export controls on China like that disabled corgi from a meme. I won't use anything Anthropic out of principle now. Compete fairly or die.
Tried claude code, and have an empty unresponsive terminal.
Looks cool in the demo though, but not sure this is going to perform better than Cursor, and shipping this as an interactive CLI instead of an extension is... a choice
I think it's a smart starting point as it's compatible with all IDEs. Iterate and learn and then later wrap the functionality up into IDE plugins.