Comment by diggan
5 days ago
> AI discourse would be more effective if we could all see the actual work one another is doing with it
Yes, this is a frequent problem both here and everywhere else. The discussions need to include things like exact model version, inference parameters, what system prompt you used, what user prompt, what code you gave it, what exactly it replied and so much more details, as currently almost every comment is "Well, I used Sonnet last week and it worked great" without any details. Not to mention discussions around local models missing basic stuff like what quantization (if any) and what hardware you're running it on. People just write out "Wow fast model" or stuff like that, and call it a day.
Although I understand why, every comment be huge if everyone always add sufficient context. I don't know the solution to this, but it does frustrate me.
There's many examples of exactly what you're asking for, such as Kenton Varda's Cloudlfare oauth provider [1] and Simon Willison's tools [2]. I see a new blog post like this with detailed explanations of what they did pretty frequently, like Steve Klabnik's recent post [3], which while it isn't as detailed has a lot of very concrete facts. There's even more blog posts from prominent devs like antirez who talk about other things they're doing with AI like rubber ducking [4], if you're curious about how some people who say "I used Sonnet last week and it was great" are working, because not everyone uses it to write code - I personally don't because I care a lot about code style.
[1]: https://github.com/cloudflare/workers-oauth-provider/
[2]: https://tools.simonwillison.net/
[3]: https://steveklabnik.com/writing/a-tale-of-two-claudes/
[4]: https://antirez.com/news/153
Maybe I should have been more specific, I was talking specifically about discussions and comments on forums like HN and r/localllama, not that people who are writing blogposts aren't specific enough in their blogposts.
> The discussions need to include things like exact model version, inference parameters, what system prompt you used, what user prompt, what code you gave it, what exactly it replied and so much more details, as currently almost every comment is "Well, I used Sonnet last week and it worked great" without any details...Not to mention discussions around local models missing basic stuff like what quantization (if any) and what hardware you're running it on.
While I agree with "more details", the amount of details you're asking for is ... ridiculous. This is a HN comment, not a detailed study.
> the amount of details you're asking for is
I'm not asking for anything, nor providing anything as "a solution", just stating a problem. The second paragraph in my comment is quite literally about that.
I feel like that would get tiresome to write, read, and sort through. I don't like everyone's workflow, but if I notice someone making a claim that indicates they might be doing something better than me, then I'm interested.
Maybe keeping your HN profile/gist/repo/webpage up to date would be better.