Comment by ivanovm
2 days ago
I find the very popular response of "you're just not using it right" to be big copout for LLMs, especially at the scale we see today. It's hard to think of any other major tech product where it's acceptable to shift so much blame on the user. Typically if a user doesn't find value in the product, we agree that the product is poorly designed/implemented, not that the user is bad. But AI seems somehow exempt from this sentiment
> It's hard to think of any other major tech product where it's acceptable to shift so much blame on the user.
It's completely normal in development. How many years of programming experience you need for almost any language? How many days/weeks you need to use debuggers effectively? How long from the first contact with version control until you get git?
I think it's the opposite actually - it's common that new classes of tools in tech need experience to use well. Much less if you're moving to something different within the same class.
> LLMs, especially at the scale we see today
The OP qualifies how the marketing cycle for this product is beyond extreme, and its own category.
Normal people are being told to worry about AI ending the world, or all jobs disappearing.
Simply saying “the problem is the user”, without acknowledging the degree of hype, and expectation setting, the is irresponsible.
AI marketing isn't extreme - not on the LLM vendor side, at least; the hype is generated downstream of it, for various reasons. And it's not the marketing that's saying "you're using it wrong" - it's other users. So, unless you believe everyone reporting good experience with LLMs is a paid shill, there might actually be some merit to it.
9 replies →
It is completely typical, but at the same time abnormal to have tools with such poor usability.
A good debugger is very easy to use. I remember the Visual Studio debugger or the C++ debugger on Windows were a piece of cake 20 years ago, while gdb is still painful today. Java and .NET had excellent integrated debuggers while golang had a crap debugging story for so long that I don’t even use a debugger with it. In fact I almost never use debuggers any more.
Version control - same story. CVS for all its problems I had learned to use almost immediately and it had a GUI that was straightforward. git I still have to look up commands for in some cases. Literally all the good git UIs cost a non-trivial amount of money.
Programming languages are notoriously full of unnecessary complexity. Personal pet peeve: Rust lifetime management. If this is what it takes, just use GC (and I am - golang).
> git I still have to look up commands for in some cases
I believe that this is okay. One does not need to know the details about every specific git command in order to be able to use it efficiently most of the time.
It is the same with a programming language. Most people are unfamiliar with every peculiarity of every standard library function that the language offers. And that is okay. It does not prevent them from using language efficiently most of the time.
Also in other aspects of life, it is unnecessary to know everything by memory. For example, one does not need to know how to e.g. replace a blade on a lawn mower. But that is okay. It does not prevent them from using it efficiently most of the time.
The point is that if something is done less often, it is unnecessary to remember the specifics of it. It is fine to look it up when needed.
> It is completely typical, but at the same time abnormal to have tools with such poor usability.
The main difference I see is that LLMs are flaky, getting better over time, but still more so than traditional tooling like debuggers.
> Programming languages are notoriously full of unnecessary complexity. Personal pet peeve: Rust lifetime management. If this is what it takes, just use GC (and I am - golang).
Lifetime management is an inherently hard problem, especially if you need to be able to reason about it at compile time. I think there are some arguments to be made about tooling or syntax making reasoning about lifetimes easier, but not trivial. And in certain contexts (e.g., microcontrollers) garbage collectors are out of the question.
Nitpick: magit for emacs is good enough for everyone whom I’ve seen talk about it describe as “the best git correct” and it is completely free.
Linus did not show up in front of congress talking about how dangerously powerful unregulated version control was to the entirety of human civilization a year before he debuted Git and charged thousands a year to use it.
This seems like a non sequitur. What does this have to do with this thread?
2 replies →
Ok. You seem to be taking about a completely different issue of regulation.
Hmmm, I don't see it? Are debuggers hard to use? Sometimes. But the debugger is allowing you to do something you couldn't actually do before. i.e. set breakpoints, and step through your code. So, while tricky to use, you are still in a better position than not having it. Just because you can get better at using something doesn't automatically mean that using it as a beginner makes you worse off.
Same can be said for version control and programming.
i guarantee you there were millions of people that needed to be forced to use excel because they thought they could do the calculations faster by hand.
we retroactively assume that everyone just obviously adopts new technology, yet im sure there were tons and tons of people that retired rather than learning how computers worked when the PC revolution was happening.
> How many days/weeks you need to use debuggers effectively
I understand your point, but would counter with: gdb isn't marketed as a cuddly tool that can let anyone do anything.
>It's hard to think of any other major tech product where it's acceptable to shift so much blame on the user.
Is that perhaps because of the nature of the category of 'tech peoduct'. In other domains, this certainly isn't the case. Especially if the goal is to get the best result instead of the optimum output/effort balance.
Musical instruments are a clear case where the best results are down to the user. Most crafts are similar. There is the proverb "A bad craftsman blames his tools" that highlights that there are entire fields where the skill of the user is considered to be the most important thing.
When a product is aimed at as many people as the marketers can find, that focus on individual ability is lost and the product targets the lowest common denominator.
They are easier to use, but less capable at their peak. I think of the state of LLMs analogous to home computing at a stage of development somewhere around Altair to TRS-80 level. These are the first ones on the scene, people are exploring what they are good for, how they work, and sometimes putting them to effective use in new and interesting ways. It's not unreasonable to expect a degree of expertise at this stage.
The LLM equivalent of a Mac will come, plenty of people will attempt to make one before it's ready. There will be a few Apple Newtons along the way that will lead people to say the entire notion was foolhardy. Then someone will make it work. That's when you can expect to use something without expertise. We're not there yet.
> It's hard to think of any other major tech product where it's acceptable to shift so much blame on the user.
Maybe, but it isn't hard to think of developer tools where this is the case. This is the entire history of editor and IDE wars.
Imagine running this same study design with vim. How well would you expect the not-previously-experienced developers to perform in such a study?
No one is claiming 10x perf gains in vim.
It’s just a fun geeky thing to use with a lot of zany customizations. And after two hellish years of memory muscling enough keyboard bindings to finally be productive, you earned it! It’s a badge of pride!
But we all know you’re still fat fingering ggdG on occasion and silently cursing to yourself.
> No one is claiming 10x perf gains in vim.
Sure they are - or at least were, unitl the last couple years. Same thing with Emacs.
It's hard to claim this now, because the entire industry shifted towards webshit and cloud-based practices across the board, and the classical editors just can't keep up with VS Code. Despite the latter introducing LSP, which leveled the playing field wrt. code intelligence itself, the surrounding development process and the ecosystem increasingly demands you use web-based or web-derived tools and practices, which all see a browser engine as a basic building block. Classical editors can't match the UX/DX on that, plus the whole thing breaks basic assumptions about UI that were the source of the "10x perf gains" in vim and Emacs.
Ironically, a lot of the perf gains from AI come from letting you avoid dealing with the brokenness of the current tools and processes, that vim and Emacs are not equipped to handle.
7 replies →
Huh? Most people use tools like vim for productivity...
I agree with you that AI dev tools are overhyped at the moment. But IDEs were, in fact, overhyped (to a lesser degree) in the past.
What I like about IDE wars is that it remained a dispute between engineers. Some engineers like fancy pants IDEs and use them, some are good with vim and stick with that. No one ever assumed that Jetbrains autocomplete is going to replace me or that I am outdated for not using it - even if there might be a productivity cost associated with that choice.
Excellent point. But I do think that forcing people to use IDEs for productivity was a thing for awhile. But still agree that the current moment is a difference in kind not just in scale.
New technologies that require new ways of thinking are always this way. "Google-fu" was literally a hirable career skill in 2004 because nobody knew how to search to get optimal outcomes. They've done alright improving things since then - let's see how good Cursor is in 10 years.
>It's hard to think of any other major tech product where it's acceptable to shift so much blame on the user.
Apple's Response to iPhone 4 Antenna Problem: You're Holding It Wrong https://www.wired.com/2010/06/iphone-4-holding-it-wrong/
I don't see how the Antennagate can be qualified as "acceptable" since it caused a big public uproar and Apple had to settle a class action lawsuit.
https://www.businessinsider.com/apple-antennagate-scandal-ti...
it didnt end the iphone as a brand, or end smart phones altogether though.
how much did that uproar and settlement matter?
Mobile phone manufacturers were telling users this long before the iPhone was ever invented.
e.g., Nokia 1600 user guide from 2005 (page 16) [0]
[0] https://www.instructionsmanuals.com/sites/default/files/2019...
The important difference is that in your example, it was the manufacturer telling customers they're holding it wrong. With LLMs, the vendors say no such things - it's the actual users that are saying this to their peers.
I think the reason for that is maybe you’re comparing to traditional products that are deterministic or have specific features that add value?
If my phone keeps crashing or if the browser is slow or clunky then yes, it’s not on me, it’s the phone, but an LLM is a lot more open ended in what it can do. Unlike the phone example above where I expect it to work from a simple input (turning it on) or action (open browser, punch in a url), what an LLM does is more complex and nuanced.
Even the same prompt from different users might result in different output - so there is more onus on the user to craft the right input.
Perhaps that’s why AI is exempt for now.
Stay tuned, a new study is coming with another revelation: you aren't getting faster by using Vim when you are learning it.
My previous employer didn't even allow me to use Vim until I learned it properly so it wouldn't affect my productivity. Why would using a cursor automatically make you better at something if it's just new to you and you are already an elite programmer according to this study?
How did you measure this? Was the conslusion of your studies that typing/editing speed was the real bottlekneck for a SWE becoming 10x?
It's a specialist tool. You wouldn't be surprised that it took awhile for someone to take a big to get at typed programming, parallel programming, docker, IaaC, etc. either.
We have 2 sibling teams, one the genAI devs and the other the regular GPU product devs. It is entirely unsurprising to me that the genAI developers are successfully using coding agents with long-running plans, while the GPU developers are still more at the level of chat-style back-and-forth.
At the same time, everyone sees the potential, and just like other automation movements, are investing in themselves and the code base.
On the other hand if you don't use vim, emacs, and other spawns from hell, you get labeled a noob and nothing can ever be said about their terrible UX.
I think we can be more open minded that an absolutely brand new technology (literally did not exist 3y ago) might require some amount of learning and adjusting, even for people who see themselves as an Einstein if only they wished to apply themselves.
> you get labeled a noob
No one would call one a noob for not using Vim or Emacs. But they might for a different reason.
If someone blindly rejects even the notion of these tools without attempting to understand the underlying ideas behind them, that certainly suggests the dilettante nature of the person making the argument.
The idea of vim-motions is a beautiful, elegant, pragmatic model. Thinking that it is somehow outdated is a misapprehension. It is timeless just like musical notation - similarly it provides compositional grammar and universal language, and leads to developing muscle memory; and just like it, it can be intimidating but rewarding.
Emacs is grounded on another amazing idea - one of the greatest ideas in computer science, the idea of Lisp. And Lisp is just as everlasting, like math notation or molecular formulas — it has rigid structural rules and uniform syntax, there's compositional clarity, meta-reasoning and universal readability.
These tools remain in use today despite the abundance of "brand new technology" because time and again these concepts have proven to be highly practical. Nothing prevents vim from being integrated into new tools, and the flexibility of Lisp allows for seamless integration of new tools within the old-school engine.
One could try to be poetic with LLMs in order to make their point stronger and still convince absolutely no one who wasn't already convinced.
I'm sure nobody really reject the notion of LLMs but sure as hell do like to moan if the new technology doesn't absolutely perfect fit their own way of working. Does that make them any different than people wanting an editor which is intuitive to use? Nobody will ever know.
1 reply →
Not every tool can be figured out in a day (or a week or more). That doesn't mean that the tool is useless, or that the user is incapable.
I've spent the last 2 months trying to figure out how to utilize AI properly, and only in the last week do I feel that I've hit upon a workflow that's actually a force multiplier (vs divisor).
Cool! Contragtulations to the anecdotal feelings of productivity. Superimportant input to the discussion. Now I at least can confidently say that investors will definitely get the hundreds of billions, trillions spent back with a fat ROI and profits on top!
Thanks again!
> It's hard to think of any other major tech product where it's acceptable to shift so much blame on the user.
Sorry to be pedantic but this is really common in tech products: vim, emacs, any second-brain app, effectiveness of IDEs depending on learning its features, git, and more.
Well, surely vim is easy to use - I started it and and haven't stopped using it yet (one day I'll learn how to exit)
Just a few examples: Bicycle. Car(driving). Airplane(piloting). Welder. CNC machine. CAD.
All take quite an effort to master, until then they might slow one down or outright kill.