LLMs make significant mistakes frequently and smart people have no way of judging those mistakes outside their domain expertise. They are also sycophantic and great at being an echo chamber which makes people feel smart even if they are not.
So I think the burden of proof is on you to prove that they somehow amplify intelligence, it seems highly unlikely.
Most domains have some kind of internal consistency/theory building you can do. A smart person can certainly notice inconsistencies when trying to learn something. In fact they're likely to be points of confusion that the smart person will dive into just to try to make sense of things, even if they don't suspect the LLM is at fault.
Smart people know LLMs confabulate and tell them they’re Absolutely Right! Smart people don’t want to be embarrassed by trusting the hallucination machine and revealing their gullibility to others.
Why would a smart person go to an LLM for an answer they cannot judge or test, be succeptible to flattery and sycophancy rather than picking up on the emotional manipulation and being suspicious/sceptical of the interaction, or looking for support from an echo chamber rather than a Socratic opponent?
All of those sound like flaws and defects of dumb people?
> Why would a smart person go to an LLM for an answer they cannot judge or test, be succeptible to flattery and sycophancy rather than picking up on the emotional manipulation and being suspicious/sceptical of the interaction, or looking for support from an echo chamber rather than a Socratic opponent?
Great summary of the flaws with LLM "research"/"reasoning". It's always trying to con you, and I question the literacy and intelligence of the people who can't see this.
It helps smart people be barely more effective and helps dumb (more importantly, people who do not value effort, people who are lazy, people who are self absorbed) people shit out endless streams of worthless tokens that can swamp out everything.
Raising the noise floor like this only makes it that much harder to find "Smart" people, which we were already doing terrible at.
I use Claude every single day, but this is such a bad tradeoff. Maybe it will help me standup a quick fix when that is needed. Maybe it can help me dig through documentation to find relevant bits and figure out the unstated assumptions underlying it. Maybe it helps me generate test cases.
Meanwhile, my day to day life is now noise. All social media is noise. All content is noise. Slop pours onto me from all directions. Writing more test cases isn't helping me.
Am I smart? Am I dumb? I don't care, right now I'm deafened
All I've noticed is AI creating a perverse incentive to make everything as complicated and bureaucratic as possible, so only the people that know how to leverage AI to cut through it ever succeed.
I'm using Claude at work myself and am impressed with the product, but notice that this is the only reason I need to use it at all. Our product pages were shit to begin with, now they're AI-generated and somehow even worse. Our procedures are incomprehensible spaghetti with enough arbitrary context switching to give a sadistic Soviet municipal administrator an erection at the thought of watching anyone try to actually follow them.
Use AI to create inefficiencies, then use AI to bypass them. Those who can't do the latter will struggle to survive.
> Raising the noise floor like this only makes it that much harder to find "Smart" people,
It does give us a new heuristic, though: people who are willing to completely cut generative AI out of their lives (cold-turkey, if you ever started using it) are a much smaller group of, predominantly thoughtful, people. You do have to give up Claude to be part of this group, but from what you say, that's no great loss, and no longer being deafened is worth it.
This has considerable advantages over conventional elitism, because the barrier-to-entry is negative in almost all cases.
The one exception I've found is assistive tech, where the state-of-the-art is so poor that vibecoded slop is genuinely an improvement over the state-of-the-art, and in many cases the tooling simply isn't available to make your own assistive tech (unless you want to bootstrap an entire networked computing environment, which isn't very helpful when you want to do your online banking and do not, in fact, work at your bank).
But there are not many principled exceptions where you could seriously argue that the trade-off is worth it. Take mathematics, for example, which we often see touted as a "good use-case" of generative AI. The primary advantage of generative AI in mathematics is being able to search though a vast corpus of ivory towers and inconsistent terminology (without proper attribution) to locate and connect ideas that can help solve problems. The deficiency this is addressing is elitism, inadequate communication, and inadequate indexing within academic mathematics. This problem is entirely created by the academic mathematicians, and has been known for nearly a century (per https://news.ycombinator.com/item?id=49392265 points out one such.)
Vibe-coding assistive tech which doesn't yet exist, as a temporary scaffold to improve the quality-of-life of yourself and others in a social world dominated by non-essential access barriers is, to my knowledge, the only exception to this principle that can be justified. If you treat people who make other excuses, or who don't even bother with excuses, as not worth listening to, you lose little – and doubly-so, if you make your stance clear, so that others know the "cost" of gaining your attention.
There is zero evidence for this.
LLMs make significant mistakes frequently and smart people have no way of judging those mistakes outside their domain expertise. They are also sycophantic and great at being an echo chamber which makes people feel smart even if they are not.
So I think the burden of proof is on you to prove that they somehow amplify intelligence, it seems highly unlikely.
Most domains have some kind of internal consistency/theory building you can do. A smart person can certainly notice inconsistencies when trying to learn something. In fact they're likely to be points of confusion that the smart person will dive into just to try to make sense of things, even if they don't suspect the LLM is at fault.
No evidence but somewhat of a counterpoints:
Smart people know LLMs confabulate and tell them they’re Absolutely Right! Smart people don’t want to be embarrassed by trusting the hallucination machine and revealing their gullibility to others.
Why would a smart person go to an LLM for an answer they cannot judge or test, be succeptible to flattery and sycophancy rather than picking up on the emotional manipulation and being suspicious/sceptical of the interaction, or looking for support from an echo chamber rather than a Socratic opponent?
All of those sound like flaws and defects of dumb people?
> Why would a smart person go to an LLM for an answer they cannot judge or test, be succeptible to flattery and sycophancy rather than picking up on the emotional manipulation and being suspicious/sceptical of the interaction, or looking for support from an echo chamber rather than a Socratic opponent?
Great summary of the flaws with LLM "research"/"reasoning". It's always trying to con you, and I question the literacy and intelligence of the people who can't see this.
All at the cost of ... {List of negatives regarding the construction and powering of AI data centers }
The social costs are worse.
Most importantly, it makes investors think dumb people are smart.
It helps smart people be barely more effective and helps dumb (more importantly, people who do not value effort, people who are lazy, people who are self absorbed) people shit out endless streams of worthless tokens that can swamp out everything.
Raising the noise floor like this only makes it that much harder to find "Smart" people, which we were already doing terrible at.
I use Claude every single day, but this is such a bad tradeoff. Maybe it will help me standup a quick fix when that is needed. Maybe it can help me dig through documentation to find relevant bits and figure out the unstated assumptions underlying it. Maybe it helps me generate test cases.
Meanwhile, my day to day life is now noise. All social media is noise. All content is noise. Slop pours onto me from all directions. Writing more test cases isn't helping me.
Am I smart? Am I dumb? I don't care, right now I'm deafened
All I've noticed is AI creating a perverse incentive to make everything as complicated and bureaucratic as possible, so only the people that know how to leverage AI to cut through it ever succeed.
I'm using Claude at work myself and am impressed with the product, but notice that this is the only reason I need to use it at all. Our product pages were shit to begin with, now they're AI-generated and somehow even worse. Our procedures are incomprehensible spaghetti with enough arbitrary context switching to give a sadistic Soviet municipal administrator an erection at the thought of watching anyone try to actually follow them.
Use AI to create inefficiencies, then use AI to bypass them. Those who can't do the latter will struggle to survive.
> Raising the noise floor like this only makes it that much harder to find "Smart" people, which we were already doing terrible at.
Clarification: to value “smart” people, which we were already doing terrible at.
> Raising the noise floor like this only makes it that much harder to find "Smart" people,
It does give us a new heuristic, though: people who are willing to completely cut generative AI out of their lives (cold-turkey, if you ever started using it) are a much smaller group of, predominantly thoughtful, people. You do have to give up Claude to be part of this group, but from what you say, that's no great loss, and no longer being deafened is worth it.
This has considerable advantages over conventional elitism, because the barrier-to-entry is negative in almost all cases.
The one exception I've found is assistive tech, where the state-of-the-art is so poor that vibecoded slop is genuinely an improvement over the state-of-the-art, and in many cases the tooling simply isn't available to make your own assistive tech (unless you want to bootstrap an entire networked computing environment, which isn't very helpful when you want to do your online banking and do not, in fact, work at your bank).
But there are not many principled exceptions where you could seriously argue that the trade-off is worth it. Take mathematics, for example, which we often see touted as a "good use-case" of generative AI. The primary advantage of generative AI in mathematics is being able to search though a vast corpus of ivory towers and inconsistent terminology (without proper attribution) to locate and connect ideas that can help solve problems. The deficiency this is addressing is elitism, inadequate communication, and inadequate indexing within academic mathematics. This problem is entirely created by the academic mathematicians, and has been known for nearly a century (per https://news.ycombinator.com/item?id=49392265 points out one such.)
Vibe-coding assistive tech which doesn't yet exist, as a temporary scaffold to improve the quality-of-life of yourself and others in a social world dominated by non-essential access barriers is, to my knowledge, the only exception to this principle that can be justified. If you treat people who make other excuses, or who don't even bother with excuses, as not worth listening to, you lose little – and doubly-so, if you make your stance clear, so that others know the "cost" of gaining your attention.