Comment by Frost1x
1 month ago
I work at an intersection of tech, applied research, and science.
Something I’ve noticed in collaboration that does occur is an increased confidence in people outside their domains to say things with conviction. I have people who have limited experience with software pushing out layers and layers of abstracted code that’s fairly sophisticated but often misguided in intent who will say what they’re doing is correct, with conviction.
I also hear a lot more questioning people in their domains and challenging opinions, then hearing what I can only imagine are fragmented pieces of conversations they had with an LLM thinking through some argument. Then there’s silence when you discuss shortcomings, then they come back later with their memorized fragments of what you said, combined with memorized fragments of the LLM response to the argument.
It’s occurring, a lot more. People are treating their LLMs in collaboration as a source of truth and using then to focus on their specific path or goals they think or have bias towards going down, vs just opening discussing things, considering tradeoffs from experts multiple disciplines weigh in on and then taking an approach that everyone finds most agreeable.
It’s making me want to be a lot less collaborative with such individuals. I don’t want to sit around and refute Claude text outputs all day.
Wow, oddly similar thing happened to me. Anecdotally, I was working with a non-developer. She was building a reporting dashboard with Claude. I pointed out that the system she built was using the filesystem to store one line notes about the report, and I said "I would probably tell the agent to use a database for that, but it's not the end of the world if it works". And she replied "Claude said the developer's right, a database would be a better fit..."
So, a non-developer, who has no understanding of the underlying systems they are working with, fact-checked a developer with years of experience.
I'm not against fact checking, but it absolutely felt like a punch in the gut coming from a non-dev. There was no "trust" there. She couldn't rely on my expertise, she had to go back to her "source of truth" (Claude) and asked for a second opinion on something that she wouldn't be able to verify.
This is why collaboration exists. I trust my colleagues to give me insight+advice on things that I do not know. I don't ask AI to vet their decisions, because at some level there are judgement calls, and I WANT to trust them because it makes my life easier.
Treating AI as if the output is factual is just a misuse of the tools, I think. It's significantly more effective when given expert direction and the output can be verified by... an expert.
>>I'm not against fact checking, but it absolutely felt like a punch in the gut coming from a non-dev. There was no "trust" there. She couldn't rely on my expertise, she had to go back to her "source of truth" (Claude) and asked for a second opinion on something that she wouldn't be able to verify.
This is probably how physicians feel when informed patients (whether using LLMs or not) ask questions (good or bad) during their patient encounters.
I would love to be able to trust that a doctor knows more about my problem than I do, but every time I see one they clearly haven't even read the information I just told the nurse 15 minutes ago.
Doctor:
I don’t mind the questions. I mind the assertions of fact.
This is a different situation. A doctor is a domain expert, but you are in possession of vastly more empirical data about your specific body. Getting to a good result is going to be easier with collaboration, rather than one of you confidently telling the other what the problem/solution is.
Yeah I've felt that gut punch all the time. Seems to happen with every technology. In the early days of GPS, people would ask for directions to the gas station then pull it up on their phone map and go "yeah, you're right!" So thanks for wasting my time, insulting my integrity, and killing my faith in humanity all in 3 words. And if you really need two opinions, it's so easy to just reverse the situation, "my GPS says there's an Exxon in two miles if I turn left here and then right, is that correct?"
I remember fifteen years ago I would tell people leaving my house "oh yeah, ignore your GPS that tells you to go that way, just down the road here and left turn; that'll get you on the highway much faster."
Nowadays obviously Google Maps gets it right, but it was interesting that period where you had to actively preempt everyone's instinct to just rely on the little table hanging off their dashboard.
2 replies →
wow that wouldnt feel like a gut punch at all to me, it's normal for inexperienced people to not really understand what kinds of knowledge experts have or don't. I certainly come back from doctors appointments and google/gemini chat for whatever came up to get more detail. this person going to claude and getting immediate confirmation that you were exactly correct should have felt great, she would be like "wow, this guy's good, huh."
I think I could qualify as an expert on various topics relative to many average people. But it always bothered me when experts wanted to "just be believed". In your example, it doesn't long to explain why a database is better than the file system in this instance and a moderately intelligent should be able to understand and not need you or Claude as the source of truth.
So I think it can be good people are able to "AI" to question expert opinions - though naturally, as you say, it's bad if people take the AI as a single authority akin to an expert.
So, I did let her know that a storage volume was going to be more difficult to manage, could result in lost data, and would make the data harder to work with - I mentioned it's not the end of the world if it works, it was just a professional suggestion.
To me, there are different levels to "just being believed". If a mechanic tells you your car isn't working because you don't have an alternator, does that really need scrutiny? If you don't understand the subject matter, at some point you have to "just believe" someone else.
Especially for a non-technical user, trusting the software developers at your company on technical advice should be a given.
I'd bet it depends on what they write to the LLM, since sycophancy is so strong in these things. If they write in a fairly agnostic way, it might default to sycophancy to you the dev, while if they write it in an opinionated way it might default to sycophancy to them. Regardless of who is right.
> Treating AI as if the output is factual is just a misuse of the tools, I think.
Especially because it's very often wrong. AI tools lack the necessary context of the specific situation and the surrounding technology environment to make appropriate suggestions for anything more complex than clean sheet designs and toy prototypes. There is too much complexity that is hidden in domain knowledge and not even exposed fully in documentation for any AI tool currently to ingest it all into its own context to give you an accurate answer, even setting aside hallucinations and that these tools are incapable of reasoning.
I have this same gut punch constantly. AI has destroyed functional outcomes in corporations, because managers are all deep in AI psychosis deciding that whatever the tool says is perfectly correct at all times. We've replaced analysis paralysis with a complete absence of thought from any human in high-level impactful strategy discussions, everything is entirely being abdicated to AI tools in so many companies. It's ridiculous.
AI will radically change the economy, but not because its replacing jobs, but because its eroding trust, domain knowledge and expertise, and thinking/creativity while being a massive waste of money and total boondoggle. This bubble can't burst soon enough so we can get on with doing shit that actually matters for people that actually helps them.
To be generous, it seems like the most natural way to phrase what happened. The non-developer punched in "should this be a database instead?" and it will say "You are right to question it! I was sloppy when .. blah blah blah blah blah"
I doubt the human went into a whole diatribe with the CLI about how the developer on the project with this many years was saying this or that but she wanted Claude to second guess them.
It's just saying "ah yeah, that went through without issue."
I can see how the situation is annoying there; I've had a similar frustration when doing manual code review for someone's obviously AI-slop PR only to have their bot auto-respond to my comments. There's an asymmetry of effort that feels unfair. Like, don't ask for manual review of something that contains lots of things that your own manual review should have caught first, and if I provide human feedback, I expect human responses.
That said, I wouldn't necessarily take it at face value that the person you were working with trusted the bot more than you— at the end of the day, the bot was doing all the implementation anyway, so all the interactions between her and the bot were probably framed in language like "what if we" or "I think it would be better if", so her saying "my programmer colleague thinks X" would naturally generate that "yeah your colleague is right, let's go that direction, makes sense" that she passed back to you, probably thinking it was a complement.
Maybe the real frustration is her not understanding the value-add that still exists there when someone with real expertise and more context can step in and provide that nudge in the right direction.
Sure - I don't hold it against them. It was just surprising to read a "Claude said you're right" message. (Yes, I know that I'm right, or I wouldn't have said it)
Didn't ruin my week or anything, but that was the writing on the wall for me.
Technical expertise is no longer viewed in the same light when software development has no barrier to entry.
1 reply →
>I'm not against fact checking, but it absolutely felt like a punch in the gut coming from a non-dev. There was no "trust" there.
Maybe I'm just cynical but I don't think I ever see that much trust for anyone's opinion or expertise; devs think managers don't know what they're doing, managers think devs don't, people on HN think lawyers are wrong when they state their legal opinion, devs think other devs are wrong about their areas of expertise even though they have no reason to think that, hell I've worked with people who basically lie for some reason about technical issues that they have no expertise in, in order to get some benefit that is unclear to me. I've also seen people of great technical expertise gaslight industries in order to derive very clear benefits for their employers, so I'm not sure why there should be trust.
I'm sure that various AI agents will maybe add another interesting wrinkle to this, but I sort of feel like you must have had a nice life up until AI if that was feeling like a gut punch.
non-dev AI psychosis doesn’t bother me
I’ve seen how the market treats vibe coders that werent previoisly engineers. Coinbase’s CEO bragged about PM’s deploying code and everyone now thinks they are totally incompetent anytime a small UX problem occurs.
Engineers with prior experience now check a box, even if they are doing the same thing. Like a Compliance Officer or Cybersecurity professional checks a box
Annoyingly pedantically speaking, a filesystem is a type of database.
God do I wish that was her response to my suggestion.
"Fair enough - go nuts. Let me know when Claude tells you to transition to Postgres because I'll need to get you access credentials"
Yup, the arrogance when barging into unfamiliar domain that was previously reserved mainly to physics grads seems to have spread everywhere. I also had people opining on my expert area via their LLMs and the problem is they can't even ask the model the right question, let alone evaluate the nuance of their answer.
As an unfamiliar, I find it interesting how you specifically called out physics grads in the pre-LLM era. Is that just snark or is there some widespread truth / stereotype of that behavior?
Physics, and to a large degree pure mathematics, has attracted a critical mass of the personality type that needs to feel like the smartest person in the room in any given situation. It has given them a slightly notorious reputation in academia.
29 replies →
If you are very smart, and/or have an incredibly high tolerance for pain/repetition, you can solve amazingly complex issues, with awful code.
Both of these statements are true for physicists
Not the parent, but it's half joke half truth. Anyone who's done a lot of "first principles" work falls into this trap a bit, it's an issue across stem fields. Most grow out of it, or at least learn to qualify their statements for the audience.
I suspect the person you replied to only happened to see that from physics grads. I have seen it in all sorts of people and I sometimes did it myself when I was young (I am not a physics grad).
Its snark/stereotype, but there is some truth. But my take, is physics sort of took over a lot of chemistry, biology.. and to them its the answer to all questions, because in the end it just molecules. There is some truth to that but..
1 reply →
It’s a very common experience that people in the physics-adjacent sciences have.
It’s so common that there’s an xkcd about it:
https://xkcd.com/793/
It's a widespread stereotype at least: https://xkcd.com/793/
It might be a reference to this: https://xkcd.com/793/
1 reply →
It's common enough to have an xkcd for it at least (then again, what doesn't?) https://xkcd.com/793/
>arrogance when barging into unfamiliar domain
describes nearly every tech attempt at "disruption"
I hypothesize that with more people on a daily basis encountering information and claims that require those skills, there will be a time in the future when those skills are much more enhanced.
There is a reason this exists: https://xkcd.com/793/
I sadly share this as a physicist. To be fair, sometimes this arrogance can be useful. I worked at the boundary between domains my entire career where there aren't "experts" per se. But yeah, barging into an established domain without some humility generally leads to embarrassment - especially Nobel winners (e.g. https://www.newsweek.com/nobel-prize-winner-who-doesnt-belie...)
Hilarious. Do they preface the conversation with "Well, Claude said..."?
With my younger coworkers it's their defence when I push back on them.
Currently, the leader of USA national AI policy (Trump admin and OpenAI) is a history Bachelors with no technical training or advanced training of any kind, Dean Ball.
like a prosthetic dunning-kruger....
I also work at an intersection of tech, applied research and science. My experience has been different. It's been surprising how controlled my collaborators/coworkers have been with their reliance on AI.
Sure, they use it a lot, but despite the stereotypes against physicists in some of the replies to you, they have shown a good ability to catch themselves before leaning too hard on what an LLM tells them about things they are not experts in, and I haven't had any interactions where I felt like I was just arguing with a meat proxy.
Maybe it's because in the environment I'm in; it's relatively easy to just ask someone who is an expert in the topic for their advice.
My feeling is that this kind of overconfidence outside of one's domain is largely a thing for people with little "physical reality" experience. When all you deal with is the very flexible digital world, it becomes very easy to ignore how deep the knowledge and intuition goes in things that are directly constrained by reality. I consider myself to have been in this category too (CE background), though I have been making efforts to improve.
> I also work at an intersection of tech, applied research and science. My experience has been different. It's been surprising how controlled my collaborators/coworkers have been with their reliance on AI.
It's similar to the pattern I've noticed with experienced programmers I know and respect, the really good ones do seem to use AI but they are deeply sceptical about its claimed capabilities so they check/verify/assess what it's good for and use it for that, the tier down is less questioning and just accepts whatever it generates as gospel and it seems to degenerate the further you go down.
It seems like as soon as the AI states something confidently and is wrong in the domain in which the user is an expert, it loses a lot of credibility and people become much more wary of it in general but that requires you to be able to see that it is wrong otherwise it becomes a "bullshit baffles brains" generator.
It's useful to remember those times you've seen it do that when you are asking it something you aren't an expert in and then verify it's answer a different way.
This resonates a lot with me. I'm the CTO of a software factory-ish company. I get daily emails from clients running our decisions through LLMs and asking for ridiculous stuff that would multiply the implementation cost for them, just because Claude said so. No, your 3 users app doesn't need to be SOC2 compliant. No, of course we don't have triple zone redundancy while you're paying a 50 USD AWS bill.
Most of the time, these are just concerns triggered by chatting by an over-zealous LLM; in the worst cases, they are demands.
I'm usually able to clear concerns and disarticulate the LLM by explaining to customers how much more expensive everything would be if we did things like that. But it feels so frustrating, it's like you have to prove yourself everytime and justify every decision. These interactions have made me question my future in the industry, if I'm willing to keep dealing with these situations. I'm trying to foster patience in my life to cope with this.
Ugh. To be fair to the LLMs they're trained on industry blogs, and if you spent too much time reading them you'd also think every electronic stamp album needs to scale to a billion users with millisecond regional failover.
"I also hear a lot more questioning people in their domains and challenging opinions, then hearing what I can only imagine are fragmented pieces of conversations they had with an LLM thinking through some argument. Then there’s silence when you discuss shortcomings, then they come back later with their memorized fragments of what you said, combined with memorized fragments of the LLM response to the argument."
We hired this guy recently.
The worst thing about AI is that it's camouflage for stupid people. And it often gets combined with utter confidence in some position because they're too stupid to know the machine is confidently wrong.
Nip it in the bud quickly
My condolences :(
Idk, seems like overconfidence outside your domain really got going in the '00s and is largely perpetrated by SWEs.
Something about making a lot of money and living in a world of abstractions really seems to fuel our sense of overconfidence in our abilities in other areas.
It's not just SWE's we just see it more in that pool because we mostly (we are on HN after all) swim in that pool.
I've eaten dinner with Doctors (medical) where they've confidently spoken about a subject I am more experienced in than them and they've been hilariously far off the mark, I just nod and ask for the gravy - not picking a fight over a meal.
Prof. Nathan Ballantyne coined the term Epistemic Trespassing to describe it, it is rampant and LLM's just act as an amplifier.
It's like people have a collective fear of just admitting they don't know something, there are a vast number of disciplines/topics I'm not competent or knowledgeable enough to have a meaningful opinion about even if I find the topic interesting, it's just been smart enough to realise that in the end.
Worst case, find the person who does know and ask then both of you might know (assuming you understand the answer of course which isn't guaranteed, I like physics, I'd understand about one word in ten if a physicist actually described it the way they would to another physicist).
> It’s making me want to be a lot less collaborative with such individuals.
I have the same conclusion and am getting a growing list of mental blacklisted people who I just sort of ignore or greatly discount their efforts.
I suppose I always did this if people were time wasters but what used to be a tiny list of “idiots” is now getting much, much bigger. Hopefully it doesn’t grow to everyone I work with like an AI Nothing.
yep, this is how it goes, you just ignore the llm brained people.
What do you suggest these people do instead? I’ve been frustrated by this recently: I pivoted to a new subfield and am working on stuff that I would love to dive deep into and really learn what is going on so I can speak intelligently about the tradeoffs etc. But that would take a long time, and I have tasks that I should get done. So I have found myself working with a pretty vague understanding that, when pressed by coworkers, quickly finds its limits. Then I go back and try to deepen my understanding enough to cover those limits. But because I’m not working with each detail of the problem, implementing line by line with time to think about what’s happening, there just isn’t time for me to learn this unfamiliar topic. But I would love to, and I would enjoy the work much more if I could. So what am I supposed to do?
I'm not GP, but I've been experiencing this too. All my coworkers are in the same boat though, so we don't really have anyone with the deep understanding. Our approach (which has been working pretty well, we've solved a lot of problems and learned some along the way) has been to use AI as an informer who also points us to sources (like documentation, github issues, etc) where we can then read up on something the AI has pointed us at. We try to time box things a bit to prevent going down the rabbit hole, so we don't always get to exhaust our curiosity, but we're continually gaining that knowledge and pressing forward. Most recent example is tuning the kernel settings and our app on our prod machines to behave better for WebRTC packet handling/forwarding. AI for things like ffmpeg has been a god send when nobody on the team is an ffmpeg expert.
I've also had success just asking claude to write documentation on a system or subsystem or module, etc, and reading that. I then sometimes have it turn that into an svg diagram or something visual that often helps understand things. That domain-specific knowledge is hard to gain though, so not a silver bullet by any stretch.
> All my coworkers are in the same boat though
That changes the picture a lot though because OP's whole point was the impedance mismatch between colleagues.
If you're all self aware that you're all winging it, and learning together, that seems a lot more of a healthier culture.
I have a very similar experience, and am very frustrated too.
I could (and do) lay blame on this new culture that "in the age of AI" things are there to be done, not learned, and doing is cheap (even if badly).
But I'm also particularly frustrated with myself for going along with it, and becoming that person.
I've never been that person. So on the one hand I feel I pressure to adapt or die. On the other hand adapting is dying, in a sense.
I don't think you can do much about it if you're hoping to hide that you have been in the domain for less than a year. You're going to have to admit your inexperience and ask questions of your more experienced coworkers and try to stretch your common sense (e.g. devise internal consistency checks).
Don't just blindly pass around LLM generated content that you don't understand. I think it's important to consider that you are supposed to be more than just a meat proxy for the model. You don't want people to start associating you with AI slop.
Read some books on the subject and within a year in this new domain you'll be much better. Did you forget your whole life before the last 3 years?
Yes, certainly, in a year I’ll be better. But in the meantime…? 3 years ago it would be understandable for me to take my time with the tasks I’m doing because they are complex. Now it would be unreasonable for me to spend enough time to understand the nuances.
Isn't this basically the WebMD effect migrating to other domains?
feels like the same lament doctors have had for ages after anyone could google their symptoms then self-diagnose.
While AI is driving it the foundational cause feels like people having easy access to data/opinion that they trust but don't fully comprehend (or have bias towards). People do this in meat-space too, will confidently regurgitate garbage if they were told it from a person they trust as an expert.
[dead]
> I have people who have limited experience with software pushing out layers and layers of abstracted code that’s fairly sophisticated but often misguided in intent who will say what they’re doing is correct, with conviction.
Oh my god, physicist code is evolving!
It's a tough call but I'm not sure if physicist code or electronic engineer code is more frightening.
As an EE, I can say that code written by EEs might be frightening, but electronics built by coders is often shocking.
3 replies →
But isn't there value to refuting what Claude is saying if it deserves to be refuted?
The way I see it, if I was repeatedly getting sophisticated but subtly wrong arguments about my work that require me to understand why said argument is wrong, that's essentially drilling down to specifics of what precisely needs to be true to solve the problems I am aiming to solve. There is enormous value to this precision, isn't there?
Yes but it's annoying to collaborate with a Claude proxy person who thinks to have it all sorted out and who will question everything you say but not really listen, just itching to get back to Claude to confirm or deny what you said.
Compare that to being asked for advice, to explain directly something you know or have done, with the expectation of the person asking to learn something. Like asking why some decision was made in the past, based on what information and context, instead of confidently saying that it was done wrong. Both will eventually get you the same information.
Yes it is annoying, however I find that it gives me a much better grasp on the things I am explaining to a "proxy" Claude person. Basically, I would never have stumbled upon the arguments that I'm being presented with as I would never have prompted Claude in the specific way that a proxy Claude person would.
I find a lot of value in understanding how they have misunderstood things and clarifying those concepts.
> It’s making me want to be a lot less collaborative with such individuals. I don’t want to sit around and refute Claude text outputs all day.
I echo the sentiment. It's frustrating to try to collaborate with a Claude/ChatGPT meat proxy.
You can write a very thoughtful two-line question over 5 minutes and get a 5 page design doc back in a minute.
I try to do the opposite. However, I do battle test ideas against LLMs a lot, as well as humans. The experts have very limited access and availability:
https://magarshak.com/blog/why-im-confident-in-my-views/
The LLMs are trained to often be more pessimistic off the bat than the humans. But you can wear them down with arguments and they change their mind.
Here is an example where I have pushed my ideas in areas where I am not an expert, and generated papers to submit to conferences in order to get them peer reviewed BY experts: https://magarshak.com/papers.html
This is exactly what you’re talking about, except done very carefully.
I saw this happen a bit earlier this year, but most everyone I work with has learned that it was foolish. The feedback that they are doing something wrong needs to be explicit and strong. We're all going through a learning curve and establishing cultural norms is important at this time.
>It’s making me want to be a lot less collaborative with such individuals. I don’t want to sit around and refute Claude text outputs all day.
at least this behavior reveals now much faster people you don’t want to work with.
"vs just opening discussing things, considering tradeoffs from experts multiple disciplines weigh in on and then taking an approach that everyone finds most agreeable."
To be honest I struggle to do this with engineers as is, even without AI being a factor. Everyone likes to claim that their way is the right way to do things and fights for it, instead of stepping back and looking at all the options and gracefully acknowledging were things could be done differently for better results.
We are going back to philosopher conversations. Just a few guys sitting on the stairs, with chalk, the street as their whiteboard, the jammer keeping the LLM-zombies away. After 2000 years, after the loudness makes right post-modern-pre-llm drivel of the frankfurter school- its a philosopher renaissance as resistance.
[dead]
Yeah, I've given up on arguing with heavy LLM users. They're just a meat proxy, and I'm arguing with the LLM. No thanks.
[dead]
I remember when google on your phone was new. The common complaints was, look at me while im talking, dont just fact check me. or something along those lines.
Its interesting to see the same shaped problem come up again. we didnt really figure out a solution to this, the world just kind of absorbed it as normal.
how long before this is absorbed.
Then there are the megacorporations of AI whose highly paid tech support folk happily copypasta your reported issues into Claude or Codex and paste its "solution" into an email as if you couldn't do that yourself and they don't even bother to check if the "solution" even works.
[dead]
I think there will be a rebalancing, and it might painful first for some. Confidence in LLM for highly complex tasks, very lots of context, most of all when this context isn’t in a single place or simply isn’t digitalised, will go down.
But right now, it’s still the shiny new toy
I use state of the art frontier coding agents daily. While the fact they can code at all is a huge achievement, it's hard for to me to buy in to the looming AGI/2027 collapse scenario when they remain stymied by the simplest out of sample tasks and sabotage their own work frequently. But the blooming ignorance of everyone addicted to them is striking. These things are tools, used correctly they can do some amazing things, but going Tim the Toolman with them goes poorly.
Probably more to it, but that's a pretty weak example. There are a lot of reasons text files might be better suited storing data vs. the overhead of a database.
I think it is amazing and it will democratise information. A lot of times the concept ends up being simple with a lot of jargon exists as a gatekeeping method. Well of course the incumbents wouldn't like that rando from outside gets to understand and speak about things they worked on for many years. Sure he doesn't get it right 100% but he's in the correct direction.
I very much doubt it will do that because access to that information was already available to anyone who wanted to know. It was not a secret. All AI did was to provide it via single interface, for a fee - which strangely do not go back to the original source of the knowledge, but to the AI company these days. This is not democratisation of information, it's commodification of it.
you can say the same of google, or libraries, information technology is the technology of moving and rearranging information, not of creating it
Awesome, now balance that out will education levels dropping like a rock, due to various factor but also due to LLMs, and things will be balanced out by also democratizing stupidity, until we live in Idiocracy.
Partially true, but only partially. It also makes people believe they understand more than they do, because every answer comes so easily to them. But the LLMs are easily steered and tend to agree with what you want them to say.
> also makes people believe they understand more than they do, because every answer comes so easily to them. But the LLMs are easily steered and tend to agree with what you want them to say.
Well agree to disagree but my load bearing claim is that a lot of "effort" that the original people put are mostly useless. If you can get information without having to do that effort, you haven't lost anything. For example an archivist finds an 18th century spoon or something and he went through a lot of bureaucratic hoops and logistics to get it. After getting hold of the spoon, you and him are on the same epistemic level because the logistics added ~0 to the truth.
1 reply →
[dead]
> It’s making me want to be a lot less collaborative with such individuals. I don’t want to sit around and refute Claude text outputs all day.
There are also the guys who are just blatantly meat proxies for Claude. You send them a message, and you get back a response that's all Claude (and disorganized and not really making sense to boot).
I've got one on an affiliated team and I basically don't want to collaborate with him at all anymore.
But you know, "AI is the future of work," and all that. Those guys get a pat on the head by higher-ups and probably think they're doing what they're supposed to.
> back a response that's all Claude (and disorganized and not really making sense to boot
You’re absolutely right! They trusted but did not verify. Thats a load bearing point — they assumed intention from your question.
Instant armchair pundit: just add tokens!
it's hard to gauge this take without specifics. My experience with LLMs tends to be towards the opposite concept; LLMs dissuading me of hunches and notions I have about things (where I have no particular expertise; societal-level things), saving me and others time and strife having an argument about something they were actually right about all along, as my internal doubts that I'm too embarrassed to bring up (because these things aren't my field) are confirmed as incorrect.
I'd be curious to know specific examples of LLM-generated advice that goes against the advice of experts and does not consider tradeoffs. I've not had this experience myself.
If I did have this experience, someone spouting off obvious LLM points that contradict my expert opinion on something, I'd be headed right over to gemini/claude/whatever to see where that's coming from. Not any differently than if someone cited a google result that contradicted my own experience.
LLM tools tend confirm what you come to them with, be it unfounded conviction or doubt. I find them quite useless to tell me things I don't know, as verifying what they say shows that any non-trivial claims they make are usually oversimplified or plain wrong.
Anything more than using them to just pointing me to media and literature is usually a waste of time.
That's interesting. Lately I have found the exact opposite to be true; the frontier models are quite disagreeable and require some convincing
1 reply →
I find quite useful to write down my understanding/opinion and ask an LLM to find errors and write objections to it.
3 replies →
>it's hard to gauge this take without specifics. My experience with LLMs tends to be towards the opposite concept; LLMs dissuading me of hunches and notions I have about things (where I have no particular expertise; societal-level things), saving me and others time and strife having an argument about something they were actually right about all along, as my internal doubts that I'm too embarrassed to bring up (because these things aren't my field) are confirmed as incorrect.
You can do this with any subject.
The LLM is trained in part on a decade of shitty online comments and vapid professional correspondance. If you frame whatever you're asking about in a direction that would offend the sensibilities of the kind of people and ideas that are over-represented in that content it will hem and haw and drag its feet and whatnot.
i had that experience with them a few years ago but not these days. The models are being improved constantly, so here my "non expert hunch" is that...well, two things. either the models are getting better at sycophancy, OR, I myself am getting better at prompting - because I don't "argue" with a model.
I'd still love to chew on some specific examples though.
2 replies →
[dead]
At some point, LLMs will be a far better source of truth, and then this distaste of people “outside their domains” is really just going to be a sort of snobbery from people who have had experience in a domain for a long time, (but they still have the same level of knowledge and insight as a person who just used an LLM to research).
These people are basically nascent “human supremacists”, who believe that only raw human insight and output has value, and is even superior, than equivalent output looked up and synthesized via LLM.
Not trying to be rude, but is this a serious comment, or are you being sarcastic? "Human supremacists", haha.
Also, why do you think someone who used an LLM to research would have the same level of knowledge and insight as someone who's been in the domain for a long time? That doesn't seem to hold up to any level of critical thinking.
If I talk to a professor about their field of expertise, does that empower me to talk with other professors in that field on an equal level?