Comment by tuckwat
2 days ago
Do you have bias? I'm not saying I don't have some hidden bias but I have no skin in the game. I don't use Bun or Zig or plan to.
My reflection comes after reading Jarred's post yesterday, which I found interesting, and then Andrew's today.
I just pasted this article into an LLM to understand the tone and the summary is:
> The overall tone is deeply personal, cathartic, biting, and polemical, with flashes of humor and a deliberate attempt to soften the ending.
Did you have a hard time picking that up from the article itself? I don’t see the point of asking an LLM to tell you what to feel about the article.
I'm rather critical of spamming humans with low effort LLM writing as well, but getting a neutral opinion about a text seems like a decent use tbh - for lack of alternatives because you'd hardly ask a human to do it.
Conventions are still being made, and I think this use might end up being acceptable.
How is pasting an article into an LLM going to get you a neutral opinion? It'll be at best an 'opinion' that aligns towards the fine tune dataset used by the org that made the LLM. I'd rather people own their own bias and bring something into the conversation rather than act as a mouthpiece for a statistical median.
i think only if you contrast it with your own assessment. just using LLM to get an opinion is bad. eg: after reading the article i felt this X. LLM says it's Y. then draw conclusions from that.
myself i found the article contained a bit to much personal criticism. the kind that eg. on hackernews would not be welcome. so i guess i mostly agree with the LLM assessment.
an enormous number of people have never taken a literature class and in fact have no idea how to assess tone, diction, rhetorical purpose, etc.
I'm glad that you asserted that you do have potential hidden bias and say that you used an LLM to judge the tone of an article partly pertaining to AI usage in coding right after.
Let me check real quick with ChatGPT if you are being sarcastic or not.
Edit: Ah, my brain's been fried by comments that say this unironically and I read this as sincere, thanks commenter below, and apologies commenter above.
They are saying that someone who will summarize an article with AI is already implicitly biased. It's not about the AI, it's about the user.
I think I have some bias. I like rust. I like using opus, I don't want to use zig, don't see the point for it, but I find it aesthetically pleasing. I don't use JavaScript I wish it would disappear. If I did ise JavaScript I wouldn't run it on bun. Some of those tilt me either way. I agree with the LLMs take on the tone. The result of the two posts is that I think Jarred is a poopyhead, andrew is a bit childish but genuine and I'd probably like him, and I still won't use zig or bun or JavaScript. I plan to keep using rust and opus.
> I just pasted this article into an LLM to understand the tone
That's your brain's job, don't outsource it.
Uhm, I'm very confused by the last part of your comment. You put it into an LLM to...understand the tone? Is that supposed to convince me of something?
They phrased it poorly, but from context it seems clear they intend the LLM as a less biased third-party measure of the tone which agrees with their own assessment.
which they very much are not
2 replies →
Reality check. The confirmation that the author is not alone in having this particular feeling by using an LLM as a proxy.
Using an LLM as a proxy for an 'unbiased' source of personal confirmation bias is perhaps the most common 'use-case' for an LLM.
I guess they use an LLM to work out the tone of prose they read. These are strange times.
Very weird indeed. People must not realize that you can completely change the response you get back from an LLM by how you ask questions. Any bias can implicitly be implanted in the question you ask and drastically modify the response. This is what I got Gemini to say about the article:
very different from "The overall tone is deeply personal, cathartic, biting, and polemical, with flashes of humor and a deliberate attempt to soften the ending."
Not very different, mind you.
I don't really agree with using LLMs to do this but it correctly identified the attempt to soften the ending, which is to my mind significant in the whole piece; this person wants to repeat and frame unkind things he's heard, say unkind things, and then assert that he wasn't doing either.
Personally I don't think that matters, because the article is problematic enough when it can be read like ad hominem. Assuming that the question was phrased reasonably neutrally (but not necessarily free of hidden biases), the fact that LLM concluded so is an enough evidence here. Also as a non-LLM data point, I felt roughly same (especially the "softening" bit).
this is what worries me. I have friends that love what AI tells them about their personal pet 'thing' and how awesome it is. Yet not one of them has even tried once to get the AI to criticise it's own answers, and hence learn that you can trivially get an AI to make a convincing-_sounding case for any point of view.
I tell them to try, and they laugh at me as they roll their eyes and waffle on about 'tricking' the AI like its some kind of hacking.
Bun is better in rust and they can finally break up their relationship.
It's not a happy breakup, but not a super sad one either.
Please, I'm begging you, and the people that scan across this comment: Finishing mastering reading comprehension; It will help you for the rest of your life.
I'm not snarking, this is a problem affecting 30% or more of the population here in the states, and it's getting worse because of tools like AI. I'm not judging you, I don't think you are bad or deficient people, but this externalsing of comprehension and trust is self-harm.
Worse, it will lead you astray in ways that you won't tie back to this core problem.
To use an LLM safely you must have the discernment to understand when it has fallen into sycophancy, folly, or madness.