Comment by vessenes
14 days ago
My son assessed it for a class a few years ago after finding out it wouldn’t give him “con” view points on unions, and he got interested in embedded bias and administered the test. I don’t have any of the outputs from the conversation, sadly. But replication could be good! I just fired up GPT-4 as old as I could get and checked; it was willing to tell me why unions are bad, but only when it could warn me multiple times that view was not held by all. The opposite - why unions are good - was not similarly asterisked.
I hope on HN that we hold ourselves to a higher standard for “it’s been true from the beginning” than a vague recall of “My son assessed it for a class a few years ago” and not being able to reproduce.
I literally went back to the oldest model I could access and hand verified that in fact it does what I described, which is lecture you if you don't like unions and goes sweetly along if you do like unions. I feel this is a fair and reasonably well researched existence proof for a Saturday afternoon, and propose that it might be on you to find counter examples.
You made a claim about political surveys, and linked one in particular, providing a labeling of the tool.
Your follow up response did not reference any of those surveys and did run through the types of questions on those surveys. You apparently only did questions about unions.
Is that what you would fair and reasonable?
They were referring to your original claim about Pew research assessing the models as highly liberal when that’s apparently not even one of their ratings.
This is clear because they referenced your quote about it being from the beginning.
No one was arguing that you typed in a question about unions.
The GP put in the work to verify his own memory, after acknowledging the gaps. And then you belittled him.
He met the “standard” or guidelines of our community in a way you have not.
>The GP put in the work to verify his own memory, after acknowledging the gaps.
The original claim didn’t say anything about it being the experience of their son for specific questions about unions. It was much broader than that. And at least partially inaccurate, given the stated result isn’t even one of the results.
>And then you belittled him.
If asking for a higher standard of evidence for a broad claim than referencing a previous experience and then trying again, but not even sharing the link from a tool that makes it easy to share the conversation from, is considered belittling, then maybe the castrations going on in these models is the right way to go for this crowd. I, personally, aim for a more truth-seeking standard.
>He met the “standard” or guidelines of our community in a way you have not.
These are two different things, and you clearly understand that but are intentionally conflating them. Regardless, if this is where are, maybe HN no longer is the place for me.
That claim isn't something Peter made up, it's the claim made by Meta's own researchers. You're picking an argument with them, not HN posters.
Anyway it's trivially true. I think most of us remember the absurdities the first generation LLMs came out with. Prefering to nuke a city than let a black man hear a slur, refusing to help you make a tuna sandwich etc. They were hyper-woke to a level way beyond what would be considered acceptable even in places like US universities, and it's great to see Facebook openly admit this and set fixing it as a goal. It makes the Llama team look very good. I'm not sure I'd trust Gemini with anything more critical than closely supervised coding, but Llama is definitely heading in the right direction.
Peter’s claim I was asking about was one about being labeled as something via a Pew research or similar survey. And the response I got was about their personal experience asking a questions about unions. Do you think that those are the same claims and equivalent?
>Prefering to nuke a city than let a black man hear a slur, refusing to help you make a tuna sandwich etc. They were hyper-woke
On its own, all this tells me is that the non-human, non-conscious tool was programmed specifically to not say a slur. To me that seems like something any reasonable company trying to create a tool to be used by business and the general population might incorporate while it is still learning to otherwise refine that tool.
And I took the Pew survey mentioned above and it didn’t ask me if I would say a racial slur.
Finally, if anyone, from any point on the political spectrum, thinks that a tool being limited to not respond with racist terms, is a reflection of its overall political leaning, I suggestion you look inward.
[flagged]