Comment by NichoPaolucci
16 hours ago
I don’t think they’re “talking down”. If anything - it’s way more difficult to distill something into a genuinely easy to digest format. I personally think that they aren’t immediately capable of this, and so we get word salad instead. Extra prompting required to strip extraneous prose out.
Maybe I am dumb and it IS talking down to me, but there have been many occasions where I’m reading AI generated docs / plans and it makes absolutely no sense, but looks really in depth at a glance.
It doesn't seem like word salad as such. There's normally a coherent point expressed, it's just obscured by circuitous sentence structures, unusual word choices, "verbing weirding nouns", metaphors, etc. Could be a result of training that rewards novel/surprising language, but it also feels like it could be an artifact of models imperfectly compressing high-level multidimensional reasoning into language that's easy for them to process but cognitively taxing for humans.
Or, maybe they are dumb using smart words to sound smart in RLHF.
They aren’t reasoning though, are they?