Comment by HarHarVeryFunny

> What would be more fascinating is for it to explain how it came to these conclusions

The thing is that it didn't "come to these conclusions". The "conclusions" were present pre-formed in the training data, and what it spat out was some statistical mashup of that.

If you asked the model how it came to these conclusions, you'll get more of the same - it'll look at what it spat out and re-ingested and give you a statistically plausible mashup of how the authors of the training data might have said they came to such conclusions.

It's all just prediction - "what would the training data say?" (taking into account what was reinforced/amplified by post-training).