Comment by MarkusQ

My point was, LLMs always generate fiction (bad fiction, but fiction nonetheless); what they generate is plausible, grammatical text that may have many points of congruence with the truth, but are not in any way constrained to do so. In short, fiction.

If they could reliably generate high quality non-fiction, that would be news.