Comment by Retric

Even individual words can be an issue. Humans understand when you want a new story about a magic school there’s issues using Hogwarts as the name.

LLM’s will happily regurgitate multiple paragraphs practically verbatim if you don’t specifically design them not to behave in this way.

Because ultimately when you design a system to best predict the next output based on a given input perfect replication of prior training is the correct response mathematically.