Comment by JCharante

your (or anyone's) pre-training data isn't really useful so don't worry, people overestimate the utility of unstructured data