Comment by HenryNdubuaku

100%, data was honestly most of the work, Needle 3 is trained on 360B tokens of structured data and we spend way more time on the generation pipeline than on the model. On Led Zeppelin, Needle doesn't actually need to know it, arguments are copied from the request so it just lifts the name into the artist field. The knowledge went into the engram btw, 70M of the 121M params are n-gram tables, so it can tell artist vs song without an MLP. Also yes, "play their second album" won't work, that needs the world knowledge it doesn't have.