Comment by specproc

I see LLMs best used as part of a more traditional NLP pipeline.

For example, an approach that does me well is clustering then using LLMs on representative docs. Tools like bertopic are great for this.

I also don't see a clear cut difference between the two in certain areas. Embeddings are critical in LLM pipelines, but for me anyway, also "old school" tools.

I think NLP as described in the article is certainly under threat, but the tools and approaches compliment LLM use well, are far more efficient, and distinguish the pros from the neophytes.

If you're using LLMs for NLP-type tasks, but don't know the NLP tools, you're missing out.