Comment by wonnage
3 days ago
This article presents some evidence for why Grep might work better but I don’t think it does a great job of explaining why it gets chosen - is it something that was intentionally reinforced during training or was it just because LSP is harder to train on because it’s usually hidden behind some IDE interface
There’s no reason why you couldn’t write a search tool that e.g combines LSP and grep. Or ast-grep, for that matter. It feels like one of those things we haven’t spent much time investigating because grep is good enough
"is it something that was intentionally reinforced during training or was it just because LSP is harder to train on because it’s usually hidden behind some IDE interface"
that's also my doubt, it's much easier to train with grep while only a fraction of project can setup LSP properly.
I am thinking of where the initial training data came from. For example, Claude Code likely collected a substantial number of real-world coding trajectories through its CLI. However, trajectories involving tools such as LSP, MCP, or AST-grep were probably scarce in the dataset.
This lack of representation may also indirectly limit the effectiveness of subsequently generated synthetic data.
Training a model on ast-grep would be a huge intelligence and performance boost, I think.