Comment by mritchie712
5 hours ago
this is cool, but:
> This replaces about 500 lines of standard Python
isn't really a selling point when an LLM can do it in a few seconds. I think you'd be better off pitching simpler infra and better performance (if that's true).
i.e. why should I use this instead of turbopuffer? The answer of "write a little less code" is not compelling.
This line comes from a specific customer we migrated from Elastic Search, they had 3k lines of query logic, and it was completely unmaintainable. When they moved to Shaped we were able to describe all of their queries into a 30 line ShapedQL file. For them the reducing lines of code basically meant reducing tech-debt and ability to continue to improve their search because they could actually understand what was happening in a declarative way.
To put it in the perspective of LLMs, LLMs perform much better when you can paste the full context in a short context window. I've personally found it just doesn't miss things as much so the number of tokens does matter even if it's less important than for a human.
For the turbopuffer comment, just btw, we're not a vector store necessarily we're more like a vector store + feature store + machine learning inference service. So we do the encoding on our side, and bundle the model fine-tuning etc...
> > This replaces about 500 lines of standard Python
> isn't really a selling point when an LLM can do it in a few seconds.
this is not my area of expertise, but doesn't that still assume the LLM will get it done right?
Shorter code is easier to understand and maintain, for both man and machine
This idea that it no longer matters because Ai can spam out code is a concerning trend.