Comment by piker
2 days ago
This looks awesome.
Why Python? Probably because most of the SDKs are python, but something that could be ported across languages without requiring an interpreter would have been really amazing.
2 days ago
This looks awesome.
Why Python? Probably because most of the SDKs are python, but something that could be ported across languages without requiring an interpreter would have been really amazing.
That's the key question. It feels like many of these tools are trying to solve a systems-level problem (cross-language model execution) at the application layer (with a Python library).
A truly universal solution would likely need to exist at a lower level of abstraction, completely decoupling the application's language from the model's runtime. It's a much harder problem to solve there, but it would be a huge step forward.
for js/ts you have vercel aisdk [0], for c++ you have [1], for flutter/reactnative/kotlin there is [2]
[0] https://github.com/vercel/ai
[1] https://github.com/ClickHouse/ai-sdk-cpp
[2] https://github.com/cactus-compute/cactus
we essentially built the gateway as a service rather than an SDK: https://github.com/portkey-AI/gateway