Comment by hrpnk

+1 on the OTel mention. Having telemetry in place in a system, one would typically implement a single behavioral tracking SDK on top. Adding yet another SDK for LLMs is a hard ask given how specific the implementation will be. Backing back on a standard you offer value-added insights on top.

On the other hand, if you target just the applications that implement an API behind an LLM, you will have customers expecting value-added services on top of telemetry, like prompt optimization, classification, result caching, etc.

Your choice which direction and target group you will focus on first.