Comment by david_shaw
20 hours ago
Nice work! I'm sure the data gleaned here is illuminating for many users.
I'm surprised that there isn't a stronger demand for enterprise-wide tools like this. Yes, there are a few solutions, but when you contrast the new standard of "give everyone at the company agentic AI capabilities" with the prior paradigm of strong data governance (at least at larger orgs), it's a stark difference.
I think we're not far from the pendulum swinging back a bit. Not just because AI can't be used for everything, but because the governance on widespread AI use (without severely limiting what tools can actually do) is a difficult and ongoing problem.
I had to vibe code a proxy to hide tokens from agents (https://github.com/vladimirkras/prxlocal) because I haven’t found any good solution either. I planned to add genai otel stuff that could be piped into some tool to view dialogues and tool calls and so on, but I haven’t found any good setup that doesn’t require lots of manual coding yet. It’s really weird that there are no solutions in that space.
nice, I'm working on something similar with macroons so the tokens can be arbitrarily scopes in time and capability too.
Mine uses an Envoy sidecar on a sandbox container.
https://github.com/dtkav/agent-creds
Yes, I was just thinking about how, as engineers, we're trained to document every thought that has ever crossed our minds, for liability and future reference. Yet once an LLM is done with its task, the "hit by a bus" scenario takes place immediately.
Yes, I think you can actually later store this in a database and start querying and optimizing what is happening there. Even you can start using these files or a destilation of these as long term memory.