Comment by phillipcarter
2 days ago
> but I think the Otel metrics is over-engineered.
What about OTel metrics is difficult?
You can set up receivers for other metrics sources like stasd or even the DD agent, so there's no need to immediately replace your metrics stack.
One critical problem for me: no support for raw metrics.
Sometimes, you just want to export ALL of your metrics to the server and let it deal with histograms, aggregation, etc.
Another annoyance is the API, you can't just put "metrics.AddMeasurement('my_metric', 11)", you have to create a `Meter` (which also requires a library name), and then use it.
My foray into otel with aws lambda was not a success (about 6 months ago). Many of my issues were with the prom remote writer that I had to use. The extension was not reliable. Queue errors were common in the remote writer. Interop with Prometheus labels was bad. And the various config around delta and non-delta metrics was a bit of a mess. The stack I was using at least didn’t support exponential histograms. Got it to work mostly after days of fiddling but never reliably. Ripped it out and was happier. Maybe a pure OTEL stack would have been a much better experience than needing the prom remote writer - which I’d like to try in the future.
I’d certainly appreciate hearing success stories of OTEL + serverless.