Comment by wvh
2 days ago
What's your opinion on OTel when trying to keep things small and performant? I've got some experience working with OTel the last few years, and I'm a bit afraid of the expanding scope and complexity compared to "simpler", more targeted solutions, like for instance Vector.
I'm just asking because you mention OTel and "other ways" in your post, and you must have a good overview over the options and where the market is headed.
It's actually not clear to me that Vector is any simpler than OTel imo (VRL is way more complicated than OTTL for instance). You can also use otel collector builder (ocb) to build a slimmed binary.
My take is that OTel is overall the best investment, it's widely supported across the board by many companies and other vendors. It's also constantly being improved with interesting ideas like otel-arrow which will make it even more performant (and columnar friendly!)
We'll also continue invest in the OTel ecosystem ourselves in making it easier and easier to get started :)
That being said, I'm not saying that OTel collector is always the right choice, we want to meet users where they are. Some users have data that gets piped into S3 files and we ingest off of a S3 bucket just due to how they've collected data, some use Vector due to its flexibility with VRL, focus on logs, or specific integrations it provides out of the box. So the answer is always - it depends :) but I do like OTel and think the future is bright.