Comment by smetj
1 day ago
I think settling to otel as transport/wire-format is an excellent strategic choice offering most possibilities towards the future. Two concerns less.
1 day ago
I think settling to otel as transport/wire-format is an excellent strategic choice offering most possibilities towards the future. Two concerns less.
I'm less concerned about the wire format than reducing complexity and bottlenecks in a high-volume, high-throughput system. Needing an intermediate API just to ingest into ClickHouse adds another step where things can slow down or break, not to mention that a gRPC API just to convert JSON payloads into INSERTs is quite wasteful if you can just insert directly.
VictoriaLogs developer here. I agree with you - it is important to support multiple popular data ingestion protocols at the database for logs, so users could continue using the existing protocol instead of adding an intermediate proxy for converting from one protocol to another protocol.
That's why VictoriaLogs supports multiple data ingestion protocol [1], including JSON line protocol, which can be used for sending JSON logs into it without any conversions [2].
[1] https://docs.victoriametrics.com/victorialogs/data-ingestion...
[2] https://docs.victoriametrics.com/victorialogs/data-ingestion...