Comment by mikeshi42

6 months ago

Luckily ClickHouse and serious throughput are pretty synonymous. Internally we're at 100+PB of telemetry stored in our own monitoring system.

Vector supports directly writing into ClickHouse - several companies use this at scale (iirc Anthropic does exactly this, they spoke about this recently at our user conference).

Please give it a try and let us know how it goes! Happy to help :)

6 comments

mikeshi42

Reply
atombender  6 months ago

Thanks! Very familiar with ClickHouse, but can logs then be ingested into CH without going through HyperDX? Doesn't HyperDX require a specific schema that the Vector pipeline would have to adapt the payloads to?

  • mikeshi42  6 months ago

    Nope! We're virtually schema agnostic, you can map your custom schema to observability concepts (ex. the SQL expression for TraceID, either a column or a full function/expression will work).

    We don't have any lock in to our ingestion pipeline or schema. Of course we optimize a lot for the OTel path, but it works perfectly fine without it too.

    • atombender  6 months ago

      Very happy with CH, but I'm sadly disappointed with HyperDX:

      * Not possible to reorder columns?

      * Not possible to wrap cells?

      * The doesn't seem to be a concept of "field popularity" as Kibana has (where you can also "pin" fields)?

      * The log view's chart is very simplistic. No breakdown. Time selector is very primitive. Look at Google Cloud's log view if you want to see something good here.

      * No field value autocompletion when using the query builder?

      * Live view is annoying as hell. It scrolls to the top every few seconds even when there is no need data.

      * Chart view is nearly useless. I tried to create a chart showing two time series calculations, average and P95 of a metric, and it doesn't draw it correctly, and the series get messed up, and overall I think it's not usable. Happy to explain further, but even a cursory test should reproduce this. Fortunately Grafana can access CH data and do a much better job here.

      * The drill-down sidebar doesn't seem to have any idea what's important or not, so some fields I will never want to filter on are high up in the list and others are further down. I can't rearrange the fields?

      * Lots of other nuisances.

      Overall I found it much, much weaker than Kibana for both logs and charts, and Kibana is already pretty atrocious at both, so.

      I can probably live with it because I'm desperate to replace Elastic with CH, but I think I will kiss the functionality of the Kibana UI.

      2 replies →