Comment by CubsFan1060
14 hours ago
What does the layout in click house look like? Do the input logs need to have a very defined structure?
14 hours ago
What does the layout in click house look like? Do the input logs need to have a very defined structure?
Not really, ClickHouse is super forgiving so you can do something like:
ClickHouse is extremely performant even in the cases of e.g.: SELECT count(*) FROM `events` WHERE `raw` LIKE '%hello world%'
Of course, the more columns you splat out (e.g. like correlation_id, user_id, order_id, etc) the better you can index and expect those queries to perform but in general I don't bother outside the obvious core domain ones (exampled above), the performance is so good that unindexed queries are significantly faster than indexed queries in Loki. I have reached the point where I JSON extract on-the-fly for the WHERE clause with very large queries with no meaningful performance issues.