Comment by hodgesrm
2 years ago
> You should be using ReplacingMergeTree if you are doing updates at the current moment.
Indeed. Altinity and other community users like ContentSquare made numerous contributions to make it more usable. It's a promising approach to updates at scale and has improved markedly over the last few months.
That said you can't currently use RMT very efficiently in S3 because of overall limitations in MergeTree S3 table storage. We need to think about whether the improvements we're proposing will also enhance RMT. Thanks for bringing that up.
Yeah, its the don't treat S3 like a disk issue. I'm looking at S3 only for cold storage but need AWS VPC Gateway Endpoint support for S3 access since we are on-premise.
*Yes you can use a vpc gateway but need public IPs to waste to setup the BGP/IP routes.
What's the "AWS VPC Gateway Endpoint" support you refer to? Do you mean an S3 proxy? If so one of our PRs that just merged may be helpful. [0]
[0] https://github.com/ClickHouse/ClickHouse/pull/51749
I don't think that will work. Here is the issue logged. This is about a networking feature used for AWS Direct Connect users to access S3 over a dedicated private connection.
https://github.com/ClickHouse/ClickHouse/issues/53761#issuec...
1 reply →