Comment by raw_anon_1111
7 hours ago
Redshift is used at the largest e-commerce site in the world and was built specifically to “shift” away from “Big Red” (Oracle).
7 hours ago
Redshift is used at the largest e-commerce site in the world and was built specifically to “shift” away from “Big Red” (Oracle).
What can I say, I expected more than what they actually offer. A Redshift job can fail because S3 tells it to slow down. How can I make this HA performance product slower given its whole moat is an S3 based input output interface.
As a compute engine its SQL capabilities are worse than the slowest pretend timeseries db like Elasticsearch.
Are you trying to treat an OLAP database with columnar storage like an OLTP database? If you are, you would probably have the same issue with Snowflake.
As far as S3, are you trying to ingest a lot of small files or one large file? Again Redshift is optimized for bulk imports.
Redshift does not fit into aws ecosystem. If you use kinesis, you get up to 500 paritions with a bunch of tiny files, now I have to build a pipeline after kinesis that puts all of it into 1 s3 file, only to then import it into redshift which might again put it on s3 backed storage for Its own file shenanigans.
Clickhouse, even chdb inmemory magic has better S3 consumer than Redshift. It sucks up those Kinesis files like nothing.
Its a mess.
Not to mention none of its Column optimizations work and the data footprint of gapless timestamp columns is not basically 0 as it is in any serious OLAP but it is massive, so the way to improve performance is to Just align everything on the same timeline so its computation engine does not beed to figure out how to join stuff that is Actually time Aligned
I really can’t figure out how anyone can do seriously big computations with Redshift. Maybe people like waiting hours for their SQL to execute and think software is just that slow.
1 reply →