Comment by denom
5 hours ago
Ducklake works well. So far it is still under active development (post AWS acquisition). There is support for local disk-based and remote S3 based datalakes, so integration tests become really easy. Seems stable and all the bugs I've found have already been patched.
Any risks in terms of using it on gcp/azure?
Praying for the poor Ducklake employees at AWS to keep whatever demon works there at bay who insists that every single data product has to be bolted onto AWS Glue (and its various Data Catalog, Data Lake, etc. partners in crime) in the most clumsy way possible. And please keep it away from Athena, possibly the worst major query engine in terms of support for query features and file formats. I don't want to see AWS-era Ducklake become another failure like S3 Table Buckets are (imo at least).