Comment by hobofan

16 hours ago

I'm glad that DuckDB has a foundation in place and hope it is resilient enough to push the DB forward when the time comes.

Out of all the big orgs, Amazon is probably the one that has the least regard for keeping technically interesting projects alive, and the certainly will bulldoze it for some dumb reason when the next re-org comes.

I think 'least regard' belongs to Oracle. For some reason I thought DuckDB was one of Gabe's ideas over at DuckDuckGo :-).

That said, the acquisition by Amazon is interesting in that it seems to overlap with their existing DB offerings? Which makes me wonder what exactly it is that they are getting out of this.

Surely they are better than Google in this regard.

  • AFAIK Google is much better at not tearing well-working teams apart in reorgs, and letting people go they have acquihired a few quarters earlier.

    Of the few examples I have I my head, I'd even say that the fate of a product that has been acquired by Google is probably even better than those of Google's internally developed products. e.g. Waze is still alive and kicking 13 years after acquisition under its own brand and hasn't been completely swallowed by Google Maps. The Nest brand also stuck around for quite some time.

    • Google is better but it's an incredibly low bar.

      Amazon will drive these people to quit within a couple years over filling out MBRs and threats about how the MBR isn't good enough (MBR = monthly business report, pure bullshit theatre that drives the whole company mad 2/4 weeks every month).

      Google would drive them to quit over a longer timeframe, with no threats or harrassment, just because every time they try to ship something cool, someone else has a reason not to do it and that kills you eventually.

      11 replies →

    • To be fair, I suspect most of the core team is effectively rich after an acqu-hire.

      If I had 5 million I’m going to retire and never code for money again.

      I do want to make small video games though, make some music. Pay for a friend’s kids college.

      I wouldn’t waste a single extra hour making more money. Usually the golden handcuffs fall off after a year or two. That’s why Heroku went to crap, all the core people left.

  • Google has such a much better reputation when it comes to contributing to and understanding open source. It’s unfair to Google to put Amazon above them.

    Heck AWS is the sole reason all these projects needed to go through these license changes to prevent AWS from completely destroying their business models.

    • Google had a better reputation. But these days I’m really not convinced they’re significantly better than Amazon.

      Weirdly I’d put Microsoft above modern-Google, and that’s still a low bar.

    • What business model do MIT licensed projects even have? If you want to offer support contracts and actually make people use them, don't use MIT. You can't have you cake (accepting public contributions and free labor) and eat it (sole profiteering) too.

      14 replies →

    • ..a strategy valid in the good ol' days when a complete rewrite of the currently non-licenseable non-protectable underlying desings and ideas would not cost you 200$ on tokens to reimplement in a different or even in the same language.. maybe SW patents will be a thing soon .. and inadvertently become another nail in the coffin of FOSS and SW development in general, who knows

  • One of the things that has hampered the adoption of Dart/Flutter has been the fear Google would abandon it. I always presumed them to be in the lead regarding this kind of shittiness, but this is based on gut feel rather than analysis.

> Out of all the big orgs, Amazon is probably the one that has the least regard for keeping technically interesting projects alive

I'm curious what you mean by this. I would have said the exact opposite - AWS tends to keep projects around for a very long time. They haven't acquired very many open-source projects, but the few that they have are all still running as far as I know.

  • Fig, was one of them but was only partially opens source IIRC. CLI autocomplete and script team sync. Turned into/integrated into Amazon Q, which turned into Kiro CLI, which AFAIK doesn't share any functionality with the original product nowadays.

Yup, they are very serious about DynamoDB and it is a very good product. Wish they released more information about it, though.

  • Man I admire AWS marketing team for convincing people DynamoDB is a very good product.

    100 items max per Transaction BatchGet 100 items, 16 MB max low write limits on same key Item size 400 KB max etc.

    • DynamoDB has basically two legitimate use-cases that I'm familiar with:

      1. You're selling a system to a customer to use within their own AWS account, that you will have no access to, and it needs a transactional datastore (not just an object bucket) of some kind. The fact that it costs nothing by default (particularly valuable when the customer is trying to deploy a proof-of-concept), scales more-or-less perfectly without anybody touching it, requires zero day-to-day maintenance by you or the customer, and all it will ever ask is that you throw money at it, is very, very much a feature. One example I'm familiar with in the wild is Teleport: https://goteleport.com/docs/reference/deployment/backends/#d...

      2. You have a huge OLTP workload that fits Dynamo's KV patterns (e.g. Amazon.com shopping carts, which is what it was originally built for). You don't care how much DynamoDB costs (in either dollars or engineering limitations) because any alternative would melt your face off if you even tried.

      Most of the pain that comes from Dynamo is people who try to use it as a primary datastore in place of a relational database just to get the serverless pricing model. It's not worth giving up the flexibility on greenfield systems. It does become worth it to give up the flexibility when your system is mature and you don't have genuine flexibility anymore anyway.

      1 reply →

    • Some of the largest, highest TPS, highest throughput systems in the world use DDB.

      The constraints are what let this happen. Unconstraining it might make a better generalist product but part of what you're opting into with DDB is the dumb "put an item in get an item out semantics" and the other side is knowing that it will still work if that volume increases dramatically.

      1 reply →

    • The reason it's good is because, although it is bad, you never have to worry about it in terms of maintenance, admin or scale. It's like lambda for databases.

      For the same price, you can run a much more capable PSQL instance with way better features, but now you're on the hook for it being up 24/7.

      3 replies →

    • Thanks, I don't really get any chance to seriously use or manage OLTP databases (mostly working with OLAP ones myself) so it is good to know about this. The 400KB limit is indeed very limiting -- I had to truncate some data because of that.

      I wish I could get a role to work on OLTP databases. PostgreSQL seems to be a fascinating topic so that's on my plate.

  • They have published 2 academic papers about it. What more do you want, operational details?

    • For example, I would like to know how to calculate the charge of incremental export before I run one.

      One blog says it is charged by the amount of change logs (but the official doc doesn't say so), which makes sense. But how do I estimate the amount? My hunch is: Put + Write + (1~100) * TransitWrite + Update + Delete + (1~25) * BatchWrite.

  • https://www.amazon.science/publications/dynamo-amazons-highl...

    • It is their previous one. I did read another paper about DynamoDB, but still there is not much details.

      For example, I want to know how to calculate the charge of incremental export. One blog says it is charged by the amount of change logs (but the official doc doesn't say so), which makes sense. But how do I estimate the amount? My hunch is: Put + Write + (1~100) * TransitWrite + Update + Delete + (1~25) * BatchWrite.

What good is a foundation when it's core member is owned by a public corporation? Like we know what happens, the pattern isn't unique or rare. Corporate interests will always take priority over community interests, corporations care about profit not community. If Amazon cares about the community as they care about their own workforce, that does not bode well.

> Amazon is probably the one that has the least regard for keeping technically interesting projects alive

Well ...

https://killedbygoogle.com/

Either way that news is awful. Europeans selling out to US corporations and then wondering why Europe is not competitive in software engineering anymore. Funniest thing is Draghi keeps on saying that; well, recently we also heard that the USA has full access to all national police databases in the EU now. It seems as if Washington remote-proxy-controls the EU. Quite amazing to see, too. The amount of bribe money flowing must be legendary.