Comment by wis
1 year ago
But the article mentions that they intend to have full compatibility:
> Our goal is to build a reimplementation of SQLite from scratch, fully compatible at the language and file format level, with the same or higher reliability SQLite is known for, but with full memory safety and on a new, modern architecture.
author here: fully compatible at the language and file format level.
Further down the post I actually call out explicitly that we do intend to get rid of some of the baggage.
If you "intend to get rid of some of the baggage" you won't be fully compatible.
libSQL already isn't fully compatible: as soon as you add a RANDOM ROWID table, you get "malformed database schema" when using the (e.g.) sqlite3 shell to open your file (also Litestream doesn't work, etc).
And that's fine, as there probably is no better way of doing what you needed to do. But it's also taking what SQLite offers and breaking the ecosystem, under the covers of "we're compatible" without ever calling out what compromises are being made.
Note how the SQLite documentation that introduces STRICT tables very clearly documents the backwards compatibility issues of the feature and how to get around them: https://sqlite.org/stricttables.html#backwards_compatibility
You also never got round to documenting the internal Virtual WAL APIs you exposed. This is something where SQLite is lacking, where you could've made an impact without any compatibility issues, and pressure upstream to release something by doing it first/better. Alas, you did it for Turso's exclusive benefit.
That's fine though. Full compatibility doesn't have to mean full backwards compatibility. I think of it as what's Typescript to Javascript.
1 reply →
i would guess "full memory safety" is going to be impossible, at least at compile time. I'd guess that if for no other reason than performance SQLite uses data oriented techniques that effectively reduces pointers to indices, which will no longer have ownership or lifetime tracking in the rust compiler.