SQLite is abolutely amazing as an app format! I couldn't list how many tools are available to read SQLite data, or how easy and friendly they are. Even its CLI does wonders when you're dealing with data with it. SQLite has been around for 20+ years and is one of the most heavily tested softwares in the world.
SQLite is very simple, yet very reliable and powerful. Using SQLite as file format might be the best decision an engineer can take when it comes to future-proofing preservation of data.
Actually used it for a desktop blogging app a few years ago. It was great! I could set up a blog skeleton, send the file to a family member. They could focus on writing content and hitting deploy.
I remember someone mentioning the Acorn image editor on Mac uses sql files to store image data. It probably makes backwards compatibility much easier to work with.
It does, here's a schema from an image I just saved with the latest version. Pretty simple.
CREATE TABLE image_attributes ( name text, value blob);
CREATE TABLE layers (id text, parent_id text, sequence integer, uti text, name text, data blob);
CREATE TABLE layer_attributes ( id text, name text, value blob);
Also, document-based apps that use Apple's Core Data framework (kinda ORM) usually use SQLite files for storage.
Bit unrelated rant but I'm still not sure why ZIP has been adopted as an Application File Format rather than anything else. It is a remanent of a DOS era with questionable choices, why would you pick it over anything else?
Also architecturally suitable for the common case of collecting heterogeneous files in existing and new formats into a single file, as opposed to designing a database schema or a complex container structure from scratch.
Any multi-file archive format would do, but ZIP is very portable and random access.
Because Windows can view and extract them out of the box without installing any additional applications. If it supported anything better out of the box I'd guess people would use that instead.
gzip is not an archive container. You're thinking of .tar.gz which is a "tape archive" format which is compressed using gzip. Zip is by itself both a compression and an archive format, and is what documents like epub or docx use
Somehow my first thought from the title was using sqlite as a format for applications. So like a replacement for ELF. I think this idea is both fascinating and horrifying.
I worked @fzakaria on developing that idea. It actually worked surprisingly well. The benefits are mostly in the ability to analyze the binary afterward though rather than any measurable benefit in load time or anything like that though. I don’t have the repo for the musl-based loader handy, but here’s the one for the virtual table plugin for SQLite to read from raw ELF files: https://github.com/fzakaria/sqlelf
SQLite is abolutely amazing as an app format! I couldn't list how many tools are available to read SQLite data, or how easy and friendly they are. Even its CLI does wonders when you're dealing with data with it. SQLite has been around for 20+ years and is one of the most heavily tested softwares in the world.
SQLite is very simple, yet very reliable and powerful. Using SQLite as file format might be the best decision an engineer can take when it comes to future-proofing preservation of data.
I see no downside in using sqlite as an application file format.
Actually used it for a desktop blogging app a few years ago. It was great! I could set up a blog skeleton, send the file to a family member. They could focus on writing content and hitting deploy.
https://blog.project-daily.com/pages/file-format_3705.html
I think I use SQLite like that (to some extent):
- https://github.com/rumca-js/Internet-Places-Database
For UI I use HTML, because it already provides components with bootrap, and everybody can use it without installation of any software.
All data comes from a single SQLite that is easy read, and returns data.
My database is really big, so it takes time to browse it, I wanted to provide more meaningful way to limit scope of searching
I remember someone mentioning the Acorn image editor on Mac uses sql files to store image data. It probably makes backwards compatibility much easier to work with.
It does, here's a schema from an image I just saved with the latest version. Pretty simple.
Also, document-based apps that use Apple's Core Data framework (kinda ORM) usually use SQLite files for storage.
Previously: https://news.ycombinator.com/item?id=23508923
[flagged]
The previous discussion usually has useful and interesting conversations that are nice to revisit.
1 reply →
It’s a helpful link, not a criticism.
4 replies →
Bit unrelated rant but I'm still not sure why ZIP has been adopted as an Application File Format rather than anything else. It is a remanent of a DOS era with questionable choices, why would you pick it over anything else?
- archiver format to stow multiple files in one; your actual files (in your choice of format(s)) go inside
- files can be individually extracted, in any order, from the archive
- thousands of implementations available, in every language and every architecture. no more than 32KiB RAM needed for decompression
- absolutely no possibility of patent challenges
Also architecturally suitable for the common case of collecting heterogeneous files in existing and new formats into a single file, as opposed to designing a database schema or a complex container structure from scratch.
Any multi-file archive format would do, but ZIP is very portable and random access.
Because Windows can view and extract them out of the box without installing any additional applications. If it supported anything better out of the box I'd guess people would use that instead.
"The operating system makes it easy to mess with" doesn't seem like a particularly useful property for application file formats.
1 reply →
It works well enough. What could, for instance, epubs gain by having another base format instead?
I think most format use "gzip" instead of "zip".
gzip is not an archive container. You're thinking of .tar.gz which is a "tape archive" format which is compressed using gzip. Zip is by itself both a compression and an archive format, and is what documents like epub or docx use
Searched for this topic:
> and is backwards compatible to its inception in 2004 and which promises to continue to be compatible in decades to come.
That is pretty amazing. You could do a lot worse.
Same as .zip, .xml, .json and many others.
Doesn't mean that whatever the app stores inside will remain backward compatible which is the harder problem to solve.
Still helpful!
Somehow my first thought from the title was using sqlite as a format for applications. So like a replacement for ELF. I think this idea is both fascinating and horrifying.
I worked @fzakaria on developing that idea. It actually worked surprisingly well. The benefits are mostly in the ability to analyze the binary afterward though rather than any measurable benefit in load time or anything like that though. I don’t have the repo for the musl-based loader handy, but here’s the one for the virtual table plugin for SQLite to read from raw ELF files: https://github.com/fzakaria/sqlelf
Forget elf, imagine having a SQLite file that stores elf, exe and DMG binaries. I would not mind working on something like this.
wonder if this would make hot-swap functions easier, if every function had its own section and every section was in the db