Comment by whartung
1 year ago
What has been your experience writing the "off line first" style PWA?
There's an appeal to it, basically using the web purely as distribution, and the browser as the runtime. But I'm concerned with the fiddly-ness of local storage and such like that. The "out of site, out of mind" nature of it. The data not being in a "~/.app/app.dat" file, etc. The idea of it potentially just up and vanishing with a browser update. And, heck, just the complexity of dealing with the schema versioning in the native web model.
As well as the portability of data (say when you copy over to a new computer).
The idea of simple web distribution is compelling. Now you need nothing more than a github account, and off you go. No server, no nothing.
But I still feel (perhaps ignorantly) that the data situation is still on shaky ground. (Discounting the whole lack of something like SQLite, etc.)
And, the idea of bundling something like SQLite as a webassembly blob just makes me itch.
Just curious how that's worked out.
There's definitely a lot of fiddlyness when it comes to the web stack.
Having said that, indexeddb has been very reliable. There are some annoying limitations though. For example, if you want to create an index on a store using multiple keys, you can't specify which key should be ascending or descending.
If you add the PWA to your home page, i.e. install it, then the browser must not clear indexeddb on an update or something. And as far as I can tell, browsers do respect that.
My overall experience building PWA has been mixed. I think it really depends on what app you're making. For a note taking app, where the data can be backed up to the cloud encrypted and synced, I really think a PWA is the best option. But if you want tighter integration with the platform e.g. accessing the file system, then I would stay away from PWA.
Another issue is ios safari which is missing a lot of features for PWAs and I think unlikely to support them in the future. E.g. background sync, offline push notifications, etc.
Are you considering PWA for any particular app?
I'm just writing little stuff in Java, and use that as my x-platform environment, but even it has its challenges.
So, I was considering a PWA. I don't need tight local integration per se. I do have a conceptual problem not being able to save a file or read a file, even if it's just chosen by the user. I THINK I can do that. I THINK you can pop a file chooser to read a file, but not necessarily just open any file you want. If you can read/write a file as directed by a user, that would be fine. I can see the rest being stored in the IndexDB. And that gives the option of an export/backup if it becomes necessary for peace of mind.
Then there's the whole x-browser game, I'm not really looking at mobile, just desktop, but I'd like to support FF/Chrome/Safari/Edge.
It's not rocket science stuff by any means. But I don't want a user to call up and talking about how their data suddenly vanished because they went to get the latest, and I did something innocent/ignorant (or not), or THEY did something (deleted their PWA icon, copied it, moved it, etc.), or the browser did something.
I have no experience with it, so for now, I'm not comfortable because I don't have my "hands" on the data.
If I can export it, then the comfort level goes way up. Inform the user to back up "just in case", especially early on when we're all still bumping in the dark.
To be clear, I don't want a server side, I have no interest in standing up a service component. I have less interest in maintaining a service component. I'd like the code to be local and "standalone" as practical.
But I also want something a touch more than "git clone app; cd app; ./build".
If you only care about desktop and don't want to maintain a server, I do suggest taking a look at webview [1]. It uses the platform's own web browser engine (WebKitGTK on linux, WebKit on mac, WebView2 on win) which you can use to run your app using the web stack. Essentially, it gives you something like electron.js but without the excessive bloat. And you get access to the file system and everything else on the desktop.
I believe there are other libraries/frameworks on github with a similar approach you might want to have a look at.
[1] https://github.com/webview/webview
1 reply →
You mention in the sample page that Safari on desktop "cannot install". But in recent versions of macos, it has "save to dock" in the "Share" menu.
This creates Unforget.app in ~/Applications which appears to work. I've also successfully used this with Tana in the past. Did you run into trouble with that or have you not tested it?
Oh yeah you're right. I'll update the instructions. Thanks!
1 reply →
I've been pondering similar regarding data persistence. One half-assed idea I have is simply to prompt the user to save the data regularly. A reminder with a button/link which triggers data serialisation from storage and then triggers the save dialogue. The serialisation/save could be triggered manually any time. A reciprocal import would also be needed of course.
Another similarity half-baked plan is for a PWA to regularly post (encrypted?) data to a service I'd host that simply forwards the data to the user's choice of cloud storage. It would require initial setup and probably regularly refreshing token, but should be workable.
The self-hosted app Actual Budget [1] sort of takes the second approach, where each client maintains their own SQLite database in local storage. However, instead of just hosting a single file database, the server maintains a list of migrations and is able to resolve conflicts between multiple clients. Clients push/pull the migrations when syncing to keep their database up-to-date. E2EE is also supported. The original creator, James Long, has done talks about the architecture if you want to learn more [2, 3].
[1] https://actualbudget.org/
[2] https://www.youtube.com/watch?v=2dh_gtndayY
[3] https://www.localfirst.fm/7
Ive got a PWA forth coming, i expect to make a electron or other companion app to make.it self hostable via webrtc.
> The idea of it potentially just up and vanishing with a browser update.
Anyone writing applications in the browser should be very cautious to note that many browsers will jettison some types of their website storage if disk space is low.
This is frustrating at the best of times, but if there's no server-side backup then frustration would be an understatement.
I am not sure if this caveat applies to localStorage but from experience it does to IndexedDB — seemingly even in Electron applications (??).
> from experience it does to IndexedDB — seemingly even in Electron applications (??).
oh fun! I haven't run into that. But if you're using electron or any platform's local webkit wrapper, then it makes a lot more sense to forgo the browser's storage and directly use sqlite + server-side backup if possible.
I think the safest offline storage is to use the browser file system (https://developer.mozilla.org/en-US/docs/Web/API/File_System...) so the data is out of the browser's hands.
It works pretty well with the only downside being the user needs to re-select the file/directory to use each time they use the app and there is no way to force the file picker to start in a specific directory other than the "well known directories".
Now, you only need to select the directory once, and the browser will retain permissions
https://developer.chrome.com/blog/persistent-permissions-for...
Thanks for the heads up!
!The data not being in a "~/.app/app.dat" file, etc.
It is possible to do this now in chromium based browsers. Refer https://developer.chrome.com/docs/capabilities/web-apis/file...
It seems it's using `better-sqlite3`[0] and saving the database to a file called "private/unforget.db"[1].
(Although, I don't have experience with SQLite, so that file might be as volatile as `localStorage`.)
[0] https://www.npmjs.com/package/better-sqlite3
[1] https://github.com/computing-den/unforget/blob/master/src/se...
Yes that's the database on the server for storing encrypted notes and syncing. My experience with SQLite has been very positive. It really is rock solid.
Also, wanted to mention that Unforget doesn't use browser's localStorage. LocalStorage has a low limit and the browser can nuke it when it wants. Instead we use IndexedDB which has much higher limits and persistent.
I’ve been using Omnivore (a “read it later” app), which I otherwise love, but even though it’s an iOS app, offline support is one of its biggest weaknesses.
As far as I know, it’s using some caching library at the HTTP level (as opposed to handling its own persistence at the application layer), and in falls apart every time I have a poor connection, remaining broken (even after fully going offline) until I’m back online.
This was my concern with building a PWA, which is why SidewaysData is a native desktop app. The downside of course is updates, but that's manageable.
If you have a.desktop app, zetup a webrtc connection and you can create a teathered.webapp in PWA.
SidewaysData needs to be able to work offline. It also needs access to serial/USB ports. Also, I prefer Java to TS etc anyway.