Comment by zh217
3 years ago
I actually grew really frustrated with existing solutions for organising notes:
- Almost all solutions that attempt to take seriously the concept of knowledge graphs are Software-as-a-Service. Ouch. They say you should “build a second brain”. I certainly don’t fancy the fate of my second brain (whatever that may be) to depend on the commercial success of some company.
- All existing editors, even the web-based ones, have some hangover from file-based content management. Even with ability to cross-link to nodes inside a tree structure within files, the rigidity of file boundaries is still over-constraining, especially when one is trying to grapple with unfamiliar concepts during which the structure and links are constantly shifting.
- The maths-authoring abilities (i.e. facilities to edit LaTeX formulae) at best look like an afterthought. Writing anything more complicated than Maxwell’s equations is painful.
Out of desperation I decided to roll my own. I set out the following desiderata:
- It should take the concept of knowledge graph as far as possible. The foremost thing is to do away with files, and promote vertices (nodes) and links (relationships) to first-class status.
- While upholding the previous principle, it should make editing as natural as possible, preferably confirming to existing text-editing habits. The experience should definitely not be like creating nodes and linking them together inside graph databases: slow and confusing.
- It must work without an Internet connection. Or better: it should not make any attempt to connect to the Internet at all.
- It must not be too heavy-weight, and data should be easy to back up. This precludes any JVM-related stuff, docker images, or spinning up Postgres/Neo4j/Datomic/what-have-you just to write some notes on a local machine.
- It should cater for the needs of heavy maths editing.
Now I have been using my own app, powered by sqlite for storage engine and ProseMirror for frontend, for quite some time, and I am really satisfied (hard not to be satisfied by your own work though). It is becoming increasingly polished (full-text search anywhere, very fancy query, loops in the graph, etc.) but I hesitate to open-source it, because as far as I know maintaining open-source projects can be really time-consuming, especially when people actually start using them.
I'm working on a meta-solution: https://gateway.pinata.cloud/ipfs/QmeVYAP75GAvY8Q8iSfMoWMGgT...
The idea is to center everything around a data-interchange format and then build a ton of tools. I'm sorry the presentation is still very disorderly and that paper is obsolete and so is my website rishi.is and the original declaration of intent at datalisp.is
However! things are moving rather fast lately and I have more or less completely figured out some of the things that have been blocking me so far and I hope to have working prototypes of the most basic tools by the end of the year.
Im thinking about making my own solution too using sqlite!
how do you handle graph queries and looping/traversal?
I am also wondering how i will sync across devices (in my case im thinking of a browser based ui with absurd-sql), probably will just start with the new filesystem api and syncthing or similar alternatives.
> Im thinking about making my own solution too using sqlite!
Me too. I see no problem in querying it and manipulating data. I don't really care about syncing. What I consider the most interesting and problematic are rendering the actual graph and the data, especially an anesthetic and interactive way. Because what I do NOT want to do is another browser/electron-based tool. I am only interested in desktop-native development but I have never seen components relevant to this subject. Apparently they are yet to be developed.
Sqlite supports recursive queries. It is clumsy and not too easy to write, but it works.
For syncing, what I did was to implement a simple automatic backup logic. Then if I want to move to a new device, I just copy the backup I want, which is just a sqlite db file, and restore it. It is not ideal if you are moving from device to device constantly, but it has the benefit that you can compare backups to see what exactly you have changed (I even wrote a tool to build graph visualizations from the output of sqldiff).
> I hesitate to open-source it, because as far as I know maintaining open-source projects can be really time-consuming
Open sourcing something and maintaining an open source project are two things that should not be conflated. You can have one without the other.
Do you have a contact method or a github?