Comment by curl-up
3 years ago
I am currently developing exactly this - a notetaking system similar to Roam/Obsidian/etc. which, instead of explicit links automatically searches for related notes. My approach was to utilize some sort of automatic semantic-aware keyword extraction and try to link other notes not only to the current note as a whole, but to its specific terms/sections. For example, if note is containing a recipe, ingredients would have other recepies which use them linked.
I was not aware that such a system already exists. Can you point me to some references? Is it a private system you developed? Are you willing to share more details? Thanks!
Instead of reinventing the wheel, did you consider writing an extension for Obsidian instead? That would allow you to focus on the extraction and aggregation, instead of having to invest a lot of time into also getting an usable text editor (which is a non-trivial task)?
I think I would skip trying to do any entity or term extraction initially. That's going well beyond basic linking. Stick with something like TF-IDF similarity or LDA topic modeling to keep things fast and straightforward until you have the UI/UX worked out, which is always more work than it looks like. (The reason I used neural net embeddings was that the OA API had just added them, and it was a fun excuse to try that out.)