Comment by 2ndorderthought
8 hours ago
Can anyone explain why it is a good idea to make a graphdb in typescript? This not a language flamewar question, more of an implementation details question.
Though typescript is pretty fast, and the language is flexible, we all know how demanding graph databases are. How hard they are to shard, etc. It seems like this could be a performance trap. Are there successful rbdms or nosql databases out there written in typescript?
Also why is everything about LLMs now? Can't we discuss technologies for their face value anymore. It's getting kind of old to me personally.
I needed it to be possible to run the graph in the browser and cloudflare workers, so TS was a natural fit here. It was built as an experiment into end to end type safety - nothing to do with LLMs, but it ended up being useful in the product I'm building. It's not designed for large data sets.
> It's not designed for large data sets.
How large is large, here? Tens of thousands of triples? Hundreds? Millions?
I'm working on a local-first browser extension for ActivityPub, and currently I am parsing the JSON-LD and storing the triples in specialized tables on pglite to be able to make fast queries on that data.
It would be amazing to ditch the whole thing and just deal with triples based on the expanded JSON-LD, but I wonder how the performance would be. While using the browser extension for a week, the store accumulated ~90k thousand JSON-lD documents, which would probably mean 5 times as many triples. Storage wise is okay (~300MB), but I think that a graph database would only be useful to manage "hot data", not a whole archive of user activity.
it depends on the backing store (which is pluggable). I would not want to exceed let's say 50Mb in a Y.js doc, but i've tested the in memory storage with graphs approaching 1Gb and it's been fine - like any graph it really depends on how you query it. Most of the docs I'm dealing with in production are less than 10Mb, so this is fine for my use cases, but... buyer beware!
Makes sense thanks for explaining the use case. The LLM question was only because of the comments at the time of the post.
The query syntax looks nice by the way.
thanks, it was as close to Gremlin[0] as I could get without losing type safety (Gremlin is untyped)
[0] https://tinkerpop.apache.org/
Gleam might be a great choice perhaps. Compiles to typescript and Erlang/BEAM.
You can use webassembly in both places https://developers.cloudflare.com/workers/runtime-apis/webas...
>Also why is everything about LLMs now? Can't we discuss technologies for their face value anymore. It's getting kind of old to me personally.
Fully agree with you, LLM's everywhere is getting churlsome, and trite. Sure, one can generate the code, but can one talk about the code?
Can one move, easily, beyond "but why should I?" into "I should because ..", whenever it comes to some realm, one is supposdely to have "conquered" with AI/ML/etc.?
Sure, we can all write great specs, sit back and watch the prompts rolling in off the horizon...
But seriously: the human reasoning of it is the most important part!
However everyone is busy building things secretly that they don't want anyone to know about with AI/ML (except the operators of course, duh, lol, kthxbai...) because, after all, then anyone else can do it .. and in this secrecy, human literature is transforming - imho - in non-positive ways, for the future.
Kids need to read books, and quick!
>old to me personally
I kind of regret seeing it in one of my favourite realms, retro- computing .. but .. what can we do, it is here and now and the kids are using it even if some don't.
I very much concur with you on the desire to continue being able to discuss technologies at face value, personally. We have to be reviewing the code; the point at which we don't review AI's output, is where we become subjects of it.
This is, probably, like the very early days of porno, or commercial tobacco, or indeed industrialized combustion, wherein it kind of took society a few big leaps and dark tumbles before the tech sort of stabilized. I'm not sure we'll get to the "Toyota" stages of AI, or indeed we're just going to blast right past into having AI control literally every device under the sun, whether we like it or not (and/or, have an impact on the technologically-obsolete curve, -/+'vely...)
Ain't no easily-accessible AI-hookery Typescript in 8-bit land .. I will have my retro- computing without that filthy stinking AI/ML please, mmkay, thanks!!!
I kinda feel ya.
I wrote my own in-Memory Graph (i'd rather call it storage than a DB) some years ago in golang, even there i was wondering if golang actually is the optimal technology for something like a database especially due to the garbage collection/stop the world/etc. Its just there are certain levels of optimization i will never be able to properly reach (lets ignore possible hacks). Looking at a solution in typescript, no matter how "nice" it looks, this just doesnt seem to be the correct "tool/technology" for the target.
And inb4, there are use cases for everything, and same as i wouldn't write a website in C, i also wouldn't write a database in javascript/typescript.
I just would argue this is the wrong pick.
@llms : im not even getting into this because if you dont wanne read "llm" you basically can't read 99% of news nowadays. ¯\_(ツ)_/¯
edit: im a big fan of graph databases so im happy about every public attention they get ^