Comment by sambigeara
3 days ago
Hi everyone, I'm Sam. I started Pollen as an experiment last summer, got carried away, and have landed here.
It's a single Go binary. Install it on every machine you want in the cluster and they self-organise. Topology is derived deterministically from gossiped state, so workloads land where there's capacity, replicas migrate toward demand, and survivors rehost from failed nodes. The mesh is built on ed25519 identity with signed properties; any TCP or UDP service you pin gets mTLS. Connections punch direct between peers where possible, otherwise they relay through mutually accessible nodes.
I built it because I'm fascinated by local-first, convergent systems, and because I wanted to see if said systems could be applied to flip the traditional workload orchestration patterns on their head. I also _despise_ the operational complexity of modern systems and the thousands of bolted-on tools they demand. So I've attempted to make Pollen's ergonomics a primary concern (two-ish commands to a cluster, etc).
It serves busy, live, globally distributed clusters (per the demo), but it's very early days, so don't be surprised by any rough edges!
Very happy to answer anything in the thread!
Cheers.
Docs: https://docs.pln.sh
Fascinating project – do you think there is a use case where a cluster could host web assets or a full static site? I wonder if it could act like a personal CDN in that way...
Yes! I host pln.sh (and subdomains) as assets on my prod cluster. I have a couple of nodes hosted in EU/US, but do rely on a Cloudflare and a couple of A records to land traffic on them.
This is incredible.
We’re building an AWS analogue catalogue of services (Databases, Compute, Auth, etc.) for distributed systems.
Want a job do Pollen-like dev full time?
william.blankenship@webai.com
Either way, would be great to compare notes!
Thank you!
Interesting project.
In a potential modern cloud, having a globally named primitives (computer, store, messaging) can unlock very wider applications. Have you come across any such?
To clarify, are you asking if I’ve considered incorporating those concepts into the project?
If so, I have loose ideas around how I might introduce shared state, it’s an interesting problem that’ll require a lot of thought. Early days yet, though.
You have some workload demos which all definitely try out but could you paint us an example use-case of the technology?
What are the workloads in the runtime capable of?
OK bear with me on this, it'll probably be a idle thought-stream because I don't have a concrete answer right now.
My intention is for Pollen to become a "generic blob of computational capability" into which you idly `pln seed` a workload and do not have to worry about ANY aspects of managing locality, scale, redundancy etc. You seed a workload onto any node, and you call it from any (other?) node. If you want to add more computational power to the cluster, you fire up Pollen on another machine and `pln invite` -> `pln join`.
Every node also has it's own ed25519 cert. The root key pair (the "don't lose this or you're in trouble" key pair) is used to delegate admin certs to other nodes. I'm also working on a mechanism which allows you to bake any arbitrary properties into a cert (as it stands, these are lifted into the WASM guest code for, say, in-application authz purposes). I have more ideas about how this can be extended in the future.
The root authority can invalidate a participating peer's cert at any point, currently just via a `pln deny` command which is eagerly gossiped around the cluster so other nodes stop talking to the denied node, too. I think this offers some opportunities for some fairly novel applications. Perhaps, in the future, you'll provision a node with a certain level or capability or authority to run on some external infrastructure. It'll have all of the (allowed) capabilities of your cluster, but will act like it's local to the external system. Plus, you can revoke it's access or re-set it's capabilities at any point; `pln grant` eagerly applies across the cluster, too.
The workloads, at the moment, are just anything you can compile to WASM via the Extism PDK. Stateless, for now, but with a view to add shared state and persistence in the near future!
Sorry this was rambly, hopefully it offered something useful.
Splitting a big task (like anything ML-related) into a set of smaller ones and distribute them across the "fleet" of workers. Then reap the results, stitching it back into a single artifact at the end. This could be commercially viable. This could even become a p2p platform/market where some people basically buy computation while the others offer their hardware for temporary rent to earn a few bucks. You become the coordinator that just connects the demand with the supply and become rich from just commissions alone.
1 reply →
This is very interesting! I agree about the operational complexity of many systems, cough Kubernetes.
For most systems, state storage is the toughest problem, have you considered adding some form of storage layer over the top or would you recommend another solution that allows all the workloads to share state?
I agree with operational complexity of kubernetes. That is the reason why we started writing single binary rust based kubernetes compliant platform [1]. Think of it like a minimal kubernetes rewritten in rust. Basically, you can deploy with kubectl.
I like wasm idea here. I'm probably going to add wasm as runtime for this project.
[1] https://github.com/debarshibasak/superkube
That's an interesting project, don't give up!
P.s.: it's a shame that a rust lib for containers has a hardcoded path to sock file. That's weird.
This is a cool idea!
Oo good question. I'd prefer to keep external storage solutions as an exercise for the reader. I've touched on this in other comments, but I am looking to introduce state to the cluster internally, but for more sophisticated storage solutions, I'll probably avoid steering the project towards any one solution, at least for now.
[dead]
From someone who definitely doesn't fully understand what you made, this looks really cool!
I'm seeing some functionality that seems like it could replace some personal services I currently host via my tailscale network. Am I understanding this correctly? If so, do you have a feel for what the performance implications would be?
Thanks! I think the classic answer: "it depends" applies here. It currently only supports stateless workloads (for now, see below), so if you have nice, isolated, functional workloads, the WASM seeds could be a good fit!
My original intention was that the WASM seeds would be the primary workload entity in the system, because it fits nicely with the whole local-first, self-balancing ethos (and WASM modules are blobs that can be gossiped around readily as nodes claim them). That said, you can also register generic TCP/UDP servers on a host (`pln seed 8090 some_service`), which are callable from nodes and seeds (via `pln://service/`).
On statelessness, as I've alluded to elsewhere in the thread, I'm looking at how (convergent) state can be introduced into the system and exposed to seeds, so this would ultimately add another layer of capabilities to the WASM functionality.
On performance: again, it'll depend. I noted elsewhere in the thread the distributed implications of chaining multiple seeds in a call flow. You're not _just_ dealing with CPU, WASM boundary-hopping or even traditional IO ceilings, there's also a component of synchronising gates in proxying nodes (as seed A calling seed B needs to reserve the WASM instance locally until seed B responds, which has knock on memory implications, etc). Its a fun problem, no doubt the story will get better in the coming months. For a local, simple invocation, depending on the nature of your workload, I would expect the standard WASM overheads to apply (there's only a thin layer between the API and the underlying Wazero runtime). For a lot of applications, this should be neglible.
Also a note on placement, if you run, say, 10 nodes globally, `pln seed` will, by default, only place two replicas into the cluster. However, as load is introduced, the seeds will propogate towards it, so a node that's acting as an ingress for `pln call` will generally claim the workload to benefit locality/latency.
Feel free to message here or privately if you wanted to discuss your actual use-case, would be keen to understand how people might try to use it!
this is a great direction - self organizing service meshes that don't require a infinite tower of manually configured turtles to rest upon. state management is really going to be an intersting and I encourage you to post back here with your thoughts. strong consistency kind of gets you back to turtle-land pretty quickly, and free-for-all eventual turns out not be a good foundation. the sane middle ground seems to be monotonic eventual consistency, which I think is what CRDTs get you.
I wish more projects were conceived this way, instead of assuming that there is a kuberenetes cluster another database or two, and some message queues to lash it all together.
Thank you! I suspect there'll be a fair few dragons to uncover (memory constrained nodes and partial views, disk storage, startup/shutdown patterns, etc etc), if it's worthy of a write-up then I shall certainly post it here.
[dead]