Comment by newswangerd
6 months ago
I've been doing something similar to this, except with go. In my case I have a flutter frontend and a go backend that's built using go mobile. Instead of trying to figure out how to make all of my go functions use data types that are supported by the various native frameworks, I've opted to use protobuf objects for every type that is shared between the frontend and backend. This way I can expose a single go function via the flutter FFI that takes in a binary array and then converts it to a protobuf object. This gives me a nice separation of concerns between my business logic and frontend while also providing easy to use objects for the front and backend.
Not sure that I'd recommend this approach to everyone. Protobuf code generation can be finicky to set up, but I'm doing it so that I can access go's rich array of libraries in my app.
This is what I did for BeatScratch! https://beatscratch.io
My music model is all Protobuf messages, which go from Dart/Flutter land to Kotlin/C/Swift/JS audio backends on target platforms. I also use Protobuf for saving and sharing. It’s been incredibly resilient and performant.
This is very cool! I'm going to take a look at your code.
I've been playing with the idea of creating a "protobuf db" library that would allow you define schemas in protobuf and then query them with something akin to an ORM. It wouldn't make any sense for large databases, but for embedded applications that only need to store a few MB of data, it would be perfect.
Have fun! Note that it is GPL-licensed. Also, note that the "main" branch is for a very old Dart/Flutter version (but it does correspond to what's in the App Store and on the site today).
I've been working on a separate branch, which finally builds (there were 1100+ errors), but I'm still working through iOS/macOS build things for it before merging it to main. (I've sadly had to abandon the Android build, because Google Play was a comparative pain, and FluidSynth upstream kept breaking the Android build I set up for them. But I'm reviving the project for iOS, macOS, and web at least.)
Here's the branch: https://github.com/JonLatane/BeatFlutter/tree/update-to-late...
We do something similar for the UI for our audio hardware product. AES70 control messages are sent over Flutter platform channels to a Swift backend. The glue is open source - GitHub.com/PADL/FlutterSwift.
I don't understand what you mean by frontend and backend when you mention ffi. Is this backend in a remote server or just on the same app?
I used proto buf with rust, I had a rust client that spoke to my flutter frontend via dbus. The rust client connected to my remote server via a web socket and all messages were wrapped in protobuf and sent as binary. Made everything a lot more concrete... But it basically forced me to build my own much shittier version of gRPC. Since, if the wan for your network was every killed the client was notified too late and you'd end up with missing messages if the network buffer got filled. We added a message id and acknowledgement process with sqlite backing up each message.
I still have nightmares about why I built that.
I guess a better term for it would be frontend and business logic. On iOS and Android the business logic (backend) is run using go mobile bindings and are imported directly into the native framework. For Windows, Mac and Linux, it runs as a gRPC daemon in the background. You could use C bindings for PC, but those seemed like a hassle, and I need a daemon anyway.
Have you considered just using gRPC in this case? You gain 100% language separation (no FFI) and remote client/server at the cost of a little more call overhead.
Not OP but in same situation. Not every platform can run gRPC over localhost easily or without extra privileges.
I used to use protobuf but now I just use JSON, over stdin/stdout on desktop. It’s honestly quite good.
Which platforms? My product runs gRPC client/server on macOS, Linux and Windows. No issues with privileges. Or are you trying to run it on port 443? Yeah, don't do that, run it on 8443 or whatever instead.
3 replies →
Why not ConnectRPC? It's basically gRPC but without all the strange requirements for exotic HTTP features.
2 replies →
Most of the gRPC implementations force buffering of the whole response for large unary responses. They are not really written by people who care about performance. It’s dumb because the protobuf binary marshaled format is perfectly designed for server-side incremental marshaling.
Performance is relative. gRPC is plenty fast enough for my use case, and for that matter, almost all client/server use cases that work across the Internet. If a Javascript web client against a REST backend is fast enough latency-wise, then a local gRPC connection on a single PC is gonna feel like greased lightning. Of course, there will be a few scenarios where tight coupling of client/server are required for good enough performance, but they are few and far between.
Yeah! I'm using a gRPC daemon on PC and go mobile bindings on mobile.
Hah yea. I just did a deep dive into protobufs and RPC for an embedded application. Left learning a lot, and with a headache. Part of it was because this was using heapless, and I got errors until I configured the generator to use the right Vec sizes.
That's a perfectly fine approach, Protobuf strength is exactly these kind of use cases.