Comment by debadyutirc
7 months ago
This is a question we asked 6 years ago.
What if we wrote it in Rust. And leveraged and WASM.
We have been at it for the past 6 years. https://github.com/infinyon/fluvio
For the past 2 years we have also been building Flink using Rust and WASM. https://github.com/infinyon/stateful-dataflow-examples/
Fluvio looks awesome!
Any chance you’re going to be reviving support for the Kafka wire protocol?
https://github.com/infinyon/fluvio/issues/4259
We have had folks over the years asking us about the Kafka wire compatibility. We had a project 3 years ago which we archived. I think we have a case for reviving it in the near future.
Interesting!
How would you say your project compares to Arroyo?
Fluvio is streaming transport. And we built Stateful DataFlow on top of that for Stream Processing.
Arroyo is SQL first stream processing. Fluvio is streaming transport which can send data to Arroyo and there is an integration.
Stateful DataFlow and Arroyo are similar in the stream processing pattern and the use of Apache Arrow.
The interfaces are different. Fluvio and Stateful DataFlow support for SQL is the same dialect as columnar SQL supported by Polars. The Fluvio and Stateful DataFlow paradigm is more intricate more expressive and the platform is broader and deeper.