Comment by xg15
3 years ago
It's an interesting idea, but I wonder how well the promise will hold up that this is less complex than using a message broker.
Yes, the network topology and protocol are certainly less complex, but there are now additional strong requirements how an endpoint has to store and manage existing events. (See the bits about strict time ordering, compaction, aggregation, etc).
A lot of this is effectively what a message broker is doing in a "traditional" system to guarantee consistency. Those tasks aren't gone, they are just pushed to the endpoints now.
A few issues with message brokers, esp. in the system-to-system integration:
- Security: In B2B scenarios or public APIs would you open your broker to the WWW? HTTP has a solid infrastructure, including firewalls, ddos defence, API gateways, certificate management, ... - Organisational dependencies: Some team needs to maintain the broker (team 1, team 2, or a third platform team). You have a dependency to this team, if you need a new topic, user, ... Who is on call when something goes wrong? - Technology ingestion: A message broker ingests technology into the system. You need compatible client libraries, handle version upgrades, resilience concepts, learn troubleshooting...
> Security: In B2B scenarios or public APIs would you open your broker to the WWW? HTTP has a solid infrastructure, including firewalls, ddos defence, API gateways, certificate management, ...
That's a valid point. I think it's a pity we don't have an equivalent standard for asynchronous messaging with the same support as HTTP. However, there are lots of options for presenting an asynchronous public API that would use your message broker behind the scenes, without fully exposing it: Websockets, SSE, web hooks, etc...
> Organisational dependencies: Some team needs to maintain the broker (team 1, team 2, or a third platform team). You have a dependency to this team, if you need a new topic, user,
True, but don't you have that anyway? How is this different from requesting a new database, service route, service definition, etc?
> Technology ingestion: A message broker ingests technology into the system. You need compatible client libraries, handle version upgrades, resilience concepts, learn troubleshooting...
How simple or complex this is depends on the concrete broker at hand. There are some protocols, e.g. STOMP that are simple enough that you could write your own client. And as I wrote in the parent: HTTP feeds are a technology as well. You'll have to think about troubleshooting and resiliency there as well.
My thoughts exactly. While this is more human readable on the wire, a message broker delivering the feed would provide many different other features that might be useful, such as transactions, load-balancing, guaranteed delivery and per-endpoint state to simplify the individual application instances.
For those not aware of what message brokers are, there are many to choose from such as: Mosquito, RabbitMQ, ActiveMQ, Solace... If delivery over HTTP is a requirement, many of these brokers support delivery over websockets or (in the case of Solace) also support long polling.