Comment by ltbarcly3
6 hours ago
Watch as I don't use protobuf because it is horrible.
....
Tada!
If you patch clients to google services in Python to use json instead of grpc they get faster and more reliable. A lot faster. Benchmark it!
def get_json_client() -> CloudLoggingQueryClient:
"""Client for log queries (JSON transport, avoids gRPC overhead)."""
client = google.cloud.logging.Client.from_service_account_info(...)
client._use_grpc = False
return client
For me that is how I know something like protobuf is good. It is a nuisance to manage and distribute the definitions, adds a build step even to languages with no build step normally, is slower than almost every alternative, and artificially restricts you from doing lots of common things. It's so good!
And look at the code quality of the implementation! It's like a team of interns wrote it while drunk. It is a complete spaghetti mess, but has tons of super convoluted micro optimizations that are slower than just doing the most obvious thing, but make the implementation confusing and indirect. It's trash code.
What are good alternatives when you need a common "single source of truth" schema shared between multiple languages? We use protobuf between c# and Python.
Json schema?
I quite like the look of Typespec though I haven't used it much.
I always thought Thrift was waaay better than any of the alternatives, but it always had terrible documentation and I think it died mainly because of that.
The goal is not to have a single source of truth schema. That is a means to some other goal, and it's not even a good means.
If you never change the schema then you don't have to worry about it, get things working and never look back.
If you do change your schema from time to time, you need testing between the two systems. If you have good tests again a single source of truth is fully redundant, both systems are talking just fine. If you don't have tests things can and will break all the time even using protobuf.
> The goal is not to have a single source of truth schema. That is a means to some other goal, and it's not even a good means.
It’s about data transmission. Being able to encode and decode in a type safe manner between different languages (and so, different platforms) is a goal that makes a lot of sense.
> If you do change your schema from time to time, you need testing between the two systems
Or you could just use a defined format that doesn’t require testing. I rarely use protobuf but I can see why people do. The guaranteed backwards compatibility is huge for people who can’t just publish a new web frontend at the drop of a hat.
If you understand how to evolve protobuf schema definitions, then you don’t really need testing. You instinctively know how the parser works when it is parsing data with a different schema from what it expects. And that’s a powerful thing. If your things break even when using protobuf then you don’t grok protobuf.
It’s probably not an exaggeration to say that being able to avoid tests between different systems who have different versions of the schema is a core goal of protobuf. Why? These two different systems are probably owned by different teams, and introducing explicit tests between different versions of them increases coupling between them.
Both sibling comments say one type of assurance makes the other irrelevant, but I would wager they cover different territory.
Is that a recent-ish improvement? I feel like HTTP/2 would be roughly the same performance for JSON and protobuf, so maybe this is HTTP/2 vs HTTP/3?
I think the overhead is protobuf itself but I can't check.
Comparing a wrapped C++ gRPC backed stack with an httpx/requests backed one is like comparing apples to elephants.
Protobuf simply encode things way more efficient that JSON can define a single object. You're quite frankly spewing bullshit in this whole thread.
3 replies →
Is this because load is lighter on their JSON endpoints that their gRPC ones?
[flagged]
so how do you save data over the cable when it's needed?
Eventually it’s all bytes. Where do you want to save them?
zstd?? Obviously?
JSON. /s