Go Concurrency Distilled

1 day ago (antonz.org)

The concurrency and threading in Go just feels like magic compared to every other language. I'm a goroutine addict and I refuse to be rehabilitated.

Just from observations over the years, I don't think there's any other language quite like this, in terms of how things can end up happening in any thread.

  • How about Erlang or Elixir using BEAM?

    Supposedly WhatsApp scaled to serving over 1 billion users with Erlang and BEAM.

    RabbitMQ, used by Reddit, uses Erlang and BEAM.

    Discord uses Elixer and BEAM.

    I just traveled down the BEAM rabbit hole. Fascinating story. The Ericsson Computer Science Laboratory cranked out some amazing products in the early 1990's.

    Their goal was five nines of reliability for Ericsson telephone switches.

    According to Joe Armstrong (an interesting fellow from Ericsson), the AXD301 ATM switch achieved nine nines over a nine-month period using Erlang and BEAM in 2002. That calculates out to 24 milliseconds of downtime.

    • BEAM+OTP is a masterclass in using concurrency to achieve fault tolerance. But Go achieves its "magic" by feeling like the lingua francas of programming, C and C++. Go doesn't make the developer learn too many new concepts. The runtime is self enclosed in the final binary. This commitment to the familiar programming patterns also means it allows for concurrency anti-patterns like shared memory which for Erlang+OTP's design principles is verboten.

      16 replies →

    • I looked at BEAM about a year or so ago, similar conversation here. I don't think BEAM is the same when you start looking at what part of code is executing in which thread. There's tradeoffs depending on what you're solving for, like Go makes it really simple to distribute your work across threads concurrently, but when you start looking at integrating with stuff, you run into having to do tricks to do things with unshare (ref: docker/podman/containers...) and you haven't been able to integrate into libnss since they started using some "unused linux signal" for concurrency controls (PAM used that signal).

      4 replies →

    • Yes! BEAM and OTP is amazing. Concurrency is one aspect and Go has great concurrency primitives, but what about supervision, and recovery and failure modes? Often they’re left to the developer as per Go’s philosophy which I think makes sense. OTP offers a lot of solutions to this.

      I think Go and the BEAM family languages are both great.

    • Yes. Once you know Erlang/Elixir and BEAM you realize it is at least a local optimum in languages/VMs.

      Truly something else if error handling is built into the language as a default case, not an … exception.

    • If you've written a few GenServers, I'm not sure one would describe it as "magic" in quite the same way. I wouldn't anyway.

    • > the AXD301 ATM switch achieved nine nines over a nine-month period using Erlang and BEAM in 2002. That calculates out to 24 milliseconds of downtime.

      This is misleading. I had an old Dell computer in my garage hosting a php app that hit that level of uptime as well over a 9 month period. It was 100% so actually better.

      Those uptime numbers only hold water when spread over many thousands to millions of users where you’re at large enough scale that you’re actually dealing with a meaningful volume of hardware failures.

      1 reply →

  • I was amazed how well are goroutines integrated into the language when I saw the first videos from Rob Pike. Then I actually started using Go for concurrent code, and noticed one thing, it's extremely easy to leak goroutines. There is no proper way to cancel them, they need to cooperate via select/context. Go developers eventually learn hacks to deal with it, but the simple go+chan style of programming style you see in tutorials is usually not safe. I still consider Go a remarkable piece of software. The runtime really doesn't have any seriously bad edge cases, it just works. But as a developer, I now prefer a slightly more explicit approach to concurrency. I've spent the last year developing an async runtime for Zig and I'm now more comfortable writing concurrent code in Zig than I was every using Go. I have more options for how to handle closed channels, I can cancel any operation, etc.

    • One reason it's easy to leak goroutine is that channel producer blocks waiting on consumer, once channel consumer exits the producer goroutine leaks. Go doesn't allow consumer to close the channel.

      In Rust when receivers all drop, the producer will error instead of blocking, so Rust is better in this aspect.

    • > There is no proper way to cancel them, they need to cooperate via select/context

      Isn’t this also true of threads? I know you can usually cancel them from a thread handle, but that kills the thread ~immediately without cleaning anything up, right? Presumably you pretty much always want cooperative cancellation?

      2 replies →

    • Yup. A popular way to get proper cancellation is to build exceptions into the language and specifically async exceptions so one goroutine can throw an exception into another goroutine. And Go does not have exceptions. Doing so would require all regular Go code to be exception safe, and really requires some form of try/finally or RAII but not defer. Anyways exceptions are quite far from the Go creators’ vision of the language.

  • We're heavy user of Go at work. Go also makes it way to easy to write bad concurrent code and hard to write good one.

    Stick to err/wait group and go routines and it's OK. Any PR with a channel or mutex I'll assume the author made a mistake.

  • >> in terms of how things can end up happening in any thread

    Doesn't that describe pretty much any green thread style concurrency implementation.

    • No. Preemptive scheduling plus M:N mapping combination that Go has is not common in other major implementations.

      Other languages and their implementations of green threads usually have cooperative scheduling or M:1 mapping

      22 replies →

  • Kotlin's coroutines basically had the same API as Go's. But also the ability to confine some coroutines' execution into certain threads (e.g. UI main thread).

    Then they added structured concurrency, which roughly solves the same problem as Go's context, arguably more elegantly.

  • I used to feel the same way, then I started writing Rust. I got tasks (goroutines) and channels which are largely the same as Go - except I never need to worry about race conditions or nil pointers and it's nearly impossible for LLMs to generate broken code (bad code, yes, broken code no).

    I have tried but I honestly can't go back to Go now, it's so much harder

    • This is an experience I personally had as well. It's a saving grace when working with junior developers because you know they won't end up writing parallelism-related heisenbugs.

    • I mean, that's one area where the story is not as nice in rust. Afaik the core abstraction is a bit leaky to be usable with tokio and other implementations as well.

  • Julia has a very good threading story. Task based, M:N, a lot of schedulers, structured concurrency, distributed. Sanest atomics I’ve seen. All in the stdlib.

  • I learned Haskell before that, and frankly the concurrency in Go feels similar, but is a definite downgrade due to the lack of STM.

    You can implement channels and select using STM, so these don’t have to be in the standard library. And the contentious design choices like what happens when you close the channel twice can be your choice! And going from STM to managing mutexes is a definite downgrade in abstraction power.

    The concurrency design in Haskell feels like true magic.

    • The concurrency design in Haskell is cool, though I gotta admit that I don't find it much fun to write.

      It's not because the language is "hard". I remember when I first learned Haskell a million years ago I thought it was the coolest thing ever because I had never seen anyone work at that abstract of a level before, especially in a compiled language. I got to understand the theory well enough and I know how to write a program with it, but the entire language kind of feels slapped together to me. Every time I've written anything in Haskell, I feel like I have to do a million compiler extensions, or rely on third party libraries' liberal use of Template Haskell (e.g. Lens) to make the language feel anywhere near "modern".

      Yes yes yes, I know this is a complaint about GHC, not "Haskell", but given that GHC is basically the only Haskell compiler that gets serious use I don't think it's weird to conflate the compiler and the language.

      5 replies →

  • It appears to me as if Golang has implemented part of the actor model. As I recall, it was neither set up to transfer free-form messages between the actors nor for the actors to persist beyond the given task.

    Erlang has had both for over 20 years; it has also had green threads for equally as long—something I don't know if Golang has. I'm sure Golang cannot split itself to run on multiple machines with its actor model. Erlang can.

    • Go pulled more from the ideas of Hoare's Communicating Sequential Processes (CSP) than it did from actors. In particular, communication is synchronous (excepting the use of buffered channels, though even there if the buffer is full the sender blocks) and it uses channels, while the go routines themselves are anonymous and cannot be directly communicated with (to the point you can't even get a handle for them).

      This is in contrast to, say, Erlang which is closer to actors than CSP, with its named processes (actors) and their mailboxes and no channels (though you can use a process as a channel).

  • I haven't found a Go concurrency thing yet that hasn't long-existed in Haskell before.

    I also find Haskell's concurrency in practice much easier to reason about than Go's, let me do a pitch:

    In Haskell you can just fork a thread and block till it's done. Threaded, "async" logic just looks like blocking serial code (but isn't blocking). I feel like in typical channel-based Go code I have to jump and scroll a lot in the code because of all the message-passing instead of block-scoped "blocking-style" variable use, and that this makes it hard to conclude whether the whole thing terminates or deadlocks.

    In Haskell, channels are considered low-level concurrency primitives you should only use when you have no clean high-level primitives for it. This is because they are not "structured" concurrency: When you send something into a channel, it is gone out of your scope, and you now need to track in your brain where it is, and who should consume that thing in the right way ("message-passing").

    For example, in Stolon, a high-availability Postgres orchestrator written in Go (https://github.com/sorintlab/stolon), I found the logic for failover with multiple channels and various timeouts very difficult to reason about when investigating failover bugs. I'm pretty sure that would read much easier in Haskell (see below how).

    In Haskell, you can start 2, or N, things in parallel, and easily wait till they are done. You can invoke parallel `map` easily.

        results <- mapConcurrently f mylist
    

    If f throws on any element, the whole map throws, and other threads get cancelled automatically as expected.

    You can get bounded, steaming parallelism, easily.

    You can set time limits to function calls writing

        timeout 1000 (myIoFunction ...)
    

    You can cancel any thread or computation, at any time. The same timeout function can cancel blocking IO operations, such as reading from the terminal or sockets, without having pass around `Context` objects like in Go (which, if you forget it, just makes things hang or deadlock).

    You wrap the 2 words "timeout 1000" around your function and done.

    Concurrency _composes_ in Haskell. You can write

        res :: Maybe (Maybe a) < timeout a (timeout b (myIoFunction ...))
    

    and the returned type tells you cleanly at which level the cancellation occured (no mixing into the same `error` type.

    You can build trees of parallel operations that live and die together.

    And there are no data races (because mutability is a very explicit thing), and I'm not even mentioning STM here (which allows you to do database-style transactions across variables) because that's already pointed out in another post.

    As a composed example, in Haskell you can write:

        timeout 1000 (race (downloadUrl ...) (forever (putStrLn "Still loading ...")))
    

    and that will do exactly what you think it should, with correct Ctrl+C cancellability, and good developer ergonomics.

    If you enjoy concurrency, give Haskell a shot!

  • any downsides to it in your experience?

    • Managing channels and making sure they are closed just once is quite messy compared to other languages. The Go channel axioms[1] don't make much sense: why does closing a channel multiple times panic, but reading from a closed channel returns a zero value?

      Kotlin gets this right. On send/receive, you can use trySend or tryReceive if you want to avoid exceptions. Considering Kotlin also has coroutines and structured concurrency, concurrency in Kotlin feels more ergonomic to me than Go. At least if you want to get concurrent code with least amount of bugs and not just least amount of extra keywords.

      [1] https://dave.cheney.net/2014/03/19/channel-axioms

      10 replies →

    • To the concurrency/threading; no? (there might be an "it depends situation somewhere idk about)

      But Go itself comes with it's own runtime built into the final binary. It doesn't work well in some use-cases, I mentioned some of the draw backs in a couple other comments if you want to dig those up.

      Also I saw some of the other comments. Channels are ultimately just used for message passing and aren't that complicated. You also can use mutexes or some other locking pattern. There's some primitive atomic structures available that solve some use-cases preventing you from even have to having to really deal with working between goroutines.

    • Others have mentioned the main issues, but to add; you often end up writing “ugly” code to do basic concurrency operations. Often setting up channels or workgroups then a `go func {}(…); wg.Wait();` just feels wrong and makes you thing “I must be doing something wrong, there must a better way, but that’s IS the way. It’s just go syntax quirks at the end of the day, and makes you appreciate go’s simplicity over high abstractions.

      In my experience the main hurdle was getting developers on the team onboard with go’s way. It felt like swimming upstream for my 6 year stint in go. I was in a very Java heavy “enterprise” but we were writing a kubernetes operator and I pushed to use golang because (a) I liked it, and (b) it was 2019 and the entire kubernetes ecosystem was primarily go.

      To me golang was very simple and I drank Rob Pike’s and Google’s narrative of how easy it’s to get a competent “compute science major in college”-person to pick up go. What I experienced was a form of “you can’t teach an old dog new tricks”. Lazy (and I hate to use this word) developers who gotten so used to frameworks and IDEs doing all the heavy lifting for them in Java or C# had 0 appetite forgetting all the questionable patterns they learned over the years and adopt Go’s simplicity. It was very frustrating at time, yet gave me a good eye for the actual skilled talent in the organization vs the average enterprise developer persona.

      2 replies →

    • Go and Julia are fun languages.

      In production, Go has proven solid for several years. It is best when used with the native code people ported.

      There are only two issues I encountered:

      1. getting the legacy ancient C source meta-circular Go compiler working to port the Go boot-strap compiler upgrade chain is a kick in the pants. However, once it is on a architecture it has proven rather resilient.

      2. memory limited systems can develop reliability issues, as Go programs will often ungracefully throw hard to diagnose unrelated errors during each crash. A good metric is 3:1 of your average load as a safety margin (if you see 2GiB in average RAM use, make sure to over-provision the host with 8GiB RAM etc.)

      Other than the above short list of edge cases, if you join a pure Go project it is usually pretty reliable. Most community folks interested in the language seem fairly competent at building stuff that is fun. =3

      8 replies →

  • Java is doing it better (virtual threads + structured concurrency + immutable records and immutable value types).

    Not to mention golang is not memory safe: https://www.ralfj.de/blog/2025/07/24/memory-safety.html

    • Java's virtual threads are not preemptive. Yes there is a paper where they call them preemptive but they define the term differently to claim it. Code stuck in a tight loop is not preempted.

      2 replies →

    • I don't think you understand the threading concurrency topic. Also memory safety is so far off base here, where's that coming from? Java does some stuff okay, but do you really want to defend the horrid JVM problems? Also why can't I have my memory back when it's not in use in tightly packed systems?

      It's not great for everything and neither is Go. You can find a bit more context on that in some of the other threads.

      8 replies →

  • Not really, it is not knowing enough programming languages and computing history.

Ive been writing Go for over a decade and I still feel like I never quite "got" channels. Every time I use them I need to go consult the manual, and none of the patterns feel obvious which is weird considering the rest of the language feels very obvious.

Too many years of Java and managing Threads and Runnables probably rotted my brain.

  • Try writing CSP style code and see how channels slot in. That is the design they had in mind.

  • Channels are honestly one of the most over-used things in Go. I've been writing Go professionally since 2015 and I honestly rarely use them. Programmers new to Go love to shovel them in everywhere because "why use Go if you're NOT going to use channels?" and I have to say sorry, no - write it serially, then determine if it breaches your SLOs, THEN determine if concurrency fixes it.

    • Using Go since 1.0, agree wholeheartedly. Newcomers read the docs and start throwing channels everywhere because why not.

      I always ask/tell people to write without channels, and only add them when you have justification for doing so. That leads to much more sane code.

      One pattern I see often because random blogs mention it is starting X long lived goroutines, then passing them data via channels, then receiving responses via channels, then handling. In my experience, it's 100x less error prone to just use a semaphore to start a goroutine per data, and have them do their own handling. No channels involved.

    • Yup. Go maturity is realising how little you need to use channels and Goroutines. You probably just need a setup in one place, like in front of incoming requests ... which using net/http already does for you.

      Spamming them all over the place is a red flag imo

    • Very interesting feedback. I'm a Go newbie and the goroutine/channel duality sounds delightful from where I stand, but once again I have no professional experience with Go yet, only sample programs to get used to the language.

      One question though: your advice is to write things serially first before moving to concurrency, which for me is general programming common sense, but would you argue that once you start writing concurrent code then channels are not well suited compared to "good old" sync primitives (mutexes, etc.)?

      2 replies →

    • I’d say it’s important to understand how they work but I also rarely find myself reaching for channels. I see more usage of wait groups and mutexes, but even then you can build abstractions around these in a way that can be reused without having to touch them again.

      1 reply →

    • Concurrency has nothing to do with performance and everything to do with your domain. If what you're modeling is concurrent, your code should accordingly be concurrent also.

  • It depends really on what you actually want to do. I tend to make a few helper funcs for different kinds of things I want to do. For example, a helper funcs to accept anonymous job funcs and collect output. Then you can compose programs out of those higher level blocks.

  • Arguably, Java's virtual (green) threads managed through structured concurrency and futures is a superior approach.

    • Arguably, being 10 years late to the party is pretty bad.

      Just how Go adding generics to the language didn't magically fix the billions lines of non-generic Go code, adding virtual threads to Java didn't update its entire ecosystem to take advantage of them.

      Meanwhile, the entire Go ecosystem from the beginning took advantage of goroutines, so all code you'll ever interact with will have excellent support for them.

      3 replies →

Ahh you got me. Finished the first chaper of the 'free online' Gist of Go book, then in the second chapter it turns out the first chapter was a freebie.

I used to do this, and do it well. Nowadays, I avoid it like the plague. Not just because of the advent of AI agents, but also. I usually try to condense the core business logic of the application into a tight sequencer, and then every type of slower workload has a manager for it, with queue, dispatching. All logic remains linear, easy to review and follow. Concurrency is basically just handled at the level of kicking off some work, and then funneling the result back into the sequencer. Easier to test, highly scalable concurrency.

This is fine, but it's too bad it did not mention the cardinal rule of goroutines on prod, which is "before starting a goroutine make damn sure you know how it will stop".

Goroutine leaks in prod are no laughing matter. They are difficult to debug without killing the process, and that's only useful if you are sure you're going to get stderr to get the full traces of all goroutines.

honestly the hard part of go concurrency was never starting goroutines, it's making cancellation and shutdown behave. nice to see context, races and diagnostics in one runnable place.

  • Yup. That’s because cancellation isn’t native, but part of the context object and requires cooperation. In a language where goroutine switching is preemptive rather than cooperative, I find it odd to have cooperative cancellation, until I realize that Go doesn’t have exceptions and probably will never have them.

Go concurrency seems simple on the surface, but mastering select and proper error handling takes practice. Good to see this topic distilled.

One thing I always found more work than I would expect is when you have a graph of operations, think a Makefile, but a bit dynamic. For this model completable futures and executors seem to work well (provided the graphs is smallish), but golang is (or perhaps before generics) just was difficult.

Is a Go channel equivalent to a Haskell tvar ?

  • A go channel is just a queue with a configurable amount of buffering. Buffering 0 is the most interesting as it creates a “rendezvous” channel which syncs the sender and the receiver.

    A channel of size 1 is a bit like an mvar but with support for only take and put.

Go has a really good concurrency story. Its one of the best ones out there. Some langs have async/await (usually sucks) and some nothing att all (like php)