Just want to say among many portable SIMD solutions I’ve seen recently (e.g. Fearless SIMD), this is the first that makes non-fixed vectors like SVE and RISC-V vector (RVV) easier to support. Glad to see they made this decision
How so? I imagine you'd still want to constrain the length to the maximum vector size supported by the lowest platform you want to support or you lose the portability and actually end up with code that performs much worse than the scalar alternative on some platforms.
Mojo has an even more portable simd[1] type that isn't just generic over length but also over type. In my opinion it is almost always better to specialize for each platform and use portable implementation as fallback. It's a shame that just very few languages support Zig-like comptime, because it would be excellent for specializations without introducing runtime penalties.
C++ is getting std::simd in the latest version and I am all aboard writing the vectorization with the least amount of intrinsic builtins I am able to. Even if not optimal, it's far better than the scalar ops.
Seconded!! This doesn’t really help the well established codebases much that are already doing this on a platform specific path but in general this is much appreciated for the future.
This feature opens many doors for optimizing low-level performance in Go projects, that are already running multicore. IIRC there aren’t a lot of languages with built-in std lib support for SIMD and variants. Love the way Go is trying new stuff lately.
Besides the usual C and C++, we have Java, .NET, D, Zig, Julia, Swift, Rust.
So yeah, also appreciate having Go in the group instead of manually having to write Assembly.
However not many languages adopt ways to manually write SIMD, because most of us have no idea how to write good SIMD code in first place, I surely don't.
Even with languages that adopt ways to manually write SIMD, it’s mostly left to library maintainers rather than application developers.
I work for a C++ timeseries database startup that leverages SIMD about as much as we possibly can, and except for some extremely rare places we just use libraries.
I did some testing with the experimental SIMD on a project I was doing to make speech-to-text and text-to-speech models run natively in Go (with CGO_ENABLED=0, so no C depenencies), and testing non-SIMD w/ SIMD.
I don't have formal benchmarks for that, but I can anecdotally say the SIMD work made a measurable improvement in the performance of the calculations vs. just plain Go. I'm very optimistic about how these improvements will help make the Go runtime an even better target for more of these types of work going forward, especially since it is cross-platform.
This is why I love Go. Nobody was asking for this, but they took the time to do it right and continue to Push go as a memory safe, high-level systems language.
It's been discussed for a long time, and the related proposals were heavily upvoted, including various older proposals.
As I understand it, part of the reason it took a while is that the core Go team was generally of the opinion that doing user-facing SIMD APIs the right way was to design a high-level, cross-platform API that would stand the test of time, and that was then punted a few times given its complexity and need to do other things.
Part of what helped the current approach take off was switching to a philosophy of designing a lower-level architecture-dependent API first (the 'simd/archsimd' package), and then later doing a higher-level portable API (the 'simd' package, which is topic of this blog post).
That two-level approach I think also gave some additional freedom for the design and implementation of the friendlier / high-level 'simd' package, including because the lower-level 'simd/archsimd' package is available for people who need or want to drop down.
This is kind of the opposite of Go. Not giving people what they are asking for.
There are pros and cons of course. You don't have 17 different ways to iterate over an array, so that's nice. But you also went 13 years without generics, despite them being one of the most requested features, because the designers didn't want that complexity inside Go.
Overall I think Go is better for this philosophy but there are times where the language is clearly written more for its maintainers than it's users.
Go shipped with generics (aka bounded parametric polymorphism), but only for built-in types: slices, arrays, and maps. That, with subtyping via interfaces, handled most demand for generics. The most common pain point was custom containers.
Go was first released in November 2009. Russ Cox posted "The Generic Dilemma" [1] in December 2009. The comments show the generics debate raging from the earliest days.
As a fun side note, I forgot I posted a comment on that post pointing to Ada's generics. I was in college, and Ada was our intro language.
> the designers didn't want that complexity inside Go.
Yes, with some nuance. Go's goal of writing server programs didn't require the type-system complexity and run-time hit of user-defined generics. [2]
> Go was intended as a language for writing server programs [...] Polymorphic programming did not seem essential [...] so was initially left out for simplicity.
>
> Generics are convenient but they come at a cost in complexity in the type system and run-time. It took a while to develop a design that we believe gives value proportionate to the complexity.
Some of the concerns around generics and why it took so long were for the users as well. One of the biggest draws to Go has always been that you get the performance of a compiled language and yet compile times are so low that it can feel like you're developing with an interpreted language. The design of generics needed to maintain the compile time advantage or else it wouldn't feel like Go any more.
That's not what "memory safe" means. "Memory safe" is a term of art meaning "not susceptible to memory corruption exploits", like stack and heap overflows, UAFs, and type confusion. Last I checked, there are essentially no non-contrived memory corruption exploits for Go programs; the best you get are people demonstrating register control on contrived programs.
The definition I'm giving is the same as the ISRG's definition at MemorySafety.org. It's the thing everybody is talking about when they talk about memory safety.
The claim being made here is "big if true", because it would imply a lot more languages than Go "aren't memory safe", despite decades without memory corruption exploits.
(The gist: memory safety is a term of art coined by security practitioners. Go, Python, Rust, Java, others: memory safe. C/C++: memory unsafe. Periodically, people in different slices of industry or academia come up with new definitions of memory safety that declare Rust or Go or other languages to be memory unsafe, but that is not by the broadly accepted definition across industry.)
You can write unsafe code in Go (import unsafe), but then, you can do the same in Rust. Unsafe code is not the default, and in day to day Go i rarely see the use of the unsafe package.
Very neat, and comes pretty close to how Mojo handles portable SIMD.
It's great to see two of my favorite languages finally making SIMD easy to use. It's such low-hanging fruit for performance, yet somehow languages have ignored it for years. Portable SIMD, even with some performance penalty, still beats scalar computation whenever vector operations are needed. Yet language implementations always seemed to assume that hardware-specific SIMD APIs were the only way to go. That did nothing but make SIMD unusable excepting special cases where performance is absolutely critical, rather than just something anyone can use in day to day programming.
> The new simd package hides these differences by removing fixed-size vectors from the type system, and by only supporting those operations that are in the intersection of all the different platforms, and fills gaps in the intersection with efficient emulation in terms of other SIMD instructions.
The intersection would be the operations supported by all platforms and so would not have gaps.
> The interface conversion and type switch look like they should be inefficient, but the compiler-side implementation of simd specializes code and optimizes away the type switch.
I don’t understand this - how is it able to if the same go binary might run on unknown types? I’m assuming what it means is that the switch is implemented efficiently due to CPU branch prediction? I know fearless SIMD is doing cool stuff with static dispatch so that the feature set is checked just once at program start - is that what it means it’s doing under the hood? Very unclear.
It creates multiple versions of functions referencing SIMD and lifts the dispatch switching cost to their callers.
> The AST rewrite creates multiple specialized copies of functions, variables, and types that mention simd types, where simd types are replaced with references to size-specialized types in simd/internal/bridge. Each of these bridge types is defined as an archsimd type, but with a restricted set of methods. The specialized functions, variables, and types acquire a suffix of the form @simdNNN, where NNN is either a vector length (128, 256, or 512) or 0, indicating emulation. Functions that mention simd internally, but not in their signature, are converted to wrappers that switch on the SIMD level detected at program start, and call the appropriate specialized version of that function. Specialized functions call other specialized functions directly without dispatch overhead (and perhaps with inlining). This rewrite strategy was chosen as a compromise between code duplication and SIMD performance; the overhead is hoisted as high as necessary to avoid dispatch within SIMD computations, but not higher. If SIMD dispatch appears “too low” in a computation, a gratuitous mention of a simd type will move it upwards, as in this example:
> Syntax highlighting is juvenile. When I was a child, I was taught
arithmetic using colored rods
(http://en.wikipedia.org/wiki/Cuisenaire_rods). I grew up and today I
use monochromatic numerals.
Not convinced at all by these reasons. Rob Pike has long stepped down from the Go team so it shouldn't matter? This is a website not a language change.
It's hard to make predictions with an open source project, but my personal guess is some flavor of it will land (including it is already demonstrating good results without an enormous level of code complexity in the compiler and without overly slowing down compile speeds), but I guess we'll see.
It's being driven by an external contributor who has landed some good changes in the past to the Go compiler. (I think the autovectorization work might be part of their PhD or other academic research, but not sure.)
As a first step, it might be possible to write a linter rule that rewrites suitable numeric loops to SIMD. There are already rules to rewrite several loop types, so that should be doable.
The problem with Go isn't performance but with the C/C++ interop overhead, even with the "30% less overhead" from a few updates ago which isnt true for 99% of cases, it isnt enough
its not limited but it has overhead because of the memory model of go doesnt match the C one so there has to be some sort of rerodering being done, that's what i understood atleast, and theres also the go concurrency
Use Assembly instead of CGO, isn't that scary, back in the 8 bit days we were coding Assembly aged 10, on our Spectrum, C64, Atari, Apple, Acorn, MSX,....
Already using this for foreground estimation of cutouts in my project, around 30% speedup over non-SIMD, but the algorithm is probably not very optimised yet.
I personally use both, and keep using both. There are much more Golang job in the market now. Noone planning to ditch Go in my network or unhappy with it. Highload E-commerce, logistics, etc are way easier to write in Go IMHO.
Discover a very good niche for Rust - geo spatial analytics. Would not do it Go or Python. LLMs gave a huge boost to Rust too. Claude produce a very high code ... if designed right. Lot of feature complete libraries now.
I suspect you stopped reading at web services on your link, but API is indeed the correct word to describe a set of functions from a library (built-in or not). If you disagree perhaps you should share your preferred term here?
> The term API is often used to refer to web APIs, which allow communication between computers that are joined by the internet. There are also APIs for programming languages, software libraries, computer operating systems, and computer hardware.
https://imjasonh.github.io/playground/palette-swap/ swaps colors in a provided image in wasm, entirely locally in your browser, to benchmark portable SIMD vs non-portable archsimd vs non-SIMD.
Portable SIMD is ~11% slower than non-portable SIMD in this case, but both are ~5x faster than non-SIMD.
Just want to say among many portable SIMD solutions I’ve seen recently (e.g. Fearless SIMD), this is the first that makes non-fixed vectors like SVE and RISC-V vector (RVV) easier to support. Glad to see they made this decision
We pioneered this in Highway and shared some advice on the API. Great to see this decision taken :D
https://github.com/google/highway
How so? I imagine you'd still want to constrain the length to the maximum vector size supported by the lowest platform you want to support or you lose the portability and actually end up with code that performs much worse than the scalar alternative on some platforms.
Mojo has an even more portable simd[1] type that isn't just generic over length but also over type. In my opinion it is almost always better to specialize for each platform and use portable implementation as fallback. It's a shame that just very few languages support Zig-like comptime, because it would be excellent for specializations without introducing runtime penalties.
1. https://mojolang.org/docs/std/simd/SIMD/
Curious how much of the emulation ends up in hot paths before SVE and the feature variants land.
C++ is getting std::simd in the latest version and I am all aboard writing the vectorization with the least amount of intrinsic builtins I am able to. Even if not optimal, it's far better than the scalar ops.
Seconded!! This doesn’t really help the well established codebases much that are already doing this on a platform specific path but in general this is much appreciated for the future.
Write it once with N errors, not N*M errors :)
This feature opens many doors for optimizing low-level performance in Go projects, that are already running multicore. IIRC there aren’t a lot of languages with built-in std lib support for SIMD and variants. Love the way Go is trying new stuff lately.
Besides the usual C and C++, we have Java, .NET, D, Zig, Julia, Swift, Rust.
So yeah, also appreciate having Go in the group instead of manually having to write Assembly.
However not many languages adopt ways to manually write SIMD, because most of us have no idea how to write good SIMD code in first place, I surely don't.
You probably weren't looking for a tutorial about SIMD, but just in case you were interested, Mitchell [0] did one recently that got on HN [1]
[0] Mitchell Hashimoto: "Everyone Should Know SIMD" https://news.ycombinator.com/item?id=49010648
3 replies →
Even with languages that adopt ways to manually write SIMD, it’s mostly left to library maintainers rather than application developers.
I work for a C++ timeseries database startup that leverages SIMD about as much as we possibly can, and except for some extremely rare places we just use libraries.
1 reply →
With AI I'm pretty sure SIMD will be easier to integrate when necessary.
3 replies →
Vectorizing computations has been Matlabs secret sauce.
Does matlab these days do stuff like JIT operator fusing to avoid memory roundtrips and take advantage of FMAs?
1 reply →
Julia does that too.
I did some testing with the experimental SIMD on a project I was doing to make speech-to-text and text-to-speech models run natively in Go (with CGO_ENABLED=0, so no C depenencies), and testing non-SIMD w/ SIMD.
I don't have formal benchmarks for that, but I can anecdotally say the SIMD work made a measurable improvement in the performance of the calculations vs. just plain Go. I'm very optimistic about how these improvements will help make the Go runtime an even better target for more of these types of work going forward, especially since it is cross-platform.
This will welcome more database/warehouses to be written in Go.
Personally I will implement it in https://github.com/viggy28/streambed
They could have used third party packages or Assembly directly.
This naturally is an easier way.
You're right. I could have but this encourages me to seriously consider it.
This is why I love Go. Nobody was asking for this, but they took the time to do it right and continue to Push go as a memory safe, high-level systems language.
People were definitely asking for it.
It's been discussed for a long time, and the related proposals were heavily upvoted, including various older proposals.
As I understand it, part of the reason it took a while is that the core Go team was generally of the opinion that doing user-facing SIMD APIs the right way was to design a high-level, cross-platform API that would stand the test of time, and that was then punted a few times given its complexity and need to do other things.
Part of what helped the current approach take off was switching to a philosophy of designing a lower-level architecture-dependent API first (the 'simd/archsimd' package), and then later doing a higher-level portable API (the 'simd' package, which is topic of this blog post).
That two-level approach I think also gave some additional freedom for the design and implementation of the friendlier / high-level 'simd' package, including because the lower-level 'simd/archsimd' package is available for people who need or want to drop down.
It's a nice design.
1 reply →
This is kind of the opposite of Go. Not giving people what they are asking for.
There are pros and cons of course. You don't have 17 different ways to iterate over an array, so that's nice. But you also went 13 years without generics, despite them being one of the most requested features, because the designers didn't want that complexity inside Go.
Overall I think Go is better for this philosophy but there are times where the language is clearly written more for its maintainers than it's users.
> But you also went 13 years without generics
Go shipped with generics (aka bounded parametric polymorphism), but only for built-in types: slices, arrays, and maps. That, with subtyping via interfaces, handled most demand for generics. The most common pain point was custom containers.
Go was first released in November 2009. Russ Cox posted "The Generic Dilemma" [1] in December 2009. The comments show the generics debate raging from the earliest days.
As a fun side note, I forgot I posted a comment on that post pointing to Ada's generics. I was in college, and Ada was our intro language.
[1]: https://research.swtch.com/generic
> the designers didn't want that complexity inside Go.
Yes, with some nuance. Go's goal of writing server programs didn't require the type-system complexity and run-time hit of user-defined generics. [2]
> Go was intended as a language for writing server programs [...] Polymorphic programming did not seem essential [...] so was initially left out for simplicity. > > Generics are convenient but they come at a cost in complexity in the type system and run-time. It took a while to develop a design that we believe gives value proportionate to the complexity.
[2]: https://go.dev/doc/faq#beginning_generics
Out of curiosity, I collected all proposals for Go's journey to generics. https://gist.github.com/jschaf/eaa7aff1af14ea7276a18a1b7370d...
Some of the concerns around generics and why it took so long were for the users as well. One of the biggest draws to Go has always been that you get the performance of a compiled language and yet compile times are so low that it can feel like you're developing with an interpreted language. The design of generics needed to maintain the compile time advantage or else it wouldn't feel like Go any more.
Mostly safe, contrary to other safer languages, Go memory model doesn't prevent data tearing.
go data races aren't memory safe
That's not what "memory safe" means. "Memory safe" is a term of art meaning "not susceptible to memory corruption exploits", like stack and heap overflows, UAFs, and type confusion. Last I checked, there are essentially no non-contrived memory corruption exploits for Go programs; the best you get are people demonstrating register control on contrived programs.
The definition I'm giving is the same as the ISRG's definition at MemorySafety.org. It's the thing everybody is talking about when they talk about memory safety.
The claim being made here is "big if true", because it would imply a lot more languages than Go "aren't memory safe", despite decades without memory corruption exploits.
4 replies →
Yeah, we get it, you performatively hate go.
Yes, but in practice they are extremely hard to exploit. It has been discussed extensively here on HN and in other forums.
That does not make sense to me. Go is memory-safe, but it does not guarantee data-race freedom.
So whats your point here? Haskell?
20 replies →
(removed)
Go is broadly considered to be a memory safe language.
See for example comments from tptacek like:
https://news.ycombinator.com/item?id=44672371
(The gist: memory safety is a term of art coined by security practitioners. Go, Python, Rust, Java, others: memory safe. C/C++: memory unsafe. Periodically, people in different slices of industry or academia come up with new definitions of memory safety that declare Rust or Go or other languages to be memory unsafe, but that is not by the broadly accepted definition across industry.)
10 replies →
You can write unsafe code in Go (import unsafe), but then, you can do the same in Rust. Unsafe code is not the default, and in day to day Go i rarely see the use of the unsafe package.
9 replies →
No idea why you're getting downvoted for true statement. Without a ? like in C# you're always at risk of a nil pointer being dereferenced
6 replies →
Very neat, and comes pretty close to how Mojo handles portable SIMD.
It's great to see two of my favorite languages finally making SIMD easy to use. It's such low-hanging fruit for performance, yet somehow languages have ignored it for years. Portable SIMD, even with some performance penalty, still beats scalar computation whenever vector operations are needed. Yet language implementations always seemed to assume that hardware-specific SIMD APIs were the only way to go. That did nothing but make SIMD unusable excepting special cases where performance is absolutely critical, rather than just something anyone can use in day to day programming.
I am using Go assembly for SIMD very heavily in https://github.com/rcarmo/go-pherence, this is just icing on the cake.
> The new simd package hides these differences by removing fixed-size vectors from the type system, and by only supporting those operations that are in the intersection of all the different platforms, and fills gaps in the intersection with efficient emulation in terms of other SIMD instructions.
The intersection would be the operations supported by all platforms and so would not have gaps.
> The interface conversion and type switch look like they should be inefficient, but the compiler-side implementation of simd specializes code and optimizes away the type switch.
I don’t understand this - how is it able to if the same go binary might run on unknown types? I’m assuming what it means is that the switch is implemented efficiently due to CPU branch prediction? I know fearless SIMD is doing cool stuff with static dispatch so that the feature set is checked just once at program start - is that what it means it’s doing under the hood? Very unclear.
It creates multiple versions of functions referencing SIMD and lifts the dispatch switching cost to their callers.
> The AST rewrite creates multiple specialized copies of functions, variables, and types that mention simd types, where simd types are replaced with references to size-specialized types in simd/internal/bridge. Each of these bridge types is defined as an archsimd type, but with a restricted set of methods. The specialized functions, variables, and types acquire a suffix of the form @simdNNN, where NNN is either a vector length (128, 256, or 512) or 0, indicating emulation. Functions that mention simd internally, but not in their signature, are converted to wrappers that switch on the SIMD level detected at program start, and call the appropriate specialized version of that function. Specialized functions call other specialized functions directly without dispatch overhead (and perhaps with inlining). This rewrite strategy was chosen as a compromise between code duplication and SIMD performance; the overhead is hoisted as high as necessary to avoid dispatch within SIMD computations, but not higher. If SIMD dispatch appears “too low” in a computation, a gratuitous mention of a simd type will move it upwards, as in this example:
[flagged]
For God sake, add a syntax highlighting on the official page! Otherwise this is awesome
Rob Pike seems to dislike syntax highlighting:
> Syntax highlighting is juvenile. When I was a child, I was taught arithmetic using colored rods (http://en.wikipedia.org/wiki/Cuisenaire_rods). I grew up and today I use monochromatic numerals.
https://groups.google.com/g/golang-nuts/c/hJHCAaiL0so/m/kG3B...
He is definitely not alone in thinking this, for example (but for different reasons): https://www.linusakesson.net/programming/syntaxhighlighting/
I like Linus's work, but that post smells of rotting strawmen. All the examples look manipulated into throwing the opposition under the bus.
Not convinced at all by these reasons. Rob Pike has long stepped down from the Go team so it shouldn't matter? This is a website not a language change.
Oh this is great, it was one of my biggest bugbears about Go since you almost always have to link C/C++ code to get the appropriate performance.
The one negative I'd say is that often autovectorisation is 'good enough' and this doesn't really tackle that gap.
FWIW, there is some pretty substantial autovectorization work that is already in-flight for the Go compiler.
There's a CL stack here:
https://go.dev/cl/791740
It's hard to make predictions with an open source project, but my personal guess is some flavor of it will land (including it is already demonstrating good results without an enormous level of code complexity in the compiler and without overly slowing down compile speeds), but I guess we'll see.
It's being driven by an external contributor who has landed some good changes in the past to the Go compiler. (I think the autovectorization work might be part of their PhD or other academic research, but not sure.)
As a first step, it might be possible to write a linter rule that rewrites suitable numeric loops to SIMD. There are already rules to rewrite several loop types, so that should be doable.
The poor Assembler and the unsafe package forgotten in the corner.
While reaching out to CGO is the easier way, it doesn't mean it is the only tool available in Go.
The problem with Go isn't performance but with the C/C++ interop overhead, even with the "30% less overhead" from a few updates ago which isnt true for 99% of cases, it isnt enough
Why is that the case? I don’t know low level programming so why is Go limited in interop with C?
its not limited but it has overhead because of the memory model of go doesnt match the C one so there has to be some sort of rerodering being done, that's what i understood atleast, and theres also the go concurrency
Use Assembly instead of CGO, isn't that scary, back in the 8 bit days we were coding Assembly aged 10, on our Spectrum, C64, Atari, Apple, Acorn, MSX,....
I hope portable simd will be stabilized some time in rust :/
How many "functions" compile to movd?
Already using this for foreground estimation of cutouts in my project, around 30% speedup over non-SIMD, but the algorithm is probably not very optimised yet.
Rust kind of seems to have overtaken Go in momentum recently. I wonder if Go will do well in, say, two years from now on.
I personally use both, and keep using both. There are much more Golang job in the market now. Noone planning to ditch Go in my network or unhappy with it. Highload E-commerce, logistics, etc are way easier to write in Go IMHO.
Discover a very good niche for Rust - geo spatial analytics. Would not do it Go or Python. LLMs gave a huge boost to Rust too. Claude produce a very high code ... if designed right. Lot of feature complete libraries now.
Both will do fine
[flagged]
[dead]
> Go 1.26 and 1.27 include experimental APIs for Single Instruction Multiple Data (SIMD) operations.
You'd think these people would know the meaning of API, no?
One wonders what overly-narrow definition of API you're stuck on.
One wonders why you wonder.
https://en.wikipedia.org/wiki/API
I suspect you stopped reading at web services on your link, but API is indeed the correct word to describe a set of functions from a library (built-in or not). If you disagree perhaps you should share your preferred term here?
> The term API is often used to refer to web APIs, which allow communication between computers that are joined by the internet. There are also APIs for programming languages, software libraries, computer operating systems, and computer hardware.
> API is indeed the correct word to describe a set of functions from a library (built-in or not).
Uncorroborated by WP, note.
> If you disagree perhaps you should share your preferred term here?
Library functions.