← Back to context

Comment by IshKebab

18 hours ago

It's a stack overflow which means it's using recursion and for historical reasons that don't make sense any more, stacks are teeny tiny on 64-bit Linux - apparently only 8 MB on Linux! I'm not sure why they don't raise it to something reasonable like 4 GB. I guess because they want consistency with 32-bit? Maybe we can finally change it if/when they phase out support for 32-bit Linux. Apparently it might not be that far away:

https://lwn.net/Articles/1035727/

17 comments

IshKebab

Reply
ploxiln  6 hours ago

8MB is the default per-thread stack size from glibc, also seems to be the default "ulimit" from pam or the kernel, I'm not sure. So for the main/default thread (or if not using threads) the process can use setrlimit() and for threads it can use pthread_attr_setstacksize() to get bigger stacks if it knows it may need them.

8MB is pretty huge though; musl libc is famous for defaulting to much smaller per-thread stack size of 128KB (to avoid over-committing lots of memory when there are many threads - the main dev is really principled/opinionated on this topic, but again there are a few ways for applications to explicitly size their stacks as large as they need). Linux kernel threads get a bit less than 16KB!

tosti  18 hours ago

OIC. Rust doesn't guarantee optimizing tail recursion. How unfortunate for a language that's getting widespread adoption.

  • gpm  17 hours ago

    For what it's worth there's reasonably active [1] work on implementing opt-in guaranteed tail calls - but it's not particularly fast going. LLVM (the backend rust uses) needs better support for musttail (e.g. some architectures just don't support it [2]).

    [1] https://github.com/rust-lang/rust/issues/112788

    [2] https://github.com/rust-lang/rust/issues/153827

    By-default guaranteed tail calls really isn't rust's style, because it means subtle changes (introducing a destructor, re-ordering code, etc) can change semantics without you realizing it. If you want to guarantee that a call can't allocate a new stack frame you should have to say it.

    • lioeters  16 hours ago

      Not so familiar with this area, but isn't the existing behavior of implicitly creating new stacks more of a problem than implicit tail-call elimination? Seems the latter is a kind of compiler-level optimization, of which there are already many (I think) that change the semantics internally but guarantee the outward behavior stays the same.

      But I can understand the preference for an explicit opt-in, to make clear that it is enforced and not assumed.

      5 replies →

    • jmalicki  8 hours ago

      > because it means subtle changes (introducing a destructor, re-ordering code, etc) can change semantics without you realizing it.

      No, it won't change semantics - if you say @musttail or similar, it will simply fail to compile if you, say, introduce a destructor - the semantics will not subtly change.

      4 replies →

  • shiomiru  2 hours ago

    I don't think that's related? The bug alluded to looks something like

        function rm(node) {
        for (const child of ls(node))
            rm(child);
        unlink(node);
    }

    and no amount of tail call optimization will save you here, because this isn't tail recursion. Of course you could rewrite it using an explicit stack + tail recursion, but then you might as well be using a while loop.

  • IshKebab  13 hours ago

    Do any widely used languages guarantee tail call optimization? It's a pretty niche feature.

    • gpm  13 hours ago

      Scala, ocaml, racket, clojure, zig.

      For recursion only kotlin.

      (For most of these only with syntax specifying it)