← Back to context

Comment by smasher164

4 years ago

Also, mandating copy-on-write as an implementation strategy is a huge burden to place on the host. Now you’ve made the amount of memory a process is is using unquantifiable.

7 comments

smasher164

Reply
vgel  4 years ago

It's not necessarily unquantifiable -- the kernel can count the not-yet-copied pages pessimistically as allocated memory, triggering OOM allocation failures if the amount of potential memory usage is greater than RAM. IIUC, this is how Linux vm.overcommit_memory[1] mode 2 works, if overcommit_ratio = 100.

However, if an application is written to assume that it can fork a ton and rely on COW to not trigger OOM, it obviously won't work under mode 2.

[1] https://www.kernel.org/doc/Documentation/vm/overcommit-accou...

> 2 - Don't overcommit. The total address space commit for the system is not permitted to exceed swap + a configurable amount (default is 50%) of physical RAM.

> Depending on the amount you use, in most situations this means a process will not be killed while accessing pages but will receive errors on memory allocation as appropriate.

> Useful for applications that want to guarantee their memory allocations will be available in the future without having to initialize every page.

  • smasher164  4 years ago

    You're right, "unquantifiable" was the wrong word here. I meant, a program has no real way of predicting/reacting to OOM. I didn't realize mode 2 with overcommit_ratio = 100 behaved that way, thanks for sharing.

    • vgel  4 years ago

      Yeah I think in a practical sense you're right, since AFAIK using mode 2 is fairly rare because most software assumes overcommit, and even if a program is written with an understanding that malloc can return NULL, its in the sense of

          if (!(ptr = malloc(...))) { exit(1); }

cryptonector  4 years ago

POSIX doesn't require that fork() be implemented using copy-on-write techniques. An implementation is free to copy all of the parent's writable address space.

  • int_19h  4 years ago

    An implementation of fork() that doesn't do CoW would have borderline unusable perf in many real-world scenarios.

    • cryptonector  4 years ago

      If the parent is a JVM, for sure. But a copy-on-write fork() still doesn't perform well. The point isn't to just copy the whole parent. The point is to stop copying at all.