Comment by dapperdrake
8 hours ago
Memory footprint is tiny:
Unix shell pipelines are task-parallel. Every tool gets spun up as its own unix process — think "program" (fork-exec). Standard input and standard output (stdin, stdout) get hooked up to pipes. Pipes are like temporary files managed by the kernel (hand-wave). Pipe buffer size is a few KB. Grep does a blocking read on stdin. Cat writes to stdout. Both on a kernel I/O boundary. Here the kernel can context-switch the process when waiting for I/O.
In the past there was time-slicing. Now with multiple cores and hardware threads they actually run concurrently.
This is very similar to old-school approach to something like multiple threads, but processes don’t share virtual address spaces in the CPU's memory management unit (MMU).
Further details: look up McIlroy's pipeline design.
No comments yet
Contribute on Hacker News ↗