Very confused by the article. Is memoization new to the eBPF world? Did the author only just learn about it and wanted to use it?
In reality, the article is about correctly caching a path:policy mapping while working within the limitations of eBPF and Linux filesystem semantics. If you read the article in that context rather than wondering 'what is new and interesting about memoization in eBPF?' it's a lot more interesting.
I probably would have titled this 'Calculating cache keys for filesystem paths in eBPF' or something, since that's the cool and interesting problem that was solved.
Seems like the 90% faster case is opening the same exact file every single time, which seems like a not super standard use case that will benefit the most from this caching. On an example where you never open the same file twice this will presumably be slightly slower than before, as you’re doing the same thing but writing to a cache.
So you can make the headline “Drop performance cost by 90%!” or “Modestly increase performance cost” and be correct but I don’t think either is really a reasonable description of the change.
How is the cache invalidated when:
- The permissions change?
- Directories are moved?
- Hard-links are added?
- Things are deleted?
Also: Is the cache limited in size?
I'd also be interested in this, particularly in the "move" case, which requires no privileges and can also happen without the kernel being in the loop (e.g. on a different machine, in NFS).
If your calculated policy is a function of the entire path, you'd normally need to key the calculated policy by the entire path and its relevant properties (underlying inodes) as well, no?
There are path-based LSMs in the Linux kernel that do not employ such caching. If you found a solution for this which also works in the corner cases and results in speed-up in real life scenarios, I'd be interested :)
How do your path-only rules handle the many approaches for loading a file but making it appear to have a different path, such as bind mounts for one example?
Agent is an old school way to say "constantly running daemon to accomplish some specific purpose", as someone who wrote a lot of performance monitoring agents. AI is using the term appropriately, but I think assuming it's always AI-first is slightly tragic.
Umm, wouldn't this break if you moved a directory that is somewhere up the path? Seems like a security issue if you cache what policy applies but the policy could change by user action.
Hey, author here, that is a great catch, thanks for pointing it out! If we are protecting a directory, then only people with access will be able to move the dir, so we are assuming that they don’t move (or rename) the directory maliciously. And, if we aren’t trying to protect the directory, then it doesn’t really mater to our protection whether that directory is moved.
Additionally, we are thinking of evicting the inode associated with the directory from the cache if a directory is moved. Doing this would probably catch a ton of edge cases and make it simpler.
At some point, i feel like a tech article written to a tech audience has the right to assume certain technical knowledge. eBPF is not an obscure technology in the linux world, and its impossible to make your article target everyone while still being a good article.
Very confused by the article. Is memoization new to the eBPF world? Did the author only just learn about it and wanted to use it?
In reality, the article is about correctly caching a path:policy mapping while working within the limitations of eBPF and Linux filesystem semantics. If you read the article in that context rather than wondering 'what is new and interesting about memoization in eBPF?' it's a lot more interesting.
I probably would have titled this 'Calculating cache keys for filesystem paths in eBPF' or something, since that's the cool and interesting problem that was solved.
Definitely not new in eBPF or LSMs in general.
Edit: link to the SELinux kernel Access Vector Cache (AVC): https://elixir.bootlin.com/linux/v3.4.64/source/security/sel...
Seems like the 90% faster case is opening the same exact file every single time, which seems like a not super standard use case that will benefit the most from this caching. On an example where you never open the same file twice this will presumably be slightly slower than before, as you’re doing the same thing but writing to a cache. So you can make the headline “Drop performance cost by 90%!” or “Modestly increase performance cost” and be correct but I don’t think either is really a reasonable description of the change.
How is the cache invalidated when: - The permissions change? - Directories are moved? - Hard-links are added? - Things are deleted? Also: Is the cache limited in size?
I'd also be interested in this, particularly in the "move" case, which requires no privileges and can also happen without the kernel being in the loop (e.g. on a different machine, in NFS).
If your calculated policy is a function of the entire path, you'd normally need to key the calculated policy by the entire path and its relevant properties (underlying inodes) as well, no?
There are path-based LSMs in the Linux kernel that do not employ such caching. If you found a solution for this which also works in the corner cases and results in speed-up in real life scenarios, I'd be interested :)
How do your path-only rules handle the many approaches for loading a file but making it appear to have a different path, such as bind mounts for one example?
And uh, what happens if the active user permission changes?
So you highlight "Not AI Gen" but you call the solution "Agent"? :D
Agent is an old school way to say "constantly running daemon to accomplish some specific purpose", as someone who wrote a lot of performance monitoring agents. AI is using the term appropriately, but I think assuming it's always AI-first is slightly tragic.
No. The oldschool is "daemon".
6 replies →
Umm, wouldn't this break if you moved a directory that is somewhere up the path? Seems like a security issue if you cache what policy applies but the policy could change by user action.
Hey, author here, that is a great catch, thanks for pointing it out! If we are protecting a directory, then only people with access will be able to move the dir, so we are assuming that they don’t move (or rename) the directory maliciously. And, if we aren’t trying to protect the directory, then it doesn’t really mater to our protection whether that directory is moved.
Additionally, we are thinking of evicting the inode associated with the directory from the cache if a directory is moved. Doing this would probably catch a ton of edge cases and make it simpler.
[dead]
TL;DR - use in-memory cache instead of expensive database lookups on every iteration
I don't know what any of these acronyms mean. With just a little more explanation the article could be accessible to a wider developer audience.
At some point, i feel like a tech article written to a tech audience has the right to assume certain technical knowledge. eBPF is not an obscure technology in the linux world, and its impossible to make your article target everyone while still being a good article.
Do yourself a favor and look up eBPF - it's incredibly powerful and cool tech. Basically lets you write scripts that run inside the kernel
I dont think an article title "dropping ebpf cpu cost with memoization" really needs to be accessible to a broad audience.
> Not AI Gen
I am not sure how to react ... of course good thing it's human-gen, but I still like a few AI passes over it to tighten it up. LOL