Comment by man8alexd

I can suggest measuring working set size (WSS) instead of RSS. See https://docs.kernel.org/admin-guide/mm/multigen_lru.html and https://docs.kernel.org/mm/damon/index.html