Comment by groby_b

You keep your source tree on a separate file system with version control, one hopes.

So you only snapshot the rest. (And since your source tree is under version control, finding stray garbage there is straightforward unless you checked it in)