Comment by jacquesm

The capacitor has a self inductance. That's why you use low self inductance capacitors with very short leads or traces in this role. 100 nF ceramics are fine, but you may actually need a 100 nF and a 10 nF side-by-side because of that inductance depending on how dirty your power line is. Fast clocked circuitry can be pretty nasty.