Hard to say without looking at the complete setup - and probably just a side-effect of the underlying issue. The question is, why did you have such high RTTs? That already points to a different cause.
I would even argue that NODELAY for a VoIP solution makes no sense - why are you even using TCP instead of UDP in the first place?
Then why did it stop happening when NODELAY was used?
Hard to say without looking at the complete setup - and probably just a side-effect of the underlying issue. The question is, why did you have such high RTTs? That already points to a different cause.
I would even argue that NODELAY for a VoIP solution makes no sense - why are you even using TCP instead of UDP in the first place?