Comment by kstenerud
4 days ago
Yup, unsigned math is just nasty.
Actually, unchecked math on an integer is going to be bad regardless of whether it's signed or unsigned. The difference is that with signed integers, your sanity check is simple and always the same and requires no thought for edge cases: `if(index < 0 || index > max)`. Plus ubsan, as mentioned above.
My policy is: Always use signed, unless you have a specific reason to use unsigned (such as memory addresses).
unsigned is easier: 'if(index >= max)' and has fewer edge cases because you don't need to worry about undefined behavior when computing index.
Just because it's UB doesn't mean it's not a problem, though. If you do unsigned arithmetics but don't account for the possibility of wraparound on overflow, the resulting value is well-defined, but it does you no good if you then try to e.g. index using it and cause a buffer overflow.
> The difference is that with signed integers, your sanity check is simple and always the same and requires no thought for edge cases: `if(index < 0 || index > max)`
Wait, what? How is that easier than `if (index > max)`?
Because if max is a calculated value, it could silently wrap around and leave index to cause a buffer overflow.
Or if index is counting down, a calculated index could silently wrap around and cause the same issue.
And if both are calculated and wrap around, you'll have fun debugging spooky action at a distance!
If both are signed, that won't happen. You probably do have a bug if max or index is calculated to a negative value, but it's likely not an exploitable one.
I have no clue what cases you have in mind, can you give some examples? Surely when you have index as unsigned the maximum would be represented unsigned as well?