Comment by dataflow
6 days ago
Likely vs. unlikely is rounding to 50%. Single digit is rounding to 1%. I don't think the parent was suggesting the former is better than the latter. Even before I read your comment I thought that 5% precision is useful but 1% precision is a silly turn-off, unless that 1% is near the 0% or 100% boundary.
The book Superforecasting documented that for their best forecasters, rounding off that last percent would reliably reduce Brier scores.
Whether rationalists who are publicly commenting actually achieve that level of reliability is an open question. But that humans can be reliable enough in the real world that the last percentage matters, has been demonstrated.
Your comment is incredibly confusing (possibly misleading) because of the key details you've omitted.
> The book Superforecasting documented that for their best forecasters, rounding off that last percent would reliably reduce Brier scores.
Rounding off that last percent... to what, exactly? Are you excluding the exceptions I mentioned (i.e. when you're already close to 0% or 100%?)
Nobody is arguing that 3% -> 4% is insignificant. The argument is over whether 16% -> 15% is significant.
To the nearest 5%, for percentages in that middle range. It is not just 16% -> 15%. But also 46% -> 45%.
2 replies →
The most useful frame here is looking at log odds. Going from 15% -> 16% means
-log_2(.15/(1-.15)) -> -log_2(.16/1-.16))
=
2.5 -> 2.39
So saying 16% instead of 15% implies an additional tenth of a bit of evidence in favor (alternatively, 16/15 ~= 1.07 ~= 2^.1).
I don't know if I can weigh in on whether humans should drop a tenth of a bit of evidence to make their conclusion seem less confident. In software (eg. spam detector), dropping that much information to make the conclusion more presentable would probably be a mistake.
I thought single digit means single significant digit, aka rounding to 10%?
I did mean 1%, not sure if I used the right term though, english not being my first language.
Wasn't 16% the example they were talking about? Isn't that two significant digits?
And 16% very much feels ridiculous to a reader when they could've just said 15%.
In context, the "at least 16%" is responding to someone who said 8%, and 16 just happens to be exactly twice 8. I suspect (though I don't know) that Yudkowsky would not have claimed to have a robust way to pick whether 16% or 17% was the better figure.
For what it's worth, I don't think there's anything even slightly wrong with using whatever estimate feels good to you, even if it happens not to fit someone else's criterion for being a nice round number, even if your way of getting the estimate was sticking a finger in the air and saying the first number you thought of. You never make anything more accurate by rounding it[1], and while it's important to keep track of how precise your estimates are I think it's a mistake to try to do that by modifying the numbers. If you have two pieces of information (your best estimate, and how fuzzy it is), you should represent it as two pieces of information[2].
[1] This isn't strictly true, but it's near enough.
[2] Cf. "Pitman's two-bit rule".
2 replies →
My interpretation was that Yudkowski simply doubled Christiano's guess of 8% (as one might say in conversation "oh it's at least double that", but using the actual number)