Comment by epestr
4 days ago
The sheep bit is the whole rationalist glorification of "Bayesian thinking". Stones being beliefs and sheep the reality you observe and the process Bayesian thinking.
Scott Alexanders's blog description for example: P(A|B) = [P(A)*P(B|A)]/P(B), all the rest is commentary.
That seems like a false dichotomy. I don't think either describes the median online opinion. Loud reactionary voices tend to be confident and extreme and bad times promote reactionary views.
The divination sticks passage is quite ironic here. Markos' belief is explained through punishment for out-think, but rationalist communities aren't exempt from social conditioning either. Repeated exposure makes certain reasoning feel natural, while people who dislike it may leave. That's part of what I'm criticising, rather than choosing an opposing internet camp. It could be concerning that the same rationalist thinking might carry into safety research (https://www.lesswrong.com/users/beth-barnes https://en.wikipedia.org/wiki/METR)
Every group has people susceptible to the same social pressures and of varying intelligence, and to claim yours is better at being less wrong, is a bit dangerous.
> It could be concerning that the same rationalist thinking might carry into safety research
Why? "Bayesian thinking" is really just empirical observation and statistics, which is the foundation of all modern research, security or otherwise.
Your comment does nothing but create a vague sense of unease (a hidden cabal of rationalists! they're even doing research!), without making any clear point. At least, I suppose, it's not 25,000 words long.
Well for one the main rationalist figure head argued with a straight face on how many people should be saved from getting dust in their eyes to justify torturing a person.
I mean, Yudkowsky is ridiculous, but this is an entirely different argument. That, in any case, isn't Bayesian in the slightest.
You've misunderstood the scare quotes. I meant the community's perception of "Bayesian thinking", rather than Bayesian inference itself. Calling your reasoning Bayesian doesn't establish that it is, much as {{person of particular religion}} calling themselves moral doesn't establish their morality.