← Back to context

Comment by otterley

9 months ago

Indeed, there’s nothing at all wrong with sharing anecdotes. The problem is when people make broad assumptions and conclusions based solely on personal experience, which unfortunately happens all too often. Doing so is wired into our brains, though, and we have to work very consciously to intercept our survival instincts.

4 comments

otterley

Reply
freehorse  9 months ago

People "make conclusions" because they have to take decisions day to day. We cannot wait for the perfect bulletproof evidence before that. Data is useful to take into account, but if I try to use X llm that has some perfect objective benchmark backing it, while I cannot make it be useful to me while Y llm has better results, it would be stupid not to base my decision on my anecdotal experience. Or vice versa, if I have a great workflow with llms, it may be not make sense to drop it because some others may think that llms don't work.

In the absence of actually good evidence, anecdotal data may be the best we can get now. The point imo is try to understand why some anecdotes are contrasting each other, which, imo, is mostly due to contextual factors that may not be very clear, and to be flexible enough to change priors/conclusions when something changes in the current situation.

  • otterley  9 months ago

    Agreed 100%. When insufficient data exists, you have to fall back to other sources like analogies, personal observations, secondhand knowledge, etc. However, I’ve seen too many instances of people claiming their own limited experience is the truth when overwhelming and easily attainable evidence and data exists that proves it to be false.

droopyEyelids  9 months ago

I think you might be caught up in a bit of the rationalist delusion.

People -only!- draw conclusions based on personal experience. At best you have personal experience with truly objective evidence gathered in a statistically valid manner.

But that only happens in a few vanishingly rare circumstances here on earth. And wherever it happens, people are driven to subvert the evidence gathering process.

Often “working against your instincts” to be more rational only means more time spent choosing which unreliable evidence to concoct a belief from.