Comment by dang
10 years ago
You're right to single out that sentence, or rather that word "all" in "All the interviewing and evaluating will be done in regular HN threads". We didn't stick to that, and I'm sorry. That's precisely what we changed, and yes we changed it at the last minute.
I didn't think of this experiment as having fixed rules, but rather as something we'd figure out as we went along. I tried to make that clear up front: "Note that word experiment! We'll start small and figure it out as we go. But here are the initial conditions." I can see why that wasn't enough.
It occurs to me only now that "figure it out as we go" and "changing the rules" are the same thing. So yes. We changed the rules, and I'm sorry.
The only catch with this is that "Note that word experiment! We'll start small and figure it out as we go. But here are the initial conditions." can very easily be taken to mean "we'll start small this time, and adjust things for the next time, figuring it out as we go." That's how I, and seemingly some others, interpreted it.
"We'll change things next time based on the results of this experiment" and "We'll change things this time as we see fit and appropriate" are very different, though your initial statement can be read as implying either.
I don't even think one is better than the other - I would just be more careful about communicating which it really is, next time. You guys are free to do what you want, heh. This entire "debacle," if you can even call it that, is actually a great case study in how and why clear communication is so important. The problem wasn't the decision that was made, but rather how it was communicated.
> can very easily be taken to mean "we'll start small this time, and adjust things for the next time, figuring it out as we go."
That's not what I meant at all. This is the first time this interpretation ever occurred to me.
If that's what people thought I meant, I can certainly understand some of this thread better. But it floors me that my intent would have failed so completely to come across.
Interesting. I'd interpreted it as "we'll start small this time, and adjust things for next time, if there is a next time". Usually an experiment (in science) implies that you don't change the conditions of the experiment while it's running, you let it run and record the results, whether they're positive or negative. Otherwise, you limit your ability to be surprised, which is the whole point of science.
It gets a little more muddled with social experiments, where it's often hard to delineate an "ending point" of the experiment, and there's often real collateral damage in the process. Even with social experiments, though, you get a more accurate signal if you let them run past the point where your gut tells you it's a bad idea.
Surely the point of an experiment is to run it. Observe the results. Then change things in the next run.
> That's precisely what we changed, and yes we changed it at the last minute.
no rule of law -> automatic bullshit.
I don't agree with that at all, and taken literally it would make adaptation impossible, which can't be right.
HN has never been a rule-driven, legalistic kind of place, and isn't going to get that way. But I've had to learn a lot about people who do feel this way and obviously have a lot to learn yet. My own temperament is very far from this; it's almost impossible for me not to read "we'll figure it out as we go" as a good, fine thing.
You're missing the point others are trying to make. I'm all for experimentation - by all means. I experiment literally every single day, often failing, but occasionally succeeding.
The issue was that it wasn't clear there would be rule-changing or experimentation /during/ the experiment, rather than after the experiment.
4 replies →
If the "we'll figure it out as we go" is used just to reiterate the status quo, it's not a very sound or innovative approach.
About that, I like the point zellyn makes: https://news.ycombinator.com/item?id=11633818