Comment by DebtDeflation
2 years ago
Also:
>k=2 is a bit of an odd choice for kNN classification
That's an understatement. Choosing an even number for any sort of voting algorithm doesn't make much sense, choosing 2 specifically probably makes the least sense of all.
Yeah that is a big red flag - as the OP mentions, there is basically no way of making k=2 statistically different from k=1, that's why nobody uses it.
I suppose the authors just tried many different k and selected k=2 because it performed surprisingly well (likely due to the bug the OP found out). But if the results were significantly better than k=1 or k=3, it's a bit weird the authors never double checked why that was the case. I guess it can be one of those things you settle on early in the overall process with a few experiments, and just take for granted afterwards, never checking it again, but still, it sounds like something that should pop out at some point while writing the paper...?
Yeah, I think this is one part where a reviewer or advisor could have focused questions.
There is a sentence in Appendix C: "We set k = 2 for all the methods on all the datasets and we report the maximum possible accuracy getting from the experiments for each method."
I'm not sure what the second part of that means exactly.
True. You want to always use an odd number so there are no ties.
I'm guessing they were trying a parameter sweep, and found that (thanks to the bug) they got the best results for K=2.
This too is problematic in its own sense.
Yes, agreed. One small point: for the multi-class case (more than just two classes), which include all the datasets here, you can still get ties for odd k. e.g. k=3, you can get 1 vote each for 3 different classes, etc.
Multi-class is trickier. Maybe we can break down an N-class problem into N binary-classification problems?