Comment by nimih

I agree that Elo falls rather short for multiplayer games (the article's approach probably converges much more slowly, or fails to converge, than an approach which is built around supporting multiplayer contests, and the simplification for "board zaps" is likely just plain wrong--although that might be a limitation of how they recorded their games), but I don't think individual MTG games having a substantial amount of luck should really impact the usefulness of Elo (or similar systems such as Glicko). After all, Elo is just trying to find ratings which best predict a given game outcome, so the presence of good/bad draws should still be well-modeled by that idea, and in particular, for two given players (at a particular point in time and holding particular decks[0]), it stands to reason that you should be able to still find some pair of ratings Rx and Ry s.t. P(x beats y) = 1/(1+10^(Rx-Ry)/400).

That being said, the inherent randomness of MTG maybe means that in an ill-defined, abstract sense, it takes "more skill" to improve 100 Elo points in MTG than in Chess, because X% of your games have no meaningful decisions so you have fewer places to take advantage of your superior decision-making and, further, this probably has real implications for reasonable choices of K if you're running, say, MTG Arena, but the article is pretty clear that they're not doing anything especially rigorous when picking K in the first place, and honestly (IMO) it probably doesn't matter a whole lot if you're running a Friday night beer league with some friends or whatever.

[0] I agree with the sibbling comment that deck selection and deckbuilding is a large part of what magic players mean when they discuss skill, and it seems very reasonable to allow those things to be included in our model.