Comment by cs702

5 years ago

Notably, of those bug reports, fewer than 1% (only 3 bugs) were specific to the Linux version of the game. That is, over 99% of the bugs reported by Linux gamers also affected players in other platforms. Moreover (quoting from the OP):

> The report quality [from Linux users] is stellar. I mean we have all seen bug reports like: “it crashes for me after a few hours”. Do you know what a developer can do with such a report? Feel sorry at best. You can’t really fix any bug unless you can replicate it, see it with your own eyes, peek inside and finally see that it’s fixed. And with bug reports from Linux players is just something else. You get all the software/os versions, all the logs, you get core dumps and you get replication steps. Sometimes I got with the player over discord and we quickly iterated a few versions with progressive fixes to isolate the problem. You just don’t get that kind of engagement from anyone else.

Indeed, although my kneejerk reaction to the subject line was to expect another sad article about how it's not worth it to support Linux, this author seems very happy with the decision to do so.

> Worth it? Oh, yes - at least for me. Not for the extra sales - although it’s nice. It’s worth it to get the massive feedback boost and free, hundred-people strong QA team on your side. An invaluable asset for an independent game studio.

  • Maybe there's a better term to throw at this. "Even though just 5.8% of userbase, Linux users contribute 55% of helpful QA feedback"

    • Makes total sense. You're unlikely to use Linux if you're not something of a power user, at least. I would guess there's a lot more developer type people who use Linux, and they are the people who appreciate a good bug report.

      Same people also have a better idea of what might be causing the bug, how to get more information, logs, and so on.

      1 reply →

    • I think the title was intentionally a little clickbaity but I'm not exactly bothered by it. Clickbaity is a negative when the bait-and-switch robs you of the pay-off. In this case, I think it was overall a more interesting story and so for me subverting my expectations was actually a pleasant experience.

      3 replies →

    • Reminds me of a tweet from Yann LeCun a while back where he bragged that they take down more reported content than other social sites. While at the face of it, this seems good, what it really means is that their system is worse at proactively taking stuff down before it gets reported. If they took everything down that violated their terms of service before it was noticed by someone else, they'd have a 0% follow through on these take down requests.

    • Multicapitalism might be the more academic term.

      A developer invested time to support linux not necessarily for financial return but social capital returns in the form of more helpful QA feedback.

      2 replies →

The worst part? I've seen developers and companies actually complain about this. Just 6% of sales yet look at the huge amount of work they cause us!

  • This is the case with people selling business software, meaning bugs would mean wrong accounting or worse outcomes for the customers. Yet all layers in the company would treat these reports as an annoyance.

Are there any good articles on writing helpful bug reports? Whenever I submit a bug I try to give as much detail and be as specific as possible but I always imagine some dev somewhere reading it rolling their eyes at the unnecessary paragraphs I’ve submitted.

I’ve often thought that game developers, esp indy studios, would benefit from doing first beta releases on Linux exclusively - wringing out a bunch of bugs, then doing a wider release.

>"we have all seen bug reports like: “it crashes for me after a few hours"

Here is a gem from my collection of user "bug reports". The email with this exact words and nothing else - "something is wrong, what do I do?". I was biting my fingers trying not to reply with the first thing that came into my head.

  • "We are sorry to read that. Please clearly describe your problem to us so we can help you. Feel free to call us on this phone number: XXX-XXX-XXX"

    People don't always understand our perspective, we should guide them into doing the right thing and everybody will be happy.

    Save your generic answer so you don't waste your time doing so, but being pedagogical should be part of our work in my opinion.

    • >"We are sorry to read that. Please clearly describe your problem to us so we can ..."

      What makes you think I do not do those things?

      1 reply →

  • Interpreted literally it’s a reasonable question: “What process should I follow when I encounter a problem?”

    • Well try to visit car repair shop and ask the same question as in the original email and then explain them what you really meant.

Conclusion: want good testing and bug reports of your game? Support linux.

  • Or have proper analytics. If a user emails you and says "It crashes after a few hours" rather than feeling bad for them, you ask them what their steam/game id is and you look them up in the crash reporter system and see exactly what happens rather than relying on a few programmer consumers doing the work for you.

    • But not analytics that are too good, or certain people will write a lot of inflammatory articles about how your software is nothing but evil spyware.

      Seriously, though, having a robust and automatic crash reporting system very much helps track down bugs you're never going to get a good report on or be able to directly reproduce.

      4 replies →

  • Possibly, but how many of the bugs are rare edge cases (i.e., attempting to break the game) versus problems players actually face in normal gameplay? Without knowing what priority or severity these bugs take, it is difficult to say how useful the increased volume of feedback is.

    • Even if someone is not attempting to break the game, they might accidentaly hit those rare edge cases.

      Of course, you could infer the usefulness of the increased volume of feedback from the fact that a game author is extremely happy about it.

      Overall, I don't see any particular problems with having reported issues. We definitely don't pay cloud storage per open issue or anything like that. Initial assessment of a bug report is not particularly difficult either.

      I'm not sure if your concerns translate to actual negative impact.

    • Depending on what kind of game you have, those exotic edge cases can come to be the dominant behavior in games, particularly multi-player where slight advantages can lead to some players dominating every one else.

    • I'm not a gamer, but from what I've read, the whole industry has been transformed by the incredible lengths people go to, to cheat.

      So I would expect that edge cases might be really relevant to mainstream use because everybody is trying to break the game.

    • FTA it sounds like basically all of them were normal gameplay that affected all platforms. only 3 were linux specific.

Yep, it’s a great point to raise in support of releasing software for Linux. Adept users who can produce high quality bug reports and help solve bugs across all platforms provide a lot of value, perhaps enough to justify “only” a ~5% market share.

Just from the title, I went in expecting that percentage to be much higher, but still expected I'd get a chance to complain how report quality was not taken into account (I expected Linux users to submit better reports)!

Pleasantly surprised on all accounts :D

Sometimes I forget my detailed bug reports with steps I took, even if I can't reproduce it--since it might provide a hint to someone with insight into the programs internals--, are a weird outlier.

This was my hunch, and I just wanted it to be true. I'm very pleased it turned out to be that way.

more likely s/linux/developers, which i suspect represents the majority of linux users

I am not sure why would one expect more linux bugs in a game made using a game engine (I couldn't find any info and I am assuming this game is one, correct me if I am wrong). If it is not working in linux, then it must be an engine bug, not a game one. If I am wrong then props to devs for having such a consistent game engine

Perhaps other option is using cpp, and having UB that behaves differently in different compilers. But I doubt that as well

  • OP was not saying that 38% of the bugs are Linux-specific, he was saying that Linux users report many more bugs per capita than windows users (and he sees that as a good thing).