Comment by cs702
5 years ago
Notably, of those bug reports, fewer than 1% (only 3 bugs) were specific to the Linux version of the game. That is, over 99% of the bugs reported by Linux gamers also affected players in other platforms. Moreover (quoting from the OP):
> The report quality [from Linux users] is stellar. I mean we have all seen bug reports like: “it crashes for me after a few hours”. Do you know what a developer can do with such a report? Feel sorry at best. You can’t really fix any bug unless you can replicate it, see it with your own eyes, peek inside and finally see that it’s fixed. And with bug reports from Linux players is just something else. You get all the software/os versions, all the logs, you get core dumps and you get replication steps. Sometimes I got with the player over discord and we quickly iterated a few versions with progressive fixes to isolate the problem. You just don’t get that kind of engagement from anyone else.
Indeed, although my kneejerk reaction to the subject line was to expect another sad article about how it's not worth it to support Linux, this author seems very happy with the decision to do so.
> Worth it? Oh, yes - at least for me. Not for the extra sales - although it’s nice. It’s worth it to get the massive feedback boost and free, hundred-people strong QA team on your side. An invaluable asset for an independent game studio.
Maybe there's a better term to throw at this. "Even though just 5.8% of userbase, Linux users contribute 55% of helpful QA feedback"
Makes total sense. You're unlikely to use Linux if you're not something of a power user, at least. I would guess there's a lot more developer type people who use Linux, and they are the people who appreciate a good bug report.
Same people also have a better idea of what might be causing the bug, how to get more information, logs, and so on.
1 reply →
I think the title was intentionally a little clickbaity but I'm not exactly bothered by it. Clickbaity is a negative when the bait-and-switch robs you of the pay-off. In this case, I think it was overall a more interesting story and so for me subverting my expectations was actually a pleasant experience.
3 replies →
Reminds me of a tweet from Yann LeCun a while back where he bragged that they take down more reported content than other social sites. While at the face of it, this seems good, what it really means is that their system is worse at proactively taking stuff down before it gets reported. If they took everything down that violated their terms of service before it was noticed by someone else, they'd have a 0% follow through on these take down requests.
Multicapitalism might be the more academic term.
A developer invested time to support linux not necessarily for financial return but social capital returns in the form of more helpful QA feedback.
2 replies →
The worst part? I've seen developers and companies actually complain about this. Just 6% of sales yet look at the huge amount of work they cause us!
This is the case with people selling business software, meaning bugs would mean wrong accounting or worse outcomes for the customers. Yet all layers in the company would treat these reports as an annoyance.
Huge amount of work they do for us
Marketing/PR depts never see it that way. They open up new tickets, and that's the metric they care about. Not justifying it, but marketing depts rarely get it right.
2 replies →
Are there any good articles on writing helpful bug reports? Whenever I submit a bug I try to give as much detail and be as specific as possible but I always imagine some dev somewhere reading it rolling their eyes at the unnecessary paragraphs I’ve submitted.
I can recommend https://www.chiark.greenend.org.uk/~sgtatham/bugs.html
It's interesting to compare that guide "How to report bugs effectively with ESR's "How to ask questions the smart way" [1]. They both cover similar material, but the styles are extremely different. The latter is rather hostile to the reader: "If ... then you are one of the idiots we are talking about." "If you decide to come to us for help, you don't want to be one of the losers." It's also heavy on "us vs them", how we are experts and you need to treat us properly.
You know, it just occurred to me that the "How to ask questions" document is targeted as much at hackers and how they should maintain their "standards" than at users who are asking questions. For example, the document has approving examples of "logically impeccable but dismissive" hacker answers; these make more sense as instructions on how a hacker should respond than as something relevant to a user.
I guess my point is that I had read the "How to ask questions" document for decades and viewed it as an objective document, not realizing how arrogant and "gatekeeping" it is.
[1] http://www.catb.org/~esr/faqs/smart-questions.html
8 replies →
Thank you! I’m definitely guilty of mongoose-ing and diagnosing in my bug reports so this is very helpful!
2 replies →
As a dev, I think
STEPS TO REPRODUCE Open foo click bar
EXPECTED RESULT saves bar
ACTUAL RESULT crashes with error x912344
MISC <any other logs, details here>
is the best. It gives just the right amount of detail to get started, in a good, actionable format.
Yeah, that was basically most of the bug template we were filling when I used to work QA at Warner Bros. Games.
3 replies →
Very important to also ask: Which application. Get an URL if at all possible. If someone says 'the calendar program is down', I have at least 5 candidates, and 50/50 chance something I've never seen pops up.
Users tend to rename programs, even give them names of other programs. Our entire finance department can't wrap their heads around the fact that 'Oracle' is not a bookkeeping application. Worse, they are now convinced the new version is called 'fusion'. Oracle financials? Never heard of.
Use this template:
1. When I do this
2. Here is what I expect to happen
3. But instead, this happens
One more tidbit: In the "When I do this", I find it useful to omit stuff that isn't interesting. Does the thing fail if I do something immediately? Or does it only occur if the application has been dormant for a few minutes? Does it only occur using the mouse, or can I use the keyboard?
Get a good set of steps for reproducing the problem, but wander a little off those steps to see what is actually necessary. I've been surprised more than once when something that I thought was incredibly complex only required a few steps. And also been surprised when a complex repro was actually complex (e.g., if I drag quickly it works, but if I drag slowly it doesn't? what?).
1 reply →
...4. And this is my software version and Linux distro.
1 reply →
Ugh no thanks, I'll just send them a screenshot and say "it's broken, please fix this!"
If it's good enough for my colleagues to submit tickets like this, it's good enough for me!
From "Post-surgical deaths in Scotland drop by a third, attributed to a checklist" > GitHub and GitLab support task checklists in Markdown and also project boards [...]
> GitHub and GitLab support (multiple) Issue and Pull Request templates:
> Default: /.github/ISSUE_TEMPLATE.md || Configure in web interface
> /.github/ISSUE_TEMPLATE/Name.md || /.gitlab/issue_templates/Name.md
> Default: /.github/PULL_REQUEST_TEMPLATE.md || Configure in web interface
> /.github/PULL_REQUEST_TEMPLATE/Name.md || /.gitlab/merge_request_templates/Name.md
> There are template templates in awesome-github-templates [1] and checklist template templates in github-issue-templates [2].
> [1] https://github.com/devspace/awesome-github-templates
> [2]ognarb
5 years ago
skeeter2020
5 years ago
edflsafoiewq
5 years ago
dylan604
5 years ago
WalterBright
5 years ago
blackearl
5 years ago
https://community.kde.org/Get_Involved/Issue_Reporting This one is pretty good
bug reports and communication in general: If you've got a good summary/abstract more detail is always better. I don't know if I've ever had to talk to someone on my team about how they "over communicated" (not the same as TMI!). Make the receiver responsible for saying "OK, that's enough, got it!"
Conversely I think you should generally write the smallest amount of text you can. It maximizes the chance someone else will actually read it.
2 replies →
Think about what you would want to know if someone was reporting the bug to you. Provide at least that much information.
"Just the facts, Ma'am"
https://www.youtube.com/watch?v=v4LPkmGO5Cc
Usually sending just "help" and no other data is preferred. Don't send screenshots, don't make use of the ingame reporting tools (pressing F11), don't send steps to replicate.
I’ve often thought that game developers, esp indy studios, would benefit from doing first beta releases on Linux exclusively - wringing out a bunch of bugs, then doing a wider release.
Well that might be one way to _accelerate_ your dev process.
maybe alpha, beta level you gotta be on all platforms.
>"we have all seen bug reports like: “it crashes for me after a few hours"
Here is a gem from my collection of user "bug reports". The email with this exact words and nothing else - "something is wrong, what do I do?". I was biting my fingers trying not to reply with the first thing that came into my head.
"We are sorry to read that. Please clearly describe your problem to us so we can help you. Feel free to call us on this phone number: XXX-XXX-XXX"
People don't always understand our perspective, we should guide them into doing the right thing and everybody will be happy.
Save your generic answer so you don't waste your time doing so, but being pedagogical should be part of our work in my opinion.
>"We are sorry to read that. Please clearly describe your problem to us so we can ..."
What makes you think I do not do those things?
1 reply →
Interpreted literally it’s a reasonable question: “What process should I follow when I encounter a problem?”
Well try to visit car repair shop and ask the same question as in the original email and then explain them what you really meant.
I don't hold myself and frankly answer that my crystal ball is on scheduled maintenance today.
So when will your crystal ball be ready, could you then asap adress the issue? Also keeping spare crystal balls would be really helpful.
1 reply →
Conclusion: want good testing and bug reports of your game? Support linux.
Or have proper analytics. If a user emails you and says "It crashes after a few hours" rather than feeling bad for them, you ask them what their steam/game id is and you look them up in the crash reporter system and see exactly what happens rather than relying on a few programmer consumers doing the work for you.
But not analytics that are too good, or certain people will write a lot of inflammatory articles about how your software is nothing but evil spyware.
Seriously, though, having a robust and automatic crash reporting system very much helps track down bugs you're never going to get a good report on or be able to directly reproduce.
4 replies →
Possibly, but how many of the bugs are rare edge cases (i.e., attempting to break the game) versus problems players actually face in normal gameplay? Without knowing what priority or severity these bugs take, it is difficult to say how useful the increased volume of feedback is.
Even if someone is not attempting to break the game, they might accidentaly hit those rare edge cases.
Of course, you could infer the usefulness of the increased volume of feedback from the fact that a game author is extremely happy about it.
Overall, I don't see any particular problems with having reported issues. We definitely don't pay cloud storage per open issue or anything like that. Initial assessment of a bug report is not particularly difficult either.
I'm not sure if your concerns translate to actual negative impact.
Depending on what kind of game you have, those exotic edge cases can come to be the dominant behavior in games, particularly multi-player where slight advantages can lead to some players dominating every one else.
I'm not a gamer, but from what I've read, the whole industry has been transformed by the incredible lengths people go to, to cheat.
So I would expect that edge cases might be really relevant to mainstream use because everybody is trying to break the game.
FTA it sounds like basically all of them were normal gameplay that affected all platforms. only 3 were linux specific.
Yep, it’s a great point to raise in support of releasing software for Linux. Adept users who can produce high quality bug reports and help solve bugs across all platforms provide a lot of value, perhaps enough to justify “only” a ~5% market share.
Just from the title, I went in expecting that percentage to be much higher, but still expected I'd get a chance to complain how report quality was not taken into account (I expected Linux users to submit better reports)!
Pleasantly surprised on all accounts :D
Sometimes I forget my detailed bug reports with steps I took, even if I can't reproduce it--since it might provide a hint to someone with insight into the programs internals--, are a weird outlier.
This was my hunch, and I just wanted it to be true. I'm very pleased it turned out to be that way.
more likely s/linux/developers, which i suspect represents the majority of linux users
God damnit... Clickbating is ruining any shred of useful information...
I am not sure why would one expect more linux bugs in a game made using a game engine (I couldn't find any info and I am assuming this game is one, correct me if I am wrong). If it is not working in linux, then it must be an engine bug, not a game one. If I am wrong then props to devs for having such a consistent game engine
Perhaps other option is using cpp, and having UB that behaves differently in different compilers. But I doubt that as well
OP was not saying that 38% of the bugs are Linux-specific, he was saying that Linux users report many more bugs per capita than windows users (and he sees that as a good thing).