Comment by open-paren
2 years ago
I manage a medium-sized browser extension at work. We also offer(ed) it on Firefox. But I have spent the past year struggling to get back into Mozilla store after a manual review. As far as I can tell, there are maybe two reviewers that are based in Europe (Romania?). The turn around time is long when I am in the US, and it has been rife with this same kind of "simple mistake" that takes 2 weeks to resolve. "You need a privacy policy"–we already have one. "You are using machine generated and minified code"–no you are looking at the built code, not the included source. "We cannot reproduce your source"-that's because you didn't follow instructions and are in the wrong directory. Very frustrating.
Similar boat. I release an extension with about 1 million installs across Chrome/Firefox/Edge for work.
Firefox (despite being the smallest usage) is utterly insane with regards to process. They demand a reproducible build, but then can't do things like install the right version of yarn (no - npm install -g yarn is not correct, our readme says it in bold like 5 times and provides the exact correct command to install the right version), or follow basic setup steps like "Use this version of node (complete with exact steps to install it and a script to automate that for them)".
God fucking help you if you try to do something completely crazy as a private company like - checks notes - use a private NPM module. Despite providing them with access on a pre-configured account, or offering to give a review account access according to Mozilla "It's too hard to use external accounts during review".
Honestly - having to interact with the browser review team is a BIG reason I no longer recommend Firefox. They're incompetent at best, and I'm fairly convinced they're just milking the google search deal income for as a much as it's worth - I don't think they really want to provide an alternative and secure browser anymore.
On the flip side, having to interact with addon review has raised by confidence in the browser. The steps they take to review, while not perfect, seem like they could weed out a lot of potential garbage and malware. I was expecting a much more minimal review process, which would have raised my fear about the extensions I use and set to auto-update.
Reproducible builds and open source sounds like a good thing.
I wouldn’t expect the reviewers to deal with every add-ons bespoke snowflake build. Even less so if it requires access to a private module. Mozilla should provide a baseline of how a build is intended to be done, then extensions just have to follow this template. Though yes, you would expect them to have some familiarity with basic stuff like yarn and that the baseline supports a few of the most popular builders.
We use a relatively simple build. at the base of it, if you have node and npm, a complete build is as easy as
yarn npm login
yarn --immutable
yarn build
Personally - I don't really find it reasonable to place demands on build tooling for an external company.
I'm assuming you would also find it reasonable for Google to suddenly ship chromium with a requirement that you use "google-pack" for all js builds or they don't run it?
To be entirely blunt, what exactly do you think is going to change when we're already giving them bare JS? It's not like we're shipping a binary blob here, we're literally handing them a zip file with perfectly fine & inspectable javascript inside it.
Further, do you realistically believe that a single low grade QA/Support engineer who can't even install the correct tooling is going to catch malware?
Because I read their matrix chats and I can fucking promise they aren't catching the malware all that fast....
3 replies →
It seems reasonable that they'd have a requirement there's a single file they'll run, maybe even with a predetermined name like ./build, and that's it.
The developer can then juggle all their dependencies and run make/yarn/npm/etc within that. It's really not different from having a CI build script.
1 reply →
> I'm fairly convinced they're just milking the google search deal income for as a much as it's worth
That's exactly what the ex-McKinsey C-suite are doing. Regular employee talent suffers because of it, as you've found.
This is exactly what the review process for the Play Store is like, even worse for Google TV apps. Often times just re-submitting multiple times without changing anything at all will get it pushed through.
Yeah but despite how much HN hates Google everyone here will do whatever it takes to get on their app store. Google has the power to make the entire industry their bitch.
Mozilla not so much.
2 replies →
This sounds super frustrating, as someone who has an idea or two for browser extensions I'm not looking forward to all the bureaucracy. I actually love the idea of requiring and validating reproducible builds but they really should invest in reviewers competent enough to manage that.
I do have half an idea to deal with it that I plan to try, thought it might be helpful to suggest: implement a Fisher-Price build system that checks and automates every single step and cannot go wrong. Ideally if the reviewers can run Docker, do it all in a container. Wrap package.json scripts with functions to validate the build environment before proceeding and either fix it automatically or fail and print clear instructions to the console. A preinstall hook could verify they have proper NPM auth and prompt for it if needed.
Annoying to have to do that at all though. I'm starting to come to similar conclusions on Firefox, using it currently but I've been thinking about jumping ship for a while. What browser would you recommend now? I wanted to get away from Google but I'm considering just Chromium since any remotely comparable options I've found are poorly thought out wrappers of it.
> ... it might be helpful to suggest: implement a Fisher-Price build system that checks and automates every single step and cannot go wrong...
Programming is a race between the programmers, trying to build better, idiot-proof software, and the Universe, trying to build better idiots. Do not underestimate the Universe.
1 reply →
They should switch to an fdroid like model that does public builds on cloud infra.
It sounds like they are doing their job attempting to review random code from strangers to be honest.
Honestly I have to side with Mozilla team here. Kudos to them for trying to actually care about security and privacy. I can imagine the nightmare that people are submitting and trying to recheck everything and build those random extensions with private npm repos and whatnot.
It’s funny to think of Mozilla like landed gentry where they have captured serfs (their users) and get a payout from the king (Google) for their loyalty and support.
> We cannot reproduce your source
This is the biggest issue we had, and we had to add a decent bit of complexity to our builds to support reproducible builds in the exact way they want. But the silly part is that our extension involves building a wasm file from Rust, and after some back and forth it turned out that they don't require it to be reproducible (despite being core of our extension and containing 99% of our logic), which honestly feels like it defeats the point - who cares if JS reproduces if you can hide any arbitrary possibly-malicious code in wasm.
For a while we were seriously considering putting our prebuilt wasm in the source package or on npm, just to make the "reproducible build" on AMO side simpler, despite this making it even further from how it's actually built.
What kind of harmful code could you put in WASM? You could return a string that you eval on the javascript side, so the reviewers could possibly ask for the WASM source if they saw the eval, but other than that the purpose of WASM is to be a safe sandbox after all, right?
I'm not familiar with the security guarantees of WASM in the browser but I imagine they're more along the lines of preventing data exfiltration from the browser/OS, it would be difficult to prevent something like abusing your CPU resources to mine Bitcoin in the background for example.
1 reply →
Every time I hear about the review processes for browser extensions I'm shocked that the it involves humans having to read your README and manually plumb together the build process. Sometimes I hear that reviewers are even reusing VMs when doing reviews, or even not using VMs at all. I'd have expected the review form to have a textbox where you paste your git link and a well-documented automated pipeline that stands up a specified VM with a specified amount of RAM and disk, clones the git, descends into it, and executes `docker build -t ./docker/review/Dockerfile`. I'm surprised that the reviewers themselves haven't outright demanded such tooling from their larger organization, just as a matter of job satisfaction - I can't imagine all the abuse they get from angry app owners.
Browser extensions really seem like they're slowly failing and just not supported. Kinda like PWAs.
I want to write a chat program, but it has to work on phones, and the DevEx for native phone frameworks compared to desktop apps looks like hell, and PWAs seem to be barely supported.
It's easier than ever to make a CLI or desktop app, but phones seem like the worst of all Microsoft dev history - Learn these arcane lifecycle vocab words that make no sense, like using Win32 directly, but also it changes every year or two like when MS invents a new GUI framework, but also if you can't get into The Store, nobody but your power user friends will be able to run your app anyway. What is this shit?
Someone will come up with a solution that is utterly ingenious. Like the ability to install a plugin without third party intervention with a single click.
>Browser extensions really seem like they're slowly failing and just not supported. Kinda like PWAs.
Ya, totally!
lmao
ublock origin has 8mil users on Firefox alone.
3 replies →
Also had these issues when working on my previous job's extension. The Firefox review process was a real nightmare to work with. Same heavy delays and misunderstandings your mentioned. Eventually the company just stopped updating the Firefox extension as often since usage was low and the review process was such a pain. Unfortunate for me, as the only engineer (maybe employee) at that company that used Firefox.
Same here. We even had a special "mini" Firefox version that didn't require any additional Javascript build step, to make the review easy. But there were so many issues with the review and so few users that we just decided to give up.
The whole extension change Mozilla forced on Firefox seems like some sort of sabotage.
Mozilla sneaking in more and more spyware and ad friendly functionality seems in line with the same conspirators.
And given how high profile all these changes are, it runs to the top of the company.
So, which browser are you using?
1 reply →
The problem with these types of things is that the people who are qualified to do good reviews are also the sort of people who can typically get a far more interesting job building stuff, rather than just reviewing code. It's work that does require a certain level of skill, but at the same time is also quite boring.
And that more interesting job will probably pay better as well.
Id rather hire a senior dev as a reviewer and a mid dev as the coder at a company. pay the reviewer more since they will be dealing with shit practices and having to train the dev.
Not only that, but properly reviewing code would take forever. Heck I don't know how many senior engineers at my fancy tech company could do it and reliably spot problems.
I think that is one way that “tragedy of commons”.
> that's because you didn't follow instructions and are in the wrong directory.
You just need to have a shell script in the root directory that assumes the person running it has 0 clue about your extension.
Also some of this reminds me of Apple. They clear something up, then bring it up again the next time review is needed.
Even this we had issues with - we wrapped the entire build environment and script in a dockerfile, but depending on system configuration you may or may not have to run docker with sudo - it just so happened that reviewer's environment required it, while ours didn't, and the reviewer needed specific instructions on what to do in this case.
Another time, they failed the review because the reviewer's VM _ran out of disk space_ (which we only learned after digging into the issue, as the first report just mentioned "build errors"; according to later inquiries the VM had ~9GB available) and we had to add some extra build logic to delete intermediate files, just for them. The build is quite large because it involves rust->wasm compilation, but I'd still expect the reviewer's machine to have a bit more space...
Simeon from the Firefox Add-ons team here. Sorry about the rocky experience. I realize this is a bit late for your situation, but earlier this year the source code submission docs were updated with information about the default reviewer build environment[1].
It's not a huge improvement, but it sounds like one thing we could do to improve the communications process around build errors is to include a link to this documentation in the notification email sent to developers. I'll create a ticket for this now.
[1]: https://extensionworkshop.com/documentation/publish/source-c...
Everything described here sounds like your team, your extension, and your software development process are the problem. Demanding >9GB of disk space to build a browser extension is capital F, capital I Fucking Insane. Go yell at the Rust folks about their shitty toolchain and your engineering lead for buying into it instead of blaming people who have enough problems as it is just coming into contact with the quagmire you described.
5 replies →
> that assumes the person running it has 0 clue about your extension.
I would tend to assume that a person given responsibility for reviewing this software, supposedly to protect end users, would not be this clueless.
What value is the "Firefox Store" actually offering then?
> What value is the "Firefox Store" actually offering then?
That anyone dumber than such a reviewer cannot sneak malicious extensions in.
Which, sadly, is probably a non-trivial number of submissions.
5 replies →
> I would tend to assume that a person given responsibility for reviewing this software, supposedly to protect end users, would not be this clueless.
would you do that job 8+ hours a day for little pay?
1 reply →
That's not just mozilla. Google's review team all are in India and they cannot write clear English. It's a mess.
>Google's review team all are in India and they cannot write clear English.
Which is ironic considering the reason they went to India and not other countries with cheap labor is that English is an official language there.
The problem is that the set of "Indians who can speak fluent English" and the set of "Indians who will work for the absolute lowest bid" are exclusive. And I don't blame them, really.
These execs mistake "English is an official language" for "English is a widespread first language". Only 0.02% of Indians speak English as their first language, while total speakers (of first, second, or third language) are 10.6% of the population.[0]
[0] https://en.wikipedia.org/wiki/Languages_of_India#Multilingua...
7 replies →
Also Apple and Meta. It's awful dealing with infallible gatekeepers.
Why make such a generalized statement?
I had these issues too a few years ago. Now the review time is shorter than Chrome’s and hasn’t been flagged in a few years. However my extension has about 10k users, if that makes any difference.
this seems like the kind of place where user-based reviews would be more efficient, better, and more open
having the makers of a browser do this is bound to create both efficiency and political problems for extensions. im remembering dissenter now
That's interesting to hear. Do you also offer your extension on the chrome store? How did the review process differ? I ask because I've only published on the chrome store in the past.
[flagged]
[dead]
The reproducible build requirement seems to be a major blocker for many addons, including one I use for Twitch: https://github.com/FrankerFaceZ/FrankerFaceZ/issues/1495#iss...