Apple rejected my dictation app for using the accessibility API

3 hours ago (mitmllc.com)

I actually had the almost same situation by building an offline voice dictation app for macOS and iOS, and in macOS I was confronted with the exact same situation.

However, I would like to point out that Apple isn't totally wrong here because the accessibility API unfortunately is way too broadly scoped, and because of that you literally get access to everything on the computer like you you can screenshot listen and and move the cursor... This is completely ridiculous and the proper engineering solution would actually be to phase out the accessibility API and replace it with something that is narrowly scoped so you can grant specific permissions individually.

However, Apple, being Apple, is obviously not doing anything, and instead says no accessibility permission for anything that isn't demonstrable accessible. Now, there are obviously some exceptions because Apple is not particularly well known for applying its rule consistently and granting big exceptions for itself. However, they do have a valid point on privacy and data protection. And I say that as somebody who ended up distributing my MacOS app outside the App Store because I only got approval for iOS.

That said, I would definitely appreciate if Apple would gradually improve its developer program experience, because compared to its hardware lineup, the developer program is nothing short of abysmal.

SpaceGremlin (mac alternative to WinDirStat) has a similar thing, where some features only work in the independent "SpaceGremlinPro" version downloaded from their site. However, they do some cool stuff with licensing - you can point it to the app store paid/installed version, and it detects the license and unlocks.

If you're worried about people not trusting payment to you, might be worth seeing if you could implement this, so anyone who bought on the app store can still access the full feature set. Cuts you out 30% like, but better than nothing maybe.

  • > If you're worried about people not trusting payment to you, might be worth seeing if you could implement this, so anyone who bought on the app store can still access the full feature set. Cuts you out 30% like, but better than nothing maybe.

    In other words, Apple is abusing their position by defining overly broad permissions so that they can deny them and pressure people to fork over more cash to them.

  • There is something amusing about the fact that WinDirStat, as far as I know, was based on KDirStat (now QDirStat), yet this doesn't even get mentioned on their Wikipedia page, and by and large a lot of people don't even know QDirStat exists. One time someone even asked me if they knew of a good alternative for Linux; good news!

  • Interesting idea. It would basically turn the App Store version into both a discoverability channel and a license anchor for the direct version

  • Space Gremlin isn't even available on the App Store anymore, presumably because it hasn't been updated to newer versions of macOS. Meanwhile, GrandPerspective is free and uses the exact same visualization as WinDirStat (although the UX is a bit weird for me)

I don't mean to offend, but entrusting every input to a company literally called MITM LLC has a level of absurdity that either greatly entertains or else greatly frightens me.

I recently built a similar app, and so hit the same limitations – I wasn't too upset on Mac, happy to distribute without the App Store (though it's a shame).

Where I was more frustrated was how much this limited the potential usability of the iPhone app. Because of app store restrictions it is a far worse app ... though like in your example, still useful to a degree.

I can only hope they use the new CEO as an opportunity to seriously re-evaluate their entire approach to how they work with developers, though I'm not actually expecting them to. If anything, with the increase in apps being created via AI tools I worry they will go the other way.

  • I really do understand the desires people have for iOS to be a more open platform, but I'm just gonna say very clearly: I do not want third party apps being able to do what OP's app does. My iPhone is the one computing platform I have where I get the assurance that no third party app can be spying on anything else I do on the device.

    • Yea, Accessibility features are kind of OS super-powers and you really, REALLY need to thoroughly vet apps that you grant those powers to. These apps need to be actually using Accessibility to provide assistive technology for users with disabilities. I'm usually uneasy about Apple anointing itself the gatekeeper for this, but someone has to do it.

      Lots of shady and well-known developers (like Dropbox) are notorious for trying to weasel their way into getting Accessibility permissions, so they can do god knows what with them to your system.

    • iOS generally lets you reject any permission an app asks for. This would certainly be "risky" enough that iOS would require explicit user permission, and you would be able to say no.

      On top of that, the app is completely optional: if you aren't comfortable giving it those permissions, don't install it?

    • The Accessibility permission is not granted automatically to apps on the Mac. You have to specifically allow it for an app. So you retain control and assurance even without Apple lockdown.

  • Exactly. On iOS, it completely limits the market for a good dictation app with your keyboard, because iOS just doesn't allow you to.

  • If you're in the EU, consider publishing on an alternative App Store and pointing users that way.

    If you're not, ask your representatives why you don't get the same rights.

There's a reason I don't write mobile apps, and it's all the flaming hoops you have to jump through: both in the build system and from the random whims of reviewers.

As someone who also experience pains in their hands after a couple of hours of typing... I started to use the great open source app called ghost-pepper [1] that i found on github and has been my daily driver (its like superwhisrp but oss/free and local) the maintainer is really nice and replies to DMs really quickly too.

[1] https://github.com/matthartman/ghost-pepper

  • As someone that have tried a few of these apps, I really like this one. I dictated this one just now with ghost pepper. Thanks Matt and thanks orliesaurus for sharing it here!

I’ve had lots of inconsistent app reviews from Apple. Just appeal and/or re-word your language and you’ll be ok. Plan on it taking a few weeks to fully sort out.

This is what happens when you run an OS controlled by some random big corporation. I dont mean that it's the person's fault, but just that you should not rely on Apple. they allow you to use your computer, but on their terms.

Install some GNU/Linux distro and you can do whatever you want.

  • for most people this is like saying "If you don't like being oppressed, just move to Antarctica!"

    • Maybe more like “Learn how to replace an AC filter by yourself instead of calling an AC repair company”

      I just installed PopOS on a laptop recently, and… it just worked. There’s an app store for noobs that I think installs flatpaks. GPU drivers just work. Whole disk encryption. Everything just works.

      I don’t see what else my grandma that just uses Facebook would need. Maybe automatic updates?

      17 replies →

    • > If you don't like being oppressed, just move to Antarctica

      No - moving to far away areas is not the right analogy. After all you need to have use cases where those huge companies do not control your business. So the alternative is to avoid becoming dependent on them; or cut off the dependency when possible.

  • Unless... you have a personal or professional need to use apps that don't work on Linux.

    I tried very hard to switch to Linux full time some months ago, but I couldn't find a way of getting Microsoft Office to work satisfactorily. There are clever packaged versions of Outlook and Teams, but I need full native installed versions of Word/Excel/Powerpoint, and there just wasn't a good solution. That was a deal breaker, sadly, so I'm back on Mac for the time being.

    Other examples would be some of the popular games with anti-cheat that requires Windows.

  • Fair. I run Nobara on my gaming computer and built a similar dictation tool there with no API restrictions, so the trade-off is real. For this project I chose both: App Store reach for the compliant version, direct distribution for the full one. But I know other people wouldnt be comfortable with running something like that so I built this somewhere my mom could use it

  • You're missing the point: it isn't about the OS. The direct distribution version of the app has full functionality. The problem is with the Mac App Store.

  • >This is what happens when you run an OS controlled by some random big corporation

    You get a channel for installing apps, where someone vetoes random apps that want to have access to control your whole computer and potentially steal sensitive data?

    >Install some GNU/Linux distro and you can do whatever you want.

    And any random app can get total control and steal your data, unless you know how to enable restrictions. I'd rather have restrictions as the default, and for the most naive users who'd follow every app prompt, and then cry about their lost work/private documents/money, no way to bypass them.

    • It's not true that any app can get total control of your system. If you install them via flatpak, the apps are sandboxed. Also, unless you log in as root, the apps can't do much. Wonder why the most important systems in the world and big tech's servers run GNU/Linux? There's a reason

      I dont wanna start a war over this btw, even though it may not seem :)

      1 reply →

    • > I'd rather have restrictions as the default

      Then don't install apps and use the web, mobile sandboxing is much weaker compared to any modern browser.

  • Apple is hardly a random big company. Apple's customers specifically chose to purchase the product. Most of their customers don't realize the significance of the exposure to copy and paste between Apps. Apple has taken the position that monitoring this exposure is part of their duty to the customer. Anyone that is aware of this shortcoming in Apple's product is free to purchase a different device.

The frustrating part is less that Apple has a boundary here, and more that the boundary seems opaque and inconsistently enforced

In Apple’s defense, your company name is MITM. Man In The Middle certainly falls on one side of the perception line, don’t you think?

  • Right?!

    I get that some people are unfairly targeted but some other times it's people being (extremely) naive or just playing dumb

    "Hey you know what would be cool? If we named our bluetooth speaker company bee oh emm bee!!11"

  • The acronym is unfortunate, you're not wrong. MITM here is "Moogle In The Machine" (the Final Fantasy moogle + machine learning), but the security-context joke is fair and I hear it constantly.

Some non apple apps get access to accessibility APIs. What gives?

This API is sensitive. I imagine Apple is particularly stringent as to how the access is justified. Not how it uses it but how the reason for using it is explained.

It's not like someone tests the app and all api calls to deem them reasonable or not.

  • They do literally pay people to do that. Then one of those people chose to reject this anyway.

What API are you using? I have a sandboxed app on the Mac Store that synthesizes CGEvents to simulate arbitrary keyboard actions on behalf of the user. It needs accessibility permission, of course.

  • Same approach: CGEventPost with Accessibility permission. The wrinkle was that my App Store reviewer wasn't comfortable with how I was using AX permission for auto-paste, even though the mechanism is the same as other apps already in the store. The clipboard-only version of WhisperPad needs no AX permission and that's what got through. Interesting that your sandboxed app with similar mechanics is approved.

  • Wondering the same, there is some weirdness around the clipboard and CGEvents though. Are you avoiding the clipboard entirely in your implementation?

Quick question, I assume you're getting caught by the CGEvent(PostEvent)...but I want to be sure. AX API has been gimped for over a decade so you'd have never made it into the app store that way. Just making certain, in case you have another path. It doesn't appear CGEvent is a universal approval anymore either though.

Have fought similar demons lately, feel your pain.

  • The direct version uses CGEventPost to synthesize the paste, which requires Accessibility permission. The App Store version writes to the clipboard only, so no AX permission needed and the user presses Cmd+V manually. The 2.4.5 rejection was specifically about the Accessibility permission use case. Your read sounds right that this path has been gimped for a long time.

Doesn't Wispr Flow do this though? How did they get past these limitations?

  • From what I understand Wispr Flow distributes directly from their website and doesn't ship through the Mac App Store, so they don't go through Apple's App Store review at all. They use the Accessibility API the same way the direct version of WhisperPad does. The 2.4.5 limitation really only kicks in if you want App Store presence.

I don't want random apps to paste potentially dangerous things into other apps. Its understandable.

Imagine a banking app, and for example an IBAN field.

  • Them you are free to not install them? Why ban them outright?

    I'm using https://github.com/cjpais/Handy whichseems to be doing exactly what this app does, and has a very similar background story (author couldn't type die to injury).

    • In this case it feels like it's a feature that the operating system should be providing or something that could be marked as an accessibility tool, which would allow it to use that API.

      The problem from Apples perspective could be that there is a ton of tools that require access to the accessibility API because they want to do stuff that Apple have deemed a security risk and the only way to do it is by abusing the API. Some of these are also because macOS simply lacks certain APIs.

      I think Apple overreacting due to previous API misuse by other apps.

      1 reply →

  • I see, that's a really fair point. And I can understand that banking field example. So I can see why they're guarding against it. My disagreement was less with the rule itself and whether Whisperpad's specific use case for users with mobility needs falls on the right side of it.

  • Pasting doesn't seem very unsafe. Especially not when the app can't know what it's pasting into.

No surprises here, Google has also been restricting access to its accessibility API.

  • Useful context, thanks. I hadn't realized Google was tightening similarly. Would be interesting to see how the rationales compare.

Oof, thats rough. I'll still start facing those issues, just got accepted into the apple's dev program. I predict a ton of rejections coming my way.

macOS already has a dictation feature that does this exact thing, albeit in real time. I use it extensively.

OP’s description in the linked article doesn’t say much more than this, so what am I missing with this particular app?

  • Apple's own dictation is quite limited, doesn't handle multiple languages very well, and many open source dictation models simply do better.

  • Apple's built-in dictation works for casual use, but in my own daily use the typo rate was high enough that I was constantly going back to fix things, which defeated the point (with a hand injury, those corrections cost me). WhisperPad uses Whisper models instead, doesn't cut off after 30-60 seconds like Apple's does, supports 99 languages offline, and pastes into any active field via hotkey. There's a 120-minute monthly free tier so you can see if it fits your use case. If Apple's built-in dictation handles what you need, that's a fair answer.

A company called “MITM LLC” which hijacks pastes in other apps.

I have no idea what they’re thinking. Insanity.

Eh. I think it’s fair if Apple doesn’t want to publish something on their app store.

I just wish they weren’t so obstinate about people installing from other sources without signing/notarization. I understand it from a security standpoint but it’s also nakedly self-serving.

I’m glad that they’re fine with signing in this case.

  • Fair points. The notarization-but-not-App-Store path was actually a workable middle ground in my case. Apple still gates security via notarization, but doesn't gatekeep the use case. The warnings users see when installing non-App-Store apps could be lighter without compromising security.

Accessibility things should be more useful than to just narrow accessibility uses only. Wheelchair ramps help move heavy objects. The accessibility API makes it possible to introspect all of the keyboard shortcuts an app provides for another app to list them.

Screw Apple and their persnickety, controlling myopia.

Easy, don't make apps for devices which are only leased to people.

Make apps for device, which are 100% owned by people.

This is another reason why one shouldn't become dependent on those giant companies. Just as Microsoft recently stated, you'll have to pay for GitHub CoPilot soon on a token basis. Apple controls access to its software ecosystem too.

Time to turn Linux into a platform where you can upload into a store whatever the fuck you want. And see these behemots burn.

I guess this app can still be installed locally? It's just that it can't be distributed to others due to signing requirements?

Edit: Ah, it's in the article, this is about AppStore distribution. Walled gardens are going to walled garden.

  • The direct version is fully signed and notarized by Apple, just not distributed through the App Store. Anyone can install it from mitmllc.com/whisperpad without workarounds. The 2.4.5 rejection was an App Store rule, not a general restriction on the app.

I am still not certain I understand exactly what Apple's reviewer meant by 2.4.5 in my case. My working assumption is that the concern is about an app reaching into every other app on the system to inject text, but I never got a perfectly clear explanation. (Or maybe I'm too dense to understand it.)

If anyone here has more direct experience with this guideline, especially from the App Store review side, I would like to hear it. I would rather understand the policy than just guess at it.

Add it to the antitrust pile.

Microsoft was almost broken up over not allowing third party programs to use certain APIs. Apple abuses their dominant position to suppress competition.