Comment by csmpltn
5 years ago
Here's how these things usually go: first, you start small. You build something you can show other people. You try to generate some traction and build interest around your idea, validating it at a very small scale (friends, family, "Show HN", etc). You gradually expand that cycle of people. You get into a routine of iterating on improvements, adding new features, and gradually scaling things up. You probably get a couple of people to join you on that journey, because the more popular your product becomes - the more asks you'll be getting, and the more work it takes to scale it out. Rinse and repeat.
The reality is that we can't expect every single hobby project to support every single use-case from day one. Painfully for all - the current state of things in this space is such that building accessibility into your product is not trivial and hence it gets postponed "for later".
Now, on to a completely honest question: why don't we have screen readers that can consistently translate the "visual" version to a version people with visibility impairments can understand in a more generic manner, without requiring every single piece of software to adjust itself to every single flavor of a screen reader? I'm honestly asking why can't we solve the whole problem by offloading it to the screen reader itself. Can we solve this by simply offering a text-only command-line like version of every product? (ie. as opposed to building "beautiful"/"designed" experiences that get downsampled by the screen-reader anyways)? Sort of like building a sitemap.xml file that lets the user with a screen reader do everything a user with the full-blown GUI can. Sounds like an opportunity to create a cross-industry standard?
Edit: getting downvoted for comments like this is why I'm honestly considering just closing my account and not participating in any more such conversations going forward.
I upvoted you to combat your downvote because your initial paragraphs was a very reasonable description of the journey of a lot of hobby projects and its unreasonable to expect alpha apps (built using a GUI FX in active development) racing to get out an MVP is going to have accessibility nailed out of the gate.
Your last paragraph for a magic screen reader is that it simply doesn't exist, so in its absence you need to use a GUI FX that supports accessibility for it to be accessible. If all the screen reader can see is a rendered bitmap they're not going to be able to identify what's a UI control or how to interact with it, which group of pixels is decorative and which is functional or how it will be able to determine the difference between a real App and a screenshot of it? With the recent real-time AI powering "copy text from image" maybe a generic reader is going to be more advanced than what's historically been possible.
But I don't really know how screen readers work, I'm assuming they need to work in tandem with GUI FX's which is able to describe the purpose and roles of its different UI elements, so if you use native OS controls it's going to be able to know how to inspect different controls of running Apps and how to send events to them.
> If all the screen reader can see is a rendered bitmap they're not going to be able to identify what's a UI control or how to interact with it
This has been the status quo for decades of computing, but it seems quaint in this age of instant ML-driven image recognition. Anybody doing anything in the accessibility space with ML?
Apple: https://machinelearning.apple.com/research/creating-accessib...
I read your comment and agree. Without thinking it through, it seems that if every program provided a full CLI and a sitemap like file, then a screenreader should be able to integrate with said program.
I don't know much about the accessibility business, but my impression is that the screenreaders are all very expensive. Maybe providing a universal interface as described would level the playing field, maybe that's not desired (by the big brands).
> I don't know much about the accessibility business, but my impression is that the screenreaders are all very expensive. Maybe providing a universal interface as described would level the playing field, maybe that's not desired (by the big brands).
False. I have a print-related disability and I rely on screenreaders.
I use Orca on Ubuntu. However, the default voices for Orca are deplorable (by default eSpeak text to speech engine).
Most people do not know this, but https://oralux.org sells machine learning high quality voices (NeoSpeech engine) for $35 per voice at most which work directly with Orca. Here are the English text to speech voices: https://www.oralux.org/voice.php#english_american_english
This is an interesting idea for app-interoperability in general. It brings to mind Apple Automator, which I have no idea how works, but can sometimes be used to make Apps interoperate. It'd be pretty cool if every app did have a sitemap-like API spec that that mapped out the core of the app, and the UI sat on top of that.
A lot of the UI scripting on macOS is accomplished via accessibility APIs. If you’re interested, check out Accessibility Inspector[0], which ships with Xcode. You can inspect macOS applications in a similar, albeit much more limited way to a web browser’s interactive inspect element function.
You can also accomplish UI scripting via C/ObjC/Swift using the Application Services framework [1].
[0] https://developer.apple.com/library/archive/documentation/Ac... [1] https://developer.apple.com/documentation/applicationservice...
Exactly.
We should decouple the core of the application from the UI. Have a standardized interface which exposes all available functionality to the screen reader and allows it to talk to the core directly, bypassing the UI.
Stop doing what everybody does today: stop having screen reader depend on the UI (ie. aria-labels, etc). This problem can't be solved by adding "decorations" to existing UI components. It's best solved by exposing an entirely separate interface which lets screen readers (and other hardware) interact with the core of the application directly.
This sounds like a spec, a set of language frameworks and a design pattern waiting to be written.
If it's done right and standardized it'd be worth it for everybody since even people who don't need accessibility features could then create their own interfaces to the programs they use customized for their preferences and workflow
These APIs have existed for like three or more decades for desktop apps. That's partly why most desktop UI toolkits are based on tree of widgets: Mac and Windows's native accessibility APIs expect a tree
FWIW, the oldest accessibility API was Microsoft Active Accessibility (MSAA), which was added to Windows 95 in a service pack in 1997. Classic MacOS never had an accessibility API; the Outspoken screen reader for that OS had to rely exclusively on ugly hacks (e.g. patching Toolbox functions). OS X introduced accessibility APIs for Carbon and Cocoa in an early version; not sure which one. Unix-based desktop environments got an accessibility API, AT-SPI (thank you Sun), in the early 2000s.
1 reply →