Comment by kristopolous

I'm on the other end. I use a cheap midi device to do volume control via knobs and a slider. I've been trying to explore other modalities:

https://youtu.be/rdlAZfH9Uhk

I've been working with this idea for a while. Here's a year ago https://youtu.be/0Ot0j4R36b8

Sorry for the ramble below I'm kinda obsessed about this:

The basic theory is the current bottleneck for computing is input devices not being versatile and expressive enough. The computer is waiting for us these days. How can someone do interface "chunking" (https://en.m.wikipedia.org/wiki/Chunking_(psychology)) if they have something other than buttons at their disposal. Are their new UI paradigms to exploit?

The interfacing is thru python and is part of a larger music exploration project I've got terribly documented. That's kinda the "first application" just like office productivity was the first application of WIMP. It makes the goal of intuitively navigating multidimensional problems through geospatial and tactile interfacing less abstract.

The actual script is here https://github.com/kristopolous/music-explorer/blob/master/t...

I was exploring everything from browser tab controls to instrumenting software debuggers to window management with this thing so it's kinda a mess. There's a lot of code scattered in my various projects on this topic

The biggest UI fumble is the value repetition and tactility is great until the context of the control changes and then you get a discontinuous jump.

The "obvious" workaround of using relative values has the random walk problem. You need a disengagement mode to reset it from wandering off to one of the edges. That's a flow disrupting cognitive shift (https://en.m.wikipedia.org/wiki/Cognitive_shifting)

I tried using 2 cheap arduino joy controllers (example https://satkit.com/image/cache/catalog/products/arduino-joys...) and allocate one joy for mode and the other one for value (inspired from orbitouch https://c1.neweggimages.com/ProductImage/A446_13123722022688...) it works fine with logarithmic relative values. (Going to value 0 and max are common)

Also both these models are about giving the thumbs more jobs then just a single key, an idea I stole from Maltron (http://www.micwil.com/images/gallery/maltron_keyboards_dual_...). Sliding a control right next to a conventional keyboard base should be doable on the cheap and help get towards the goal. The joys were a better solution for this but then I'd have to "sell" something which I'm trying to avoid.

My earlier method involved a USB foot pedal which acted as a quasimode for control (example: https://www.geeky-gadgets.com/wp-content/uploads/2010/01/usb... ... it can be assigned to something like "Hyper" in Xorg parlance, see here https://github.com/kristopolous/tools/blob/master/bin/footpa...) and then my lua scriptable window manager captured the key strokes and delegated things accordingly (example: https://github.com/kristopolous/crappy-notion/blob/master/mo...). It's still just "buttons" at the end though and this whole project is about integrating sliders knobs and joys to general computing.

I just hope to get everything done satisfactorily with the midi, I don't have to manufacture anything and it's a far easier technical investment for casual people and the device in the video is only $40.

I don't want to "own" anything here. I want people to liberally iterate and innovate as much as possible. I'm certainly not the smartest person in the room.