Just in the middle of listening to this episode, but this is basically awesome. I was planning to buy a StreamDeck, but with this idea in hand I am going to run with this instead and see how it works out for me. The cost is, if you want a dedicated setup (eg not just hooking up your ipad as needed) more than the highest end StreamDesk (assuming the lowest end iPad will work, plus the Luna Display dongle, plus maybe an ApplePencil) but the nice thing (for me) is that you can custom create buttons sizes and layouts, which I think will be handy.
I will likely start out creating palettes for Lightroom, Finder (I use a lot of KM based palettes for things like file renaming and tagging when I am doing my paperless filing thing and this will be pretty convenient I think) and DEVONThink, and see where it takes me. For nonscriptable apps like Lightroom, it will still be convenient to use the ability to type keystrokes to change modules, enter grid, compare, survey modes, open or close editing tools, etc.
It would be really nice if the palette design allowed the entry of title fields and shapes to group buttons, but I will get around the former with buttons that have no actions…it would also be cool if you could programmatically change the content of a button…maybe someday!
One thing I have not yet figured out. While I understand the concept of encapsulating the “works” of the code into AppleScript script libraries, which makes a whole lot of sense, my initial experiments indicate that you actually have to create a script file to attach to the AppleScript action of a button (even if that script file just contains code to execute a handler out of your script library). My question: Where is the “proper” place to store all of those script files? Just create a random folder like ~/AssessibilityScripts and drop them all in there? As far as I can tell, the palette editor does not provide a mechanism for embedding the script file directly into the plist that controls it, nor can you edit the script itself from within the editor - you have to reopen the file in the AppleScript editor.
Anyone play with this yet and know the answer to this one?
Many thanks to Sal for this awesome concept. I had no idea you could even create custom keyboards like this, but now that I do…
Ok, a little experimentation done…
Within ~/Library/Application Support/com.apple.AssistiveControl I know have a bundle called dwellControlUserPanels1.asconfig. I have no idea why it is stuck with this name, but the weird name is not affecting anything so…
This is a fairly standard app bundle. In the Contents/Resources subfolder, I have found some interesting items.
Firstly, to create the “control panel” analogous to what Sal has shown (in my case, three rows of buttons. Rows 2 and 3 contain my most commonly launched apps; I have not decided what to do with Row 1 yet…) I simply did a get info on each app I was putting on a button, copied its icon, selected the button the Panel Editor, and pasted, and the icon became the image for that button. The images were all named “untitled n”, n sequentially numbered as I pasted the images.
In the Resources folder, I found a bunch of files named Image.UUID, with UUID being obviously a unique UUID for each image. So that’s how the images got stored into the bundle.
I also found a file called Script.UUID. It is a compiled AppleScript. I had created a test script called test.scpt on my Desktop and set it as the script to run for a given button click, and it appears that when you do that, the Panel Editor copies the script into the Resource folder. If I edited the test.scpt script on the Desktop and then set that script for another button, it too was copied but did not break the first button - so basically you can create a single script file, if you want, edit it for each button, assign that file to the button, and re-edit for the next button.
It is not easy to edit the files in the Resources folder, however, so I would probably opt to create a folder somewhere, create all the necessary script in individual files, and then assign them in the Panel Editor to be copied to the Resources folder. To edit a script in the Resources folder, you would have to rename it with a .scpt extension, edit it, and rename it back…probably going to break something if you do that.
I wonder what approach Sal has taken to this process?
I think we should start a GoFundMe to get @RosemaryOrchard one of these:
(I still don’t understand how a single 
took her down at WWDC, but she’s said it quite distinctly now on two separate occasions. A leaf. Not leaves. If not for my own pedestrian misadventures, I might find that impossible to believe, but sometimes all it takes is one little thing in just the wrong place.)
For those who are interested here are the Panel Editor docs:
Thanks…read that already. Sadly it is scanty documentation…but I am figuring things out as I go along.
Right now figuring out how to script System Preferences so I can turn these keyboards on and off…I figure an Alfred workflow: Panel ON and Panel OFF…
Pretty sure you’ll need to use UI scripting for that, unfortunately. Accessibility isn’t in the System Events dictionary.
More info: https://nshipster.com/accessibility-keyboard/
I have not found a command line way to enable/disable the accessibility keyboard yet, so I may have to go with UI scripting, at least for now.
this works on Mojave (you’ll still need to confirm manually when turning it off):
tell application “System Preferences”
reveal anchor “Keyboard” of pane id “com.apple.preference.universalaccess”
end tell
delay 0.5
tell application “System Events”
click radio button “Accessibility Keyboard” of tab group 1 of group 1 of window “Accessibility” of application process “System Preferences”
click checkbox “Enable Accessibility Keyboard” of tab group 1 of group 1 of window “Accessibility” of application process “System Preferences”
end tell
It doesn’t work in High Sierra…the unfortunate quirks of UI scripting.
Alternatively there’s this panel
but for some reason the Accessibility Keyboard can’t be accessed via Siri.
Also, if you want to use Sal’s examples on Mojave you can change this key in Contents/info.plist :
<key>ASCConfigurationVersion</key>
<string>7.0</string>
from 7.0 to 4.0
I played around with this when Viticci wrote it up back in the Spring. It’s really cool, and something I didn’t know was in macOS until I read about it. It’s fun to bridge the iOS and macOS worlds.
It appears that TidBITS have written up this same idea:
So…after messing around a bit…I think the only advantage this would have over my existing config which uses the TouchBar, BetterTouchTool and Duet Display is the customizable (and potentially much larger) panel size…still it’s super cool just to see the possibilities.
Incidentally the Accessibility Keyboard works just fine with touch on a device running Duet.
-
I cannot get your AppleScript to compile (on Mojave). I admit limited experience with UI scripting in AppleScript, so I will have to spend a bit of time troubleshooting.
-
Thanks for the tip on loading Sal’s examples. I noted that they won’t load in Mojave, but had not yet gotten to delving into the how and why. I have been playing around with creating some template panels for Finder and for DevonThink to start getting a feel of what I might want to use all of this for.
@ismh and @Tjluoma: Thanks for the linkage. I have now found other references online to the idea of creating custom panels, but this was a new concept for me until I listened to the Automators episode with Sal, so I’m just getting starting.
I think sooner or later I will have to tear into the plist file format, because one thing I would really like to be able to do is make the common buttons for launching all the apps and so forth not be duplicated into each panel, making it a major chore to change them…and at this point I don’t really know how I will want all of that to settle out.
What I find exceptionally cool is the way that the panel can automatically change went you change apps, which is just a cool thing to have happen! Plus, the idea that instead of the rather more bulky StreamDeck I can just throw my MBAir and iPad into my bag and use this on the go is very cool.
Here it is in an Alfred workflow that works for me on Mojave, & doesn’t work in HS:
Thanks…just getting there. Found some garbage characters in the AppleScript which is why your suggested code didn’t work for me and fixed it up…will configure up Alfred…
Yeah I probably should have stuck it in a gist rather than try to post it directly in the forum. Glad you sorted it out.
Thanks for the Alfred workflow which I can confirm works on Catalina
I got my own panel working and even added a couple of small scripts to run Keyboard Maestro workflows.
I have got Sal’s examples running by importing them in. Just can’t get the scripts to run. Is it because they are in bundles?
I tried to do a save as to convert from a bundle to a script.
Here is a Panel I made for myself. I added some of the buttons from Sal’s Panels. I have it set up to display when Drafts is in front.
The dictation button runs a Keyboard Maestro workflow which sets up Dragon Dictate ready to use.
I have a couple of buttons which send the text to Day One app.
I don’t have Luna Display and I don’t need it. Easy to use the Apple Pencil to tap the buttons.
One of the best podcasts from the Automators. I’ve been playing with these panels for a couple of days and enjoying being good and geeky.