Feature request: Apple Shortcuts Actions

[roelvangils]

Wooshy has now become very good at swiftly finding and activating all possible UI elements and other text fragments on the screen and in web pages (based on the accessibility information that is exposed by the system). And thanks to ⌘+C , I can now easily copy the appropriate information to my clipboard. I already can't miss this functionality!

Now, here's the idea: I also happen to be a big fan of Keyboard Maestro and Apple Shortcuts to automate things. In both KM and Apple Shortcuts, there is currently no way to click (or double-click, for that matter) elements on the screen based on that element's accessibility information. In KM, however, you can use OCR to find something on the screen (or in the active app), and there is also this function:

... but it is unreliable and only works for buttons and not for other UI elements.

I see an opportunity for Wooshy to make clicking on elements in the active app a lot easier through a Shortcut Action. For example, it could work like this:

• First, you use the 🕵️ Inspector (or ⌘+C) to find out which string is required to click on something.
• In Shortcuts, you would then have an action called "Wooshy" in which you can pass that value as a paramater. You could also choose whether to click or double-click (or ⌘+O, for example).

I think this could open up a lot of new automation possibilities. Just as for example, an app like BetterTouchTool adds custom actions, like this very handy one:

... supporting Shortcut Actions in Wooshy would draw the attention of people interested in macOS Automation. Of course, it's a much more reliable and efficient way than the Keyboard Maestro "hack" that uses OCR to click somewhere.

What do you think?

[godbout]

great. thanks for bringing this up. i have some vague idea that it'd be great to support "some sort" of Apple Shortcuts "somehow", but i have no idea how and what and why. i personally don't use them at all.

i'll review what you wrote and try to understand and digest. and will probably come back with questions. this time please answer them!!! 😂️

[roelvangils]

i'll review what you wrote and try to understand and digest. and will probably come back with questions. this time please answer them!!! 😂️

I will! I turned on ALL notifications! 😃