Wrapping System Functionality in Shortcuts - Any Interest


As we know, iOS 13 allows us to provide Shortcuts to the system that take input. That opens up a whole new world of what we can build.

I’ve been playing with wrapping up some functionality for native apps into Shortcut intents, i.e.

  • Image Classification (what is in an image, i.e. ‘dog’)
  • Image Similarity (how similar are two images)
  • Text Sentiment (i.e. is this text positive or negative)
  • Text Extraction (i.e. get the URLs, dates, addresses, etc… from text)

And some other ideas too. Here are some sample videos and if anyone has any interest, happy to share or even collaborate. Pretty rough right now but you can get the idea.

1 Like

That looks interesting! Especially the image similarity thing! Can you please show how this would work?

I’m still working on the image similarity but basically you feed in two images and you get a numerical score, the lower the number the more similar the images are. It’s based on new functionality in the Vision and CoreML APIs that are in iOS 13.

It’s based on this: https://developer.apple.com/videos/play/wwdc2019/222/

1 Like

Another one is on-device OCR, something else that’s new in iOS 13.

Errr, maybe not OCR. Looks like the Vision OCR stuff uses too much RAM for an Intents extension.

Anyway, I think what I’m going to do is to expand on this a bit and then run a TestFlight with anyone interested, get some feedback and then decide if it’s worth pushing to the App Store in the fall.

1 Like

As promised, a very rough version of this is on TestFlight. The app itself is not all that interesting, it’s more about the shortcuts that are included.

Requires the iOS 13 beta.

I’d like to join, but I don’t have the iOS 13 beta installed and no device where I can install it except my daily used iPhone where I don’t want to install it… If iPadOS would support the iPad Air 1, I would install it right away^^

I am in.

I am still trying to figure out how to get a xml response from api into a JSON… so far most attempts to access the data cause it to disappear or they get rejected from various conversion shortcuts.

So you’ve got an API that returns XML and you’d like to map that to Shortcuts dictionaries?