Web scrape shortcut runs manually but not via Hey Siri

Pulling my hair out on this one. Was stoked when I got it to work as desired in the app, go to run it via Hey Siri and she can’t figure it out to save her life. “Sorry, something went wrong” or “Uh oh, there’s a problem”

The shortcut scrapes Accuweather Minutecast for the current summary (eg “No precipitation for 120 minutes”, “Rain starting in 13 minutes”)

Troubleshooting I’ve done:

  • renamed shortcut to “banana split” to eliminate potential conflict — no change
  • added breaks to “step through” the code — it borks at the regex match
  • added a pause after the regex in case the internet call needed more time — the shortcut breaks in the regex match before it even gets to the pause

I don’t know if it’s breaking on the whitespace(??) I’m kindergarten-level regex so I couldn’t figure out how to trim it and when I tried matching a different string I broke the whole thing, got frustrated, and came here. :upside_down_face:

The only action missing from the screenshot is the URL so I don’t geographically dox myself.

Running 13.5 (yes, it’s intentional)

Not sure on the answer but I can help with trimming the whitespace.

The ‘Actions’ and ‘Toolbox Pro’ apps both have trim whitespace actions.

When you run it in the app, do you see the web page? And when you try to run it from Siri, do you not see it? (Ie, it runs in the background)

Asking because it could be a user agent problem. Some websites will serve up the page you want when the user agent (browser) is one a human is likely to be using – but will return nothing or an error page when the user agent looks like a bot.

If Shortcuts is identifying itself as an automated agent, Accuweather may not be sending the results at all, in which case the regex would fail.

Speculative, but this is a common problem with web scraping generally.

No to both. In app, the icon “fills” and then the response is spoken. In edit mode, the actions highlight as it runs and then the response is spoken. From Siri, it does the Cylon side to side “thinking” and then borks.

1 Like

So…progress??

I figured out how to assign the User-Agent in the Action. The first time calling the shortcut via Siri after editing I get “Done” or “Ok”. Subsequent calls without opening in the Shortcuts app get the same bork.

User Agent was part of the problem but not the whole problem?

1 Like

That’s really weird! But progress indeed…

I gave up trying to do this via web scrape. Found out AccuWeather has a free API with 25 calls per day. (Unfortunately I burned 80% of those with a poor match syntax.)

I couldn’t figure out why Siri would tell me Done or Ok without a result.

1 Like

Good lesson – always look for an API before web scraping…

You might check if OpenWeatherMap has the data you need. I’ve found it great for widgets etc