Script to show upcoming U2 concerts


#1

I would like to have a script that shows upcoming U2 concerts from https://www.u2.com/tour

Since I’m not familiar with scraping websites (parsing elements) this might be a (nice) challenge. The examples that @simonbs posted here will definitely help though!

Unfortunately I won’t have time until this weekend to experiment. If anyone is up to this I would be happy to learn from her/him!

Desired output: Date, City, Country (in aligned columns)

Bonus: Showing a flag of the country instead of the two letter abbreviation


#2

I happened to spot this the other day and bookmark it:

If they use ISO countries it should work to convert the codes to flags!


#3

If I get that working I might be interested in showing the upcoming games of my favourite soccer team (date, day, competition, home team, away team - adding club logos)

Too many ideas, not enough time… :wink:


#4

I am not believing this! Just today I was “wishing” that I could automate my data collection from various sites and then post it into my community forum. Doing it by hand is VERY time consuming. Y’all have REALLY gotten me to thinking hard on this now! THANKS for the motivation! :+1:t2::+1:t2::+1:t2::+1:t2:


#5

In general I think it’s preferable to use an API rather than scraping, if possible. Maybe this?

https://www.songkick.com/developer/upcoming-events-for-artist


#6

I jus took a look at U2s website. Turns out they have all their concerts embedded as JSON so it was super simple to get an array of all the concerts. This should get you started.

let url = "https://www.u2.com/tour"
let req = new Request(url)
let html = await req.load()
let json = html
  .split("<script type='application/ld+json'>")[1]
  .split("</script>")[0]
let concerts = JSON.parse(json)
QuickLook.present(concerts)

#7

Yes, I think using an API is better then scraping. However, I (also) prefer to get data from the “true” source.

So I’m really happy with what @simonbs posted and will experiment with that this weekend.


#8

Sorry for the dumb question, how did you find out that the website used JSON to store the information, I am learning JS and I think it’s important, at least to me, to understand that, thanks


#9

What I do is I open up Chrome or Firefox and the network tab in the developer tools, and then load the page. In the network section you will see everything that gets loaded - CSS files, JavaScript files, PHP, JSON - anything called as part of building the web page. This would allow you to see if a JSON is loaded directly (that might not work with all pages, for example in many of my web apps you would call a PHP file which gives you a JSON output, so just searching for .json might not always work :slight_smile: ).


#10

I am using Safari, I only got CSS, GIF, JPG, JS, PNG and WOFF, there was one that took the longest to load and the type is Document, could that be the JSON document? Thanks


#11

My guess is line 126 revealed it:

<script type='application/ld+json'>


#12

Sometimes I do as Rosemary suggested other times I use Charles Proxy, either on macOS or iOS. In this case I used Charles for iOS.

I’m a long time user of Charles on macOS and while it’s only four months since the iOS version was released, it has already become one of my favorite iOS apps.


#13

With the help of @simonbs and @RosemaryOrchard:

Using this script:

let url = "https://www.u2.com/tour"
let req = new Request(url)
let html = await req.load()
let json = html
  .split("<script type='application/ld+json'>")[1]
  .split("</script>")[0]
let concerts = JSON.parse(json)

let today = new Date();
let table = new UITable();

for (concert of concerts) {
  let date = new Date(concert.startDate);
  if (date < today) continue;
  let row = new UITableRow();
  row.cellSpacing = 10;
  let country = concert.location.address.addressCountry;
  if (country == "UK") country = "GB";
  let columnOptions = [
    {value: date, format: {weekday: "short"}, align: "left", weight: 1},
    {value: date, format: {day: "numeric"}, align: "right", weight: 1},
    {value: date, format: {month: "short"}, align: "left", weight: 1},
    {value: country.toUpperCase().replace(/./g, char => String.fromCodePoint(char.charCodeAt(0)+127397)), align: "center", weight: 1},
    {value: concert.location.name, align: "left", weight: 5},
    {value: concert.location.address.addressLocality, align: "left", weight: 3},
  ];
  for (options of columnOptions) {
    let cell = createCell(options);
    row.addCell(cell);
  }
  table.addRow(row);
 };
 QuickLook.present(table);

function createCell(options) {
  let value = options["value"];
  let format = options["format"];
  let text = format ? value.toLocaleDateString("en-US", format) : value;
  let cell = UITableCell.text(text);
  cell.widthWeight = options["weight"];
  cell[options["align"] + "Aligned"]();
  return cell;
}

@simonbs: I noticed something weird I think. When invoking this Siri shortcut in portrait mode (on an iPad) I get the full list from the source URL, but when invoking it in landscape mode the list ends premature (at the second concert in Amsterdam). Is this a bug in iOS 12 or in Scriptable?


#14

Awesome! I’m glad to see that you got it working.

The shortened list is actually working as intended. What you are experiencing is a limitation in iOS. The UI presented in a Siri Shirtcut can not exceed a maximum height. That height may vary between portrait and landscape.


#15

Oh. If I start the shortcut in portrait mode and then switch to landscape mode I still have the full list. That’s good enough as a workaround, but I guess I’m just lucky then that this leg of the tour fits in (at least) one orientation…


#16

Got that working as well! Won’t share it though as it is based on the home page of my favourite team and unfortunately there’s sometimes violence between “supporters” (hooligans) of different teams, so I’d rather not share my preference online… :cry:

Bit of a bummer: the logos and the league don’t fit (in separate columns) when invoked from Siri, so while I had those working as well I ended with just day, date, time, home team, and away team. Well learned something new, and the schedule was immediately useful :smile:


#17

My favourite band’s site doesn’t have a JSON file (check them out, they’re great!) so I had to work directly on the HTML. Here’s a screenshot and the code I wrote based on what @rob shared here, so thank you for the guidance! Also, thank you @simonbs for creating such a cool app! I’m looking forward to playing around with it even more.

// Gets page url
let url = "http://www.berritxarrak.net/kontzertuak/"
let req = new Request(url)
let html = await req.load()

// Starts table
let table = new UITable()

// Gets html with concert info
let infoStart = html.indexOf('event type-event status-publish hentry active no-media')
let infoEnd = html.indexOf('</section>',infoStart+1)

let finishSearch = html.indexOf('<!-- .main-content end -->')

while(infoStart < finishSearch){

let concertInfoHTML = html.substring(infoStart,infoEnd)

// Gets concert day
let dayStart = concertInfoHTML.indexOf('<strong>')
let dayEnd = concertInfoHTML.indexOf('</strong>')
let day = concertInfoHTML.substring(dayStart+8,dayEnd)

// Gets concert month
let monthStart = concertInfoHTML.indexOf('<span class="month">')
let monthEnd = concertInfoHTML.indexOf('</span>')
let month = concertInfoHTML.substring(monthStart+20,monthEnd)

// Gets concert day of the week
let weekdayStart = concertInfoHTML.indexOf('<span class="day-name">')
let weekdayEnd = concertInfoHTML.indexOf('</span>',weekdayStart+1)
let weekday = concertInfoHTML.substring(weekdayStart+23,weekdayEnd)

// Gets concert city
let cityStart = concertInfoHTML.indexOf('/">')
let cityEnd = concertInfoHTML.indexOf('</a></h1>',cityStart+1)
let city = concertInfoHTML.substring(cityStart+3,cityEnd)
city = city.charAt(0) + city.slice(1).toLowerCase()

// Gets concert time
let timeStart = concertInfoHTML.indexOf('<span class="event-time-text"> <i class="icon-time"></i>')
let timeEnd = concertInfoHTML.indexOf('</span>',timeStart+1)
let time = concertInfoHTML.substring(timeStart+56,timeEnd)

// Adds emojis to strings
let date = '🗓 ' + month + ' ' + day
city = '🏙 ' + city
time = '🕘 ' + time

// Creates row inside the table
let row = new UITableRow()

let timeCell = UITableCell.text(time)
timeCell.leftAligned()
timeCell.widthWeight = 2
row.addCell(timeCell)

let dateCell = UITableCell.text(date)
dateCell.leftAligned()
dateCell.widthWeight = 2
row.addCell(dateCell)

// let weekdayCell = UITableCell.text(weekday)
// weekdayCell.leftAligned()
// row.addCell(weekdayCell)

let cityCell = UITableCell.text(city)
cityCell.leftAligned()
cityCell.widthWeight = 5
row.addCell(cityCell)

row.height = 40
row.cellSpacing = 1
table.addRow(row)

// Gets html with concert info
infoStart = html.indexOf('event type-event status-publish hentry active no-media', infoEnd+1)

infoEnd = html.indexOf('</section>',infoStart+1)
}

if (config.runsWithSiri) {
  Speech.speak("Here's the coming Berri Txarrak concerts.")
}

QuickLook.present(table)

#18

Now that the GM build of iOS 12 is out I also updated my iPhone X (hope I won’t regret that!).

Looks like I need to modify all my scripts. While the output was fine on my iPad Pro I get a lot of ... instead of text on the smaller iPhone screen… :cry:


#19

its giving me the error message: “TypeError: html.split is not a function” since "‘html.split’ is undefined?

can anyone help me (I’m new to scriptable but looking fwd to do more) …

thanks in advance


#20

I think the script was originally created during the beta of Scriptable. During that time, Request.load() was changed to Request.loadString().

Can you try to change let html = await req.load() to let html = await req.loadString() and see if that works?