Hello there.
I’ve worked with large CSV in Shortcuts before and the comments about unresponsiveness caught my attention.
While “Split Text” and “Repeat with Each” are pretty intuitive and really useful, they are very inefficient when working with lots of data. If you are comfortable with regular expressions you can go much faster (sometimes it feels like a superpower!
). For example, this is my optimized version of the shortcut above:
https://www.icloud.com/shortcuts/f6874826748449ac8e1b6cf229414e54
I’ve only modified the CSV conversion to Markdown part (I haven’t touched the shortcut below the “Combine Text” action) and, other than achieving the same results without using “Split Text” and “Repeat with Each”, I’ve added a question about the CSV having a header line or not.
These are the results of CSV to Markdown Table in my iPad Pro 12.9" M2:
Original (1% of data): 4.2 seconds
Original (3% of data): 16.3 seconds
Original (5% of data): CRASH
Original (10% of data): CRASH
Original (100% of data): CRASH
Optimized (1% of data): 0.1 seconds
Optimized (3% of data): 0.1 seconds
Optimized (5% of data): 0.2 seconds
Optimized (10% of data): 0.3 seconds
Optimized (100% of data): 1.7 seconds
And these are the results in my Mac Mini M1:
Original (1% of data): 1.7 seconds
Original (3% of data): 6.5 seconds
Original (5% of data): CRASH
Original (10% of data): CRASH
Original (100% of data): CRASH
Optimized (1% of data): 0.1 seconds
Optimized (3% of data): 0.1 seconds
Optimized (5% of data): 0.2 seconds
Optimized (10% of data): 0.3 seconds
Optimized (100% of data): 1.9 seconds
The data mentioned is the free dataset from World Cities Database | Simplemaps.com (44k rows, 11 columns)
The partial datasets used are from the beginning of the CSV in the link below but they’re also at https://www.atnbueno.com/automators/worldcities_partial_datasets.zip
P.S. I forgot: this is the original shortcut with the time measure added
https://www.icloud.com/shortcuts/4a405b74eb1541d1b7998c1b28b45fbd