Regexp not working

goodyear77 · November 3, 2020, 3:24pm

Hi everybody,

I had a macro that was working on one Mac, and then when I synced it with a new Mac it doesn’t work anymore. It complains that the regular expression failed to match, but when I check it on https://regex101.com/ it works fine.

Here is an example of the text:

To: servicedesk@somecompany.com
Subject: Parking space
Time: 2020-11-03T14:18:39+00:00
---------------------------------------------
Hi!

I'd like to reserve a parking spot.

I use this regexp expression which works fine on https://regex101.com:
\A(.+)(\n+)(.+)(\n+)([\s\S]+)

However, this fails in Keyboard Maestro with the following config (the clipboard contains the text when I paste it into a text editor):

Not sure what I’m doing wrong? Any help would be greatly appreciated.

Cheers!

goodyear77 · November 3, 2020, 3:25pm

Adding the screenshot from https://regex101.com/ since I could only attach one screenshot per post:

goodyear77 · November 3, 2020, 3:34pm

As long as I’m talking to myself I might as well continue It turns out that if I copy the text from a new page in TextEditor, the text is parsed correctly and the regexp works, but when I copy the text in the notes section in Microsoft ToDo, the text is not parsed correctly.

If I paste the text I copied from ToDo into a TextEditor page, the text is parsed correctly, which leads me to believe the copy operation is somehow not in plain text…

dustinknopoff · November 3, 2020, 3:41pm

That seems likely from your findings. Have you tried displaying the results of the copy action to a window or saving to a variable to determine if it’s not plaintext?

goodyear77 · November 3, 2020, 3:55pm

The macro is replicating keystrokes, so I try and replicate the keystrokes (cmd-A->cmd-C->Esc) and then pasting the results in a TextEditor window, and the text is copied correctly. I’m not really sure how to save the text into a variable and then run the regexp on the variable… Not yet at least

goodyear77 · November 3, 2020, 4:15pm

So I added the step “Display System Clipboard”, and the contents look correct, the right information is copied. So now I’m back at the question why the regexp fails…

goodyear77 · November 3, 2020, 6:34pm

Well, time to close this issue; the problem was that there was no problem: the regexp didn’t match anything since I was only putting text into variables, not matching any specific string. Once I removed the notification that the regexp failed, and disabled the option to abort the macro because of this failure, the macro completed perfectly.

You learn something new everyday…

EDIT: scratch that, this is not working. Soo frustrating, can’t understand why the regex is not parsing the clipboard.

RobTrew · November 3, 2020, 8:37pm

This may just take us back to the Jamie Zawinski quote:

[Now they have two problems](https://www.goodreads.com/author/quotes/7549076.Jamie_Zawinski)

The cost / benefit of regular expressions is often doubtful, and as they grow, they quickly approach a write-only condition.

If you are using Keyboard Maestro then it might be more productive (quicker to test and debug) to:

use an Execute JavaScript action
Split into lines ( String.split )
Find the lines that start with particular strings ( String.startsWith )

(remember that line-endings coming in from Microsoft software may not be LF delimited)

goodyear77 · November 3, 2020, 9:40pm

Thanks! Im kind of a novice, any tips on how it would look?

I’m working under the assumption that it’s the format that’s off; if I paste the text from ToDo into TextEdit, and then copy the text from TextEdit back to ToDo and run the macro, it works. So if there is a way to transform the clipboard to plain text that’d be great!

RobTrew · November 3, 2020, 11:01pm

Well, first we need to take a look at that clipboard.

Copy a sample note from ToDo
run this KM macro Clipboard viewer - Macro Library - Keyboard Maestro Discourse
and let us see what it shows you.

RobTrew · November 3, 2020, 11:48pm

(If there is already any public.utf8-plain-text among the clipboard formats in there, (and we can skip conversion from RTF/HTML) then you may be able to set your KM variables by pasting the whole of the draft below (scroll right down to the end) into a Keyboard Maestro Execute JavaScript for Automation action.

If this verbose experimental draft works, then we can give you something simpler and more accessible. (It’s faster to me to start with a lot of copy-paste)

Experimental draft for setting KM variables if the clipboard contains any UTF8

(() => {
    'use strict';

    ObjC.import('AppKit');

    // main : IO ()
    const main = () => {
        const kme = Application('Keyboard Maestro Engine');
        return either(x => x)(x => x)(
            bindLR(
                clipTextLR()
            )(txt => {
                // Lines parsed by a split
                const dict = lines(txt).reduce(
                    (a, s) => {
                        const parts = s.split(': ');
                        return 1 < parts.length ? (
                            Object.assign({
                                [parts[0]]: parts
                                    .slice(1)
                                    .join(': ')
                            }, a)
                        ) : a;
                    }, {}
                );
                // Any captured values saved to 
                // KM variables.
                return Right(
                    ['To', 'Subject', 'Time']
                    .map(k => {
                        const
                            v = dict[k],
                            varName = `ToDo${k}`;
                        return undefined !== v ? (
                            kme.setvariable(
                                varName, {
                                    to: v
                                }
                            ),
                            `${varName}: ${kme.getvariable(varName)}`
                        ) : `${k}: ?`
                    })
                )
            })
        );
    }

    // ----------------------- JXA -----------------------

    // clipTextLR :: () -> Either String String
    const clipTextLR = () => (
        v => Boolean(v) && 0 < v.length ? (
            Right(v)
        ) : Left('No utf8-plain-text found in clipboard.')
    )(
        ObjC.unwrap($.NSPasteboard.generalPasteboard
            .stringForType($.NSPasteboardTypeString))
    );

    // --------------------- GENERIC ---------------------
    // https://github.com/RobTrew/prelude-jxa

    // Left :: a -> Either a b
    const Left = x => ({
        type: 'Either',
        Left: x
    });

    // Right :: b -> Either a b
    const Right = x => ({
        type: 'Either',
        Right: x
    });

    // bindLR (>>=) :: Either a -> 
    // (a -> Either b) -> Either b
    const bindLR = m =>
        mf => undefined !== m.Left ? (
            m
        ) : mf(m.Right);

    // either :: (a -> c) -> (b -> c) -> Either a b -> c
    const either = fl =>
        // Application of the function fl to the
        // contents of any Left value in e, or
        // the application of fr to its Right value.
        fr => e => 'Either' === e.type ? (
            undefined !== e.Left ? (
                fl(e.Left)
            ) : fr(e.Right)
        ) : undefined;

    // lines :: String -> [String]
    const lines = s =>
        // A list of strings derived from a single
        // newline-delimited string.
        0 < s.length ? (
            s.split(/[\r\n]+/)
        ) : [];

    return JSON.stringify(
        main(),
        null, 2
    );
})();

goodyear77 · November 4, 2020, 12:59pm

Thank you so much for your help! This is what the clipboard viewer macro outputs:

{
  "public.rtf as string": "{\\rtf1\\ansi\\ansicpg1252\\cocoartf2513\n\\cocoatextscaling0\\cocoaplatform0{\\fonttbl\\f0\\fnil\\fcharset0 HelveticaNeue;}\n{\\colortbl;\\red255\\green255\\blue255;\\red99\\green99\\blue101;\\red52\\green79\\blue238;}\n{\\*\\expandedcolortbl;;\\cssrgb\\c46275\\c46275\\c47059;\\cssrgb\\c26275\\c41569\\c94902;}\n\\pard\\tx560\\tx1120\\tx1680\\tx2240\\tx2800\\tx3360\\tx3920\\tx4480\\tx5040\\tx5600\\tx6160\\tx6720\\sl-340\\slleading80\\pardirnatural\\partightenfactor0\n\n\\f0\\fs26 \\cf2 From: {\\field{\\*\\fldinst{HYPERLINK \"mailto:zakaria.bennani@gmail.com\"}}{\\fldrslt \\cf3 \\ul \\ulc3 zakaria.bennani@gmail.com}}\r\\\nSubject: Parking space\r\\\nTime:2020-11-04T07:36:49+00:00\r\\\n---------------------------------------------\r\\\nHi!\r\\\n\r\\\nI'd like to reserve a parking spot.\r\\\n}",
  "public.rtf as data": "{\\rtf1\\ansi\\ansicpg1252\\cocoartf2513\n\\cocoatextscaling0\\cocoaplatform0{\\fonttbl\\f0\\fnil\\fcharset0 HelveticaNeue;}\n{\\colortbl;\\red255\\green255\\blue255;\\red99\\green99\\blue101;\\red52\\green79\\blue238;}\n{\\*\\expandedcolortbl;;\\cssrgb\\c46275\\c46275\\c47059;\\cssrgb\\c26275\\c41569\\c94902;}\n\\pard\\tx560\\tx1120\\tx1680\\tx2240\\tx2800\\tx3360\\tx3920\\tx4480\\tx5040\\tx5600\\tx6160\\tx6720\\sl-340\\slleading80\\pardirnatural\\partightenfactor0\n\n\\f0\\fs26 \\cf2 From: {\\field{\\*\\fldinst{HYPERLINK \"mailto:zakaria.bennani@gmail.com\"}}{\\fldrslt \\cf3 \\ul \\ulc3 zakaria.bennani@gmail.com}}\r\\\nSubject: Parking space\r\\\nTime:2020-11-04T07:36:49+00:00\r\\\n---------------------------------------------\r\\\nHi!\r\\\n\r\\\nI'd like to reserve a parking spot.\r\\\n}",
  "public.utf8-plain-text as string": "From: zakaria.bennani@gmail.com\r\nSubject: Parking space\r\nTime:2020-11-04T07:36:49+00:00\r\n---------------------------------------------\r\nHi!\r\n\r\nI'd like to reserve a parking spot.\r\n",
  "public.utf8-plain-text as data": "From: zakaria.bennani@gmail.com\r\nSubject: Parking space\r\nTime:2020-11-04T07:36:49+00:00\r\n---------------------------------------------\r\nHi!\r\n\r\nI'd like to reserve a parking spot.\r\n"
}

I then added the action “Execute a JavaScript for Automation” and pasted the provided code into the text box. The end result was that my original macro now completes and I get a window with the following contents:

[
  "To: ?",
  "Subject: ?",
  "Time: ?"
]

I’m using the “Subject:” variable to perform a search in Outlook, and the search now completes (no error in the regexp which is a step forward), but the variable used in the search is this (not the subject line variable):

 "To: ?",

Here is a link to my original macro if you want to have a look at the whole thing.

Any idea what could be missing? Feels like we’re so close!

RobTrew · November 4, 2020, 3:36pm

Yes, the problem is visible in the clipboard listing – the line endings are Windows-style \r\n, rather than the default Mac/Unix \n, and this will have been complicating the segmentation and matching of the lines.

Your Regex anticipates \n line-breaks, and doesn’t match the actual \r\n sequences.

I’ll take a look at your macro later on.

RobTrew · November 4, 2020, 4:05pm

OK, it seems that all you are looking for is the Subject: line, and the value following that prefix.

I would personally:

clamber out of the regex tar-pit
split the lines on \r\n
find the the first line that starts with 'Subject: ', and take the rest of it.

You could do this in AppleScript or JS.

Here, FWIW, is a JS version, with source code, for pasting into a KM Execute JXA action, below:

Copy all of this and paste into the Execute JavaScript for Automation action:

(() => {
    'use strict';

    ObjC.import('AppKit');

    const main = () => {
        const
            xs = lines(clipboardText()),
            subjectLine = xs.find(
                x => x.startsWith('Subject')
            );

        return undefined !== subjectLine ? (
            subjectLine.split(': ')[1]
        ) : 'Subject not found ...'
    };

    // ----------------------- JXA -----------------------

    // clipboardText :: IO () -> String
    const clipboardText = () =>
        // Any plain text in the clipboard.
        ObjC.unwrap(
            $.NSString.alloc.initWithDataEncoding(
                $.NSPasteboard.generalPasteboard
                .dataForType($.NSPasteboardTypeString),
                $.NSUTF8StringEncoding
            )
        );

    // lines :: String -> [String]
    const lines = s =>
        // A list of strings derived from a single
        //  string delimited by newline and or CR
        0 < s.length ? (
            s.split(/[\r\n]+/)
        ) : [];

    return main();
})();

goodyear77 · November 4, 2020, 6:44pm

This works, THANK YOU SO MUCH! I’m pretty particular with my workflow, and this helps me immensely!

RobTrew · November 4, 2020, 7:04pm

Good, I’m glad that worked well : -)