Unable to get DOM element that is loaded dynamically

Scriptable
1

Hi everyone. I have some entry level JavaScript experience, and I’ve been trying to get a script to login and click some buttons on a website, but have reached a tough hurdle where I am unable to find an element using document.getElementByID() or document.contains() and remove it using .remove() .

With each, the console returns the element as “null”. I have tried setTimeout, which does delay properly, but the DOM nodes I’m interested in removing only appear after I present the web view.

See my code below (code that is specific to website has been removed for confidentiality):


var wv = new WebView();
await wv.loadURL("loginwebsite.com");

await wv.evaluateJavaScript(`

var emailTextBox = document.getElementById("username");
emailTextBox.value = "xxxxxx"

var pwTextBox = document.getElementById("password");
pwTextBox.value = "yyyyyy";

var submitBtn =
document.querySelector("body > form > div.ping-buttons > a");

submitBtn.click();`);

await wv.waitForLoad();

await wv.loadURL("SecondWebsite.com");

await wv.waitForLoad();

await wv.evaluateJavaScript(`function removeDivs () {
var divOne = document.getElementById('mydialog');
var divTwo = document.getElementById('mydialog-cover');
divOne.remove();
divTwo.remove();

}

setTimeout(removeDivs, 5000);`);


await wv.present(fullscreen=false);

I also tried a different technique for waiting for the elements to appear before trying to remove them, as shown below:

await wv.evaluateJavaScript(`function removeDivs () {
var divOne = document.getElementById('mydialog');
var divTwo = document.getElementById('mydialog-cover');
  if (document.body.contains(divOne)) {
divOne.remove();
  } else {
  setTimeout(removeDivs, 300); 
}
  if (document.body.contains(divOne)) {
divTwo.remove();
  } else {
  setTimeout(removeDivs, 300); 
}
}

removeDivs();`);

Neither work as anticipated. When the web view presents, even after a 5 second timeout, the original webpage’s JavaScript THEN runs on the page, and the divs I’m looking for are created.

Perhaps there is a way I can present the web view, let the elements populate dynamically, then evaluate my JavaScript to remove the divs? Or maybe there is something simple I am missing.

In any case, thank you all in advance. I’ve been stuck on this issue for days now. And please bear with me, I’m still learning about scriptable.

2

Alternatively, if there is a way that I can change the user agent to load the web view as a desktop page, I’m fairly confident I can remove the divs in question rather easily. I have tested this extensively on my computer. But it seems that web view just loads the page with a mobile user agent by default. Any ideas on this?

3

If it helps:

My javascript works fine and removes the elements I want if I present the webview, close the webview, then evaluate the javascript after.

Is there a way I can tell what triggers the elements to load dynamically? And then emulate it in the webview so that I don’t need to unnecessarily present the webview every time?

4

First of all, it’s very odd that the website creates the elements only when the WebView is presented. The website shouldn’t know if it is presented or not.

Do you see the elements being created when the WebView opens?

My first suggestion would be to use the setTimeout method that you already tried. Your second code example though has a bug that it will never stop if the elements don’t appear at the same time. But since you’re only interested in the removal of the elements, it’s probably not that big of a deal for you.

To send a request with your custom user agent, construct a Request with the URL and user agent, send it with .loadString() and then give the response to wv.loadHTML(request, url). The URL in there is for the WebView to know where to load further content from.

To find the trigger for the element creation is nearly impossible. First of all you could compare the HTML tree before and after presenting the WebView via wv.getHTML(). If this doesn’t really help you, you could try to wrap XMLHttpRequest and fetch in the WebView and log every call to them. But I can’t give you more instructions here since I’ve never done that and is probably difficult to get to work in this case.

Good luck!