Lazarus

Free Pascal => General => Topic started by: chuckles1066 on August 28, 2021, 03:47:16 pm

Title: Open list of URLs and save them?
Post by: chuckles1066 on August 28, 2021, 03:47:16 pm
Hello,

Is this possible?

Take a list of URLs, open the first one in Chrome, save it as an .mhtml file, close Chrome.

Rinse and repeat.

I've tried using Powershell, Python and even VBA but am struggling.

Many thanks for reading.
Title: Re: Open list of URLs and save them?
Post by: rsz on August 28, 2021, 04:13:05 pm
Hi,

Have a look at Selenium, I've used that in the past to automate browsers. I haven't used it together with Free Pascal. You'll need a Free Pascal WebDriver for it. Have a look at: https://forum.lazarus.freepascal.org/index.php?topic=50082.0 .

It might also be worth looking into writing a Chrome extension where all it does is accept a URL through some sort of IPC and then saves that URL.
Title: Re: Open list of URLs and save them?
Post by: chuckles1066 on September 04, 2021, 08:02:30 pm
Thanks for the reply.

I managed to resolve using a mix of PowerShell, JavaScript, Python and the gago library in Go to communicate with the webpage (effectively mimicking what http://wayback.archive.org does).

I ran into a lot of HTML markup issues and AJAX problems but I got around most of those using Angular.js.

Nightmare.
Title: Re: Open list of URLs and save them?
Post by: Leledumbo on September 17, 2021, 08:25:48 am
I wonder why do you need to do so. If the final purpose is to simply save the page as a file, why not just request the page then save it directly, no web browser necessary.
TinyPortal © 2005-2018