Recent

Author Topic: Open list of URLs and save them?  (Read 1740 times)

chuckles1066

  • New Member
  • *
  • Posts: 20
Open list of URLs and save them?
« on: August 28, 2021, 03:47:16 pm »
Hello,

Is this possible?

Take a list of URLs, open the first one in Chrome, save it as an .mhtml file, close Chrome.

Rinse and repeat.

I've tried using Powershell, Python and even VBA but am struggling.

Many thanks for reading.

rsz

  • New Member
  • *
  • Posts: 45
Re: Open list of URLs and save them?
« Reply #1 on: August 28, 2021, 04:13:05 pm »
Hi,

Have a look at Selenium, I've used that in the past to automate browsers. I haven't used it together with Free Pascal. You'll need a Free Pascal WebDriver for it. Have a look at: https://forum.lazarus.freepascal.org/index.php?topic=50082.0 .

It might also be worth looking into writing a Chrome extension where all it does is accept a URL through some sort of IPC and then saves that URL.

chuckles1066

  • New Member
  • *
  • Posts: 20
Re: Open list of URLs and save them?
« Reply #2 on: September 04, 2021, 08:02:30 pm »
Thanks for the reply.

I managed to resolve using a mix of PowerShell, JavaScript, Python and the gago library in Go to communicate with the webpage (effectively mimicking what http://wayback.archive.org does).

I ran into a lot of HTML markup issues and AJAX problems but I got around most of those using Angular.js.

Nightmare.

Leledumbo

  • Hero Member
  • *****
  • Posts: 8746
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: Open list of URLs and save them?
« Reply #3 on: September 17, 2021, 08:25:48 am »
I wonder why do you need to do so. If the final purpose is to simply save the page as a file, why not just request the page then save it directly, no web browser necessary.

 

TinyPortal © 2005-2018