Recent

Author Topic: Browser Automation with selenium  (Read 15477 times)

RedOctober

  • Sr. Member
  • ****
  • Posts: 452
Re: Browser Automation with selenium
« Reply #15 on: August 30, 2021, 06:31:46 pm »
I run the .exe by itself, and the same error message appears.

Jurassic Pork

  • Hero Member
  • *****
  • Posts: 1228
Re: Browser Automation with selenium
« Reply #16 on: August 30, 2021, 06:55:54 pm »
have a look in the file chromedriver.log in the project folder. When it fails i have this in the file :
Quote
[1630342063.048][INFO]: Launching chrome: "C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" --allow-pre-commit-input --disable-background-networking --disable-backgrounding-occluded-windows --disable-client-side-phishing-detection --disable-default-apps --disable-hang-monitor --disable-popup-blocking --disable-prompt-on-repost --disable-sync --enable-automation --enable-blink-features=ShadowDOMV0 --enable-logging --log-level=0 --no-first-run --no-service-autorun --password-store=basic --remote-debugging-port=0 --test-type=webdriver --use-mock-keychain --user-data-dir="C:\Users\jp\AppData\Local\Temp\scoped_dir2892_1394031789" data:,

[1630342063.084][INFO]: [281292df0dfd4ae3205b7269a71a9341] RESPONSE InitSession ERROR unknown error: Failed to create Chrome process.

« Last Edit: August 30, 2021, 07:02:18 pm by Jurassic Pork »
Jurassic computer : Sinclair ZX81 - Zilog Z80A à 3,25 MHz - RAM 1 Ko - ROM 8 Ko

RedOctober

  • Sr. Member
  • ****
  • Posts: 452
Re: Browser Automation with selenium
« Reply #17 on: August 30, 2021, 09:35:35 pm »
Hi JP.  Here is the contents of my log file.  I have to have IPV6 shut off on my server.  Would that cause this problem?  Or, maybe port 9515 is already busy?  Is there a place I can configure/reserve this port for chromedriver if it is needs this specific port/

[1630341068.643][INFO]: Starting ChromeDriver 92.0.4515.107 (87a818b10553a07434ea9e2b6dccf3cbe7895134-refs/branch-heads/4515@{#1634}) on port 9515
[1630341068.643][INFO]: Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
[1630341068.646][SEVERE]: bind() returned an error: Only one usage of each socket address (protocol/network address/port) is normally permitted. (0x2740)
[1630341068.647][INFO]: listen on IPv6 failed with error ERR_ADDRESS_IN_USE


Jurassic Pork

  • Hero Member
  • *****
  • Posts: 1228
Re: Browser Automation with selenium
« Reply #18 on: August 31, 2021, 12:25:49 am »




Hi JP.  Here is the contents of my log file.  I have to have IPV6 shut off on my server.  Would that cause this problem?  Or, maybe port 9515 is already busy?  Is there a place I can configure/reserve this port for chromedriver if it is needs this specific port/

[1630341068.643][INFO]: Starting ChromeDriver 92.0.4515.107 (87a818b10553a07434ea9e2b6dccf3cbe7895134-refs/branch-heads/4515@{#1634}) on port 9515
[1630341068.643][INFO]: Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
[1630341068.646][SEVERE]: bind() returned an error: Only one usage of each socket address (protocol/network address/port) is normally permitted. (0x2740)
[1630341068.647][INFO]: listen on IPv6 failed with error ERR_ADDRESS_IN_USE

When you have this problem it's because the webdriver is already running (see in the task manager). You must kill it before to  run your project (use the task manager).

An for the other problem on a faster computer i have no problem with fpdebug  :
I have found why you haven't the reason of the error but the message Unknown object member: "sessionId"
There is an error in the TChromeDriver.NewSession function of WebDriver4L.

With the new version of WebDriver4L (0.2) you can see why you have an error.

See  my next message for new version of WebDriver4L


« Last Edit: August 31, 2021, 12:29:51 pm by Jurassic Pork »
Jurassic computer : Sinclair ZX81 - Zilog Z80A à 3,25 MHz - RAM 1 Ko - ROM 8 Ko

Jurassic Pork

  • Hero Member
  • *****
  • Posts: 1228
Re: Browser Automation with selenium
« Reply #19 on: August 31, 2021, 12:55:30 pm »
hello,
here is the new version of WebDriver4L :
Quote
WebDriver4L   Selenium Webdriver for Lazarus  V 0.2  // Jurassic Pork 08/2021
Derivative work from Ericwang1104/WebDriver4D  : https://github.com/Ericwang1104/WebDriver4D
Source code modified to use fphttpclient, fpjson,jsonparser.
The project is in beta stage.
V 0.1 improve demo project.
V 0.2 improve WebDriver4L and demo project.
A demo project is included :  get a screenshot of the lazarus forum, get all the recent topics and display then in a memo, get stats.
the demo project has been tested on windows 10 lazarus 2.0.10   with chrome browser and on Ubuntu 20.04 Lazarus 2.0.10 with firefox browser.
You must download the selenium Webdriver for your browser and put it in your project folder. You must use a compatible driver version of your browser :
links to download webdrivers are here : https://www.selenium.dev/documentation/en/webdriver/driver_requirements/ (Quick reference)
For Chrome the selenium drivers are here : https://chromedriver.chromium.org/downloads
  If you use Chrome 92.x  on Windows  download a 92 version of the driver.
For Firefox the selenium drivers are here : https://github.com/mozilla/geckodriver/releases
For Edge Chromium the selenium drivers are here : https://msedgewebdriverstorage.z22.web.core.windows.net/
 go near to the last page with the next button and choose the same version than your ms edge chromium browser.
For IE 11 the selenium drivers are here : https://selenium-release.storage.googleapis.com/index.html?path=3.9/   (IEDriverServer 3.9)
For Safari  the selenium drivers are here : Built-in

The Software is provided “as is”, without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose and noninfringement. In no event shall the authors or copyright holders X be liable for any claim, damages or other liability, whether in an action of contract, tort or otherwise, arising from, out of or in connection with the software or the use or other dealings in the Software.

Ericwang1104/WebDriver4D has no license so this project will not be published on github. No activity since March 2019

To choose your automation browser comment uncomment this part of the code
Code: Pascal  [Select][+][-]
  1.   Try
  2.    {$IFDEF WINDOWS}
  3.      Robot := TChromeDriver.Create(nil);
  4.   //   Robot := TFireFoxDriver.Create(nil);
  5.   //   Robot := TEdgeDriver.Create(nil);
  6.   //   Robot := TIEDriver.Create(Nil);
  7.  
  8.      Robot.StartDriver(ExtractFileDir(Paramstr(0)) + '\chromedriver.exe');
  9.   //    Robot.StartDriver(ExtractFileDir(Paramstr(0)) + '\geckodriver.exe');
  10.   //   Robot.StartDriver(ExtractFileDir(Paramstr(0)) + '\msedgedriver.exe');
  11.   //   Robot.StartDriver(ExtractFileDir(Paramstr(0)) + '\IEDriverServer.exe');

the demo project has been tested on windows 10 lazarus 2.0.12   with chrome browser,  Edge Chromium and internet Explorer 11.

Friendly, J.P
« Last Edit: August 31, 2021, 12:58:57 pm by Jurassic Pork »
Jurassic computer : Sinclair ZX81 - Zilog Z80A à 3,25 MHz - RAM 1 Ko - ROM 8 Ko

Fahmy Rofiq

  • Jr. Member
  • **
  • Posts: 83
Re: Browser Automation with selenium
« Reply #20 on: September 24, 2021, 04:20:00 pm »
How to save login ?
Lazarus Trunk + FPC Fixes 32bit
Windows 10 x64

Jurassic Pork

  • Hero Member
  • *****
  • Posts: 1228
Re: Browser Automation with selenium
« Reply #21 on: September 24, 2021, 04:32:33 pm »
How to save login ?

What does it mean ?
Jurassic computer : Sinclair ZX81 - Zilog Z80A à 3,25 MHz - RAM 1 Ko - ROM 8 Ko

Fahmy Rofiq

  • Jr. Member
  • **
  • Posts: 83
Re: Browser Automation with selenium
« Reply #22 on: September 24, 2021, 04:38:05 pm »
After login to a website, how to save its cookie?
Because after application restart, it need to login again...
Lazarus Trunk + FPC Fixes 32bit
Windows 10 x64

Jurassic Pork

  • Hero Member
  • *****
  • Posts: 1228
Re: Browser Automation with selenium
« Reply #23 on: September 24, 2021, 04:46:09 pm »
Ok ,
to save cookies , history, cache you must use a profile stored in a directory.
To do this with chrome change the beginning of function TChromeDriver.NewSession in WebDriver4L unit from :
Code: Pascal  [Select][+][-]
  1. const
  2.     Chrome_Param =
  3.     '{"capabilities": {"firstMatch": [{}], "alwaysMatch": {"browserName": "chrome",'
  4.     + ' "platformName": "any", "goog:chromeOptions": {"extensions": [],'
  5.         + ' "args": []}}},'  // visible mode
  6. //      + ' "args": ["--headless","--disable-gpu"]}}},' // hidden mode
  7. //      + ' "args": ["--user-data-dir=d:/temp/lazSelenium"]}}},' // userdir
  8.     + ' "desiredCapabilities": {"browserName": "chrome", "version": "", "platform": "ANY",'
  9.     + ' "goog:chromeOptions": {"extensions": [], "args": []}}}';  
with :
Code: Pascal  [Select][+][-]
  1. const
  2.     Chrome_Param =
  3.     '{"capabilities": {"firstMatch": [{}], "alwaysMatch": {"browserName": "chrome",'
  4.     + ' "platformName": "any", "goog:chromeOptions": {"extensions": [],'
  5. //      + ' "args": []}}},'  // visible mode
  6. //      + ' "args": ["--headless","--disable-gpu"]}}},' // hidden mode
  7.          + ' "args": ["--user-data-dir=d:/temp/lazSelenium"]}}},' // userdir
  8.     + ' "desiredCapabilities": {"browserName": "chrome", "version": "", "platform": "ANY",'
  9.     + ' "goog:chromeOptions": {"extensions": [], "args": []}}}';  

and adjust the user-data-dir to a read/write directory in your system.

« Last Edit: September 24, 2021, 04:49:40 pm by Jurassic Pork »
Jurassic computer : Sinclair ZX81 - Zilog Z80A à 3,25 MHz - RAM 1 Ko - ROM 8 Ko

Fahmy Rofiq

  • Jr. Member
  • **
  • Posts: 83
Re: Browser Automation with selenium
« Reply #24 on: September 24, 2021, 04:57:10 pm »
Thanks, thats work.
Lazarus Trunk + FPC Fixes 32bit
Windows 10 x64

lazzaro2023

  • New Member
  • *
  • Posts: 26
Re: Browser Automation with selenium
« Reply #25 on: February 10, 2023, 06:43:02 pm »
Hi Jurassic Pork,  thanks for WebDriver4L.  I have downloaded your demo to my platform, which is  Laz 2.0.10  FPC 3.2 Windows Server 2016.  I have downloaded chromedriver for 92.  (which matches my Chrome version).

When I compile and run the demo, the main window is displayed as expected.  However, when I click either of the buttons, I get the following error message:


Project demoWebDriver4L raised exception class 'EJSON' with message:
Unknown object member: "sessionId"

 At address 100176865
To resolve the error, the version of chromedriver.exe must be the same as Google Chrome.

TRon

  • Hero Member
  • *****
  • Posts: 2434
Re: Browser Automation with selenium
« Reply #26 on: June 02, 2023, 07:29:31 am »
Thank you for yet another great project Jurassic Pork.

Small issue to mention: Unit WebDriver4L function TFireFoxDriver.NewSession const Firefox_Param currently reads:

Code: Pascal  [Select][+][-]
  1. const
  2.   Firefox_Param =
  3.     '{"capabilities": {"firstMatch": [{}], "alwaysMatch": {"browserName": "firefox", '
  4.     + '"acceptInsecureCerts": false, "moz:firefoxOptions": ' +
  5. //    '{"binary": "%s"}}},' +  // visible mode
  6.     '{"binary": "%s",args:"-headless"}}},' +  // hidden mode
  7.     '"desiredCapabilities": {"browserName": "firefox", "acceptInsecureCerts": false, "moz:firefoxOptions": {"binary": "%s"}}}';
  8.  

According to https://developer.mozilla.org/en-US/docs/Web/WebDriver/Capabilities/firefoxOptions that should read:

Code: Pascal  [Select][+][-]
  1. const
  2.   Firefox_Param =
  3.     '{"capabilities": {"firstMatch": [{}], "alwaysMatch": {"browserName": "firefox", '
  4.     + '"acceptInsecureCerts": false, "moz:firefoxOptions": ' +
  5. //    '{"binary": "%s"}}},' +  // visible mode
  6.     '{"binary": "%s","args":["-headless"]}}},' +  // hidden mode
  7.     '"desiredCapabilities": {"browserName": "firefox", "acceptInsecureCerts": false, "moz:firefoxOptions": {"binary": "%s"}}}';
  8.  
  9.  

Note:
Code: [Select]
args:"-headless" vs "args":["-headless"]
When fix not applied the geckodriver will report back that the JSON request is invalid.



When using the function FindElementsByClassName the gecko driver/server reports:

Code: [Select]
1685771451493 webdriver::server DEBUG -> POST /session/91cf59aa-1503-4a17-9bbe-ffa5728fa2ee/elements { "value" : "page-title-text", "sessionid" : "91cf59aa-1503-4a17-9bbe-ffa5728fa2ee", "using" : "class name" }
1685771451493 webdriver::server DEBUG <- 400 Bad Request {"value":{"error":"invalid argument","message":"unknown variant `class name`, expected one of `css selector`, `link text`, `partial link text`, `tag name`, `xpath` at line 1 column 107","stacktrace":""}}

Apparently that is not supported.
« Last Edit: June 03, 2023, 07:58:36 am by TRon »

 

TinyPortal © 2005-2018