Recent

Author Topic: How to get data in html from this Url??  (Read 2190 times)

lestroso

  • Full Member
  • ***
  • Posts: 123
    • FaSasoftware
How to get data in html from this Url??
« on: May 26, 2021, 11:27:22 pm »
Hi, i need to retrieve data from this url:" https://www.betexplorer.com/next/soccer/"

Because i need to make some operations from that url..

I need to use a command like "wordoffset" to cut the html text imported into a variable until find some team soccer...like "Germany: Bundesliga" and then get all the teams for that type of soccer Team...and put it into a field...

Can you help me with some fresh code example please??Can you help me??

Thanks A lot!! Lestroso  :-[

kapibara

  • Hero Member
  • *****
  • Posts: 533
Re: How to get data in html from this Url??
« Reply #1 on: May 27, 2021, 12:02:59 am »
No much code, but a general idea how I usually do it:

Read all of the html content into a TStringList.
Create a function that finds the first entry, then
another similar function that find a next entry

Then run the findnext routine in a loop until it returns false.

Just look for suitable keywords inside the html and strip out the data you need.

Code: Pascal  [Select][+][-]
  1. if findfirstrecord then
  2.   GrabResults()
  3. else
  4.   Exit;
  5.  
Code: Pascal  [Select][+][-]
  1. while findnextrecord do
  2. begin
  3.   GrabResults();
  4. end;
  5.  
« Last Edit: May 27, 2021, 04:04:15 am by kapibara »
Lazarus trunk / fpc 3.0.4 / Debian 10 - 64 bit

Jurassic Pork

  • Hero Member
  • *****
  • Posts: 987
Re: How to get data in html from this Url??
« Reply #2 on: May 27, 2021, 05:29:39 am »
hello,
Hi, i need to retrieve data from this url:" https://www.betexplorer.com/next/soccer/"

Because i need to make some operations from that url..

I need to use a command like "wordoffset" to cut the html text imported into a variable until find some team soccer...like "Germany: Bundesliga" and then get all the teams for that type of soccer Team...and put it into a field...

Can you help me with some fresh code example please??Can you help me??

Thanks A lot!! Lestroso  :-[

To do that you can try my library WebDriver4L using a webbrowser like chrome or firefox.

With this code :
Code: Pascal  [Select][+][-]
  1. procedure TForm1.Button1Click(Sender: TObject);
  2. var Robot : TWebDriver;
  3.     teams : TWebElements;
  4.     i : integer;
  5. begin
  6.   {$IFDEF WINDOWS}
  7.   Robot := TChromeDriver.Create(nil);
  8.   // Robot := TFireFoxDriver.Create(nil);
  9.   Robot.StartDriver(ExtractFileDir(Paramstr(0)) + '\chromedriver.exe');
  10.   // Robot.StartDriver(ExtractFileDir(Paramstr(0)) + '\geckodriver.exe');
  11.   {$ELSE}
  12.    // Robot := TChromeDriver.Create(nil);
  13.   Robot := TFireFoxDriver.Create(nil);
  14.   // Robot.StartDriver(ExtractFileDir(Paramstr(0)) + '/chromedriver');
  15.   Robot.StartDriver(ExtractFileDir(Paramstr(0)) + '/geckodriver');
  16.   {$ENDIF}
  17.   // Sleep(2000);
  18.   Robot.NewSession;
  19.   Robot.Implicitly_Wait(2000);
  20.   Robot.Set_Window_Size(1280, 1024);
  21.   Robot.GetURL('https://www.betexplorer.com/soccer/germany/bundesliga/');
  22.   teams := Robot.FindElementsByXPath('//span[@class="team_name_span"]');
  23.   For i:=0 to teams.Count - 1 do
  24.   begin
  25.    Memo1.Append(teams.Items[i].Text);
  26.   end;
  27.   Sleep(20000);
  28.   Robot.Quit;
  29. end;

i get what you can see in attachment.

Friendly, J.P
Jurassic computer : Sinclair ZX81 - Zilog Z80A à 3,25 MHz - RAM 1 Ko - ROM 8 Ko

wp

  • Hero Member
  • *****
  • Posts: 8574
Re: How to get data in html from this Url??
« Reply #3 on: May 27, 2021, 02:04:23 pm »
You can also solve this without using third-party software: Use unit fasthtmlparser which comes with FPC; it iterates through the html text character by character and fires an event OnFoundTag whenever a tag is seen and returns the text between the '<' and the '>'. And it fires an event OnFoundText whenever it sees a text node and returns the text between then '>' of the previous tag and the '<' of the following tag. Studying the nodes of the input text and introducing some state variables it is relatively straightforward to extract the desired information. Check out the attached demo. Note that you must make sure that the SSL libraries can be found - in Windows you normally must copy the libeay32.dll and ssleay32.dll into the exe folder. Note also that I did not manage to download from the given site using the fphttpclient which comes with FPC, and that I had to use synapse instead - so make sure that the synapse package is available.
Mainly Lazarus trunk / fpc 3.2.0 / all 32-bit on Win-10, but many more...

BeniBela

  • Hero Member
  • *****
  • Posts: 806
    • homepage
Re: How to get data in html from this Url??
« Reply #4 on: May 27, 2021, 03:03:28 pm »
I made my Internet Tools for such purposes:

Code: Pascal  [Select][+][-]
  1. uses simpleinternet,;
  2. writeln(process('https://www.betexplorer.com/soccer/germany/bundesliga', 'distinct-values(css(".in-match span"))').toJoinedString(lineEnding));
  3.  

 

TinyPortal © 2005-2018