Recent

Author Topic: [solved] How to download a file from the internet by code?  (Read 821 times)

Nicole

  • Sr. Member
  • ****
  • Posts: 355
[solved] How to download a file from the internet by code?
« on: August 11, 2022, 05:24:35 pm »
I have a link to an pdf file in the internet. It is posted every day new.
I need this in my software.

Not sure, but I think with TLS 1.3 on side of the server.

How can I download the file?
In Delphi it was very hard and ceased working after the update from 1.2. to 1.3. (not exactly sure about the version)
« Last Edit: August 17, 2022, 08:49:57 am by Nicole »

Handoko

  • Hero Member
  • *****
  • Posts: 4712
  • My goal: build my own game engine using Lazarus
Re: How to download a file from the internet by code?
« Reply #1 on: August 11, 2022, 05:32:27 pm »
You can download the demo "File Download" in the Networking category in the link below:
https://wiki.freepascal.org/Portal:HowTo_Demos#Networking

wp

  • Hero Member
  • *****
  • Posts: 10060
Re: How to download a file from the internet by code?
« Reply #2 on: August 11, 2022, 06:17:29 pm »
Sorry this is a very advanced sample, not good for a beginner.

For a beginner I'd recommend the attached simple code which works for most standard downloads on Windows and Linux (probably also on mac, not tested).

alaa123456789

  • Full Member
  • ***
  • Posts: 232
  • Try your Best to learn & help others
    • youtube:
Re: How to download a file from the internet by code?
« Reply #3 on: August 11, 2022, 08:43:13 pm »

Nicole

  • Sr. Member
  • ****
  • Posts: 355
Re: How to download a file from the internet by code?
« Reply #4 on: August 12, 2022, 10:19:44 am »
@wp: Thank you so much for this project!
It worked at sudden,- although not (now) for me:

Perhaps about the history: I could download a pdf-file by Delphi as file. Then I used by Delphi a command prompt pdf converter which transformed it to ASCII. This ASCII I read by Delphi "by foot" and hoped, the layout will never change.
Under the line I had no work with it, everything was done in the background of my software.

So the first step is done,
I changed the link-address
clicked at the button
and the memo is filled.

Unfortunately, the memo makes no sense.
There are 2 ways out:
Either I do save the download link into a file, not into a stream.
Or what I would prefer (because it would make me independant of this command line tool software): there is a way to convert the pdf info ASCII by Lazarus. There shall be a pdf-tool somewhere, but I am not sure about the use and which is best.

So this is, what the project one gives me, when I try to grab my file:

%PDF-1.3
1 0 obj
[/PDF /Text /ImageB /ImageC /ImageI]
endobj
7 0 obj
<< /Length 10707 /Filter /FlateDecode >> stream
x?}ko?H??????;ئ?~, ?v?nYrK??????R??ܳ?~3?,f?±?ڢ??)?8L?U?????rȋ?k?6??"?Ϳ7?????T?u]gE?5??Y???/?5??.+?1o?l󡱠?6?Ua_`~????1??^|_gC>?cv?9+?? uS?ySe7???۪???/?????}???n?3??k???~??c?۲?Mg??}?L{?4??},?!??*?????>{????9??7yE?ؚ7m????ھ@???%????6????v?}??1??????"ļ?Ð?/?Ȫj?e??
v?y?2o?*x?J??Z+??x??CUW]??U׭}m\?@#??|n@??yG??`E?6?}yS?M?My??d*??Φ?8?5??#?L??K//o??W?no׿<d/?F?????q}w???M?Iq? }???|?a|?.?<?*/?ۛ?p??N?????e??q?X?????oӰ? ڍ+Y?f??mj?ާ?}?.?.??F?;?`???D\??-?bѿ(?UuX??z?????I5???9(묃?u?w9??ƕ?߇e?{h?FSՖ?23M}?i?O?&?Hhjh*M??o?qMe?   ?S??s?5յ?s?v??b?X?4?Nr$? ?2I\k??4}??]????,?P[?A?с?͊2?L?Oڱ?FSE???Es$<T??֘l??Jr$UA|hTՅٴ3UU??*ϑR??Re??8W??*?H+!?ߕ?6B??7?8?$???Ǘ??԰??ƒ???vP?fX?2?{(??S???ѡ???&1?P??(H?(??D?Ky?]??)Q*Q8?E-)lz?(? Q?Gr?n??\*s?6?9v???GJ?????K???B??GP?d?h?@1U?Z]???*~?!?!*UȑT!?Q??*?1"U?#?
SD?
??j?Xa?T!?ň?9?6F*3V? ??V???Һ??Ë"?XN5??#?#?S???M~B???҄)M?M~


Nicole

  • Sr. Member
  • ****
  • Posts: 355
Re: How to download a file from the internet by code?
« Reply #5 on: August 12, 2022, 10:26:10 am »
so in other words, I would need this:

   Memo1.Lines.LoadFrom_aPDF_Stream(stream);

BobDog

  • Sr. Member
  • ****
  • Posts: 328
Re: How to download a file from the internet by code?
« Reply #6 on: August 12, 2022, 03:07:05 pm »

Windows only.
Used a .csv site to test, please change to your site and your filetype.
Code: Pascal  [Select][+][-]
  1. uses
  2. sysutils;
  3.  
  4.  
  5.  function  shell(s:pchar):integer ; cdecl external 'msvcrt.dll' name 'system';
  6.  
  7.  function filelength(filename:ansistring):longword;
  8. Var F : File of byte;
  9. var L:longword;
  10. begin
  11. Assign (F,filename);
  12.   Reset (F);
  13.   L:=FileSize(F);
  14.   Close (F);
  15.   exit(L);
  16. end;
  17.  
  18.  procedure loadfile(var content: ansistring;filename:ansistring);
  19.    Var Fin : File;
  20.    Var x:int32;
  21.    begin
  22.    x:=filelength(filename);
  23.    setlength(content,x);
  24.    Assign (Fin,filename);
  25.    Reset (Fin,x);
  26.    BlockRead (Fin,content[1],1);
  27.    close(fin);
  28. end;
  29.  
  30. procedure download(site:ansiString;newfile:ansistring);
  31. var
  32. f:ansistring;
  33. content:ansistring='';
  34. begin
  35. writeln('Please wait -- downloading from:');
  36. writeln(site);  
  37. if fileexists(GetCurrentDir +'/'+ newfile) then deletefile(GetCurrentDir +'/'+ newfile);
  38. f:='powershell Invoke-WebRequest '+site+ '/ -OutFile '+ GetCurrentDir +'/'+ newfile;
  39.    If (pos(' ',site) <>0) Then site:=Chr(34)+site+Chr(34);
  40.   shell (@f[1]);
  41.  if fileexists(newfile) then loadfile(content,newfile);
  42.   If (Length(content)<>0) Then
  43.     Shell(pchar( Chr(34) +GetCurrentDir +  '\'+newfile + Chr(34)));
  44.   end;
  45.  
  46. var
  47. c:ansistring= 'https://promo.betfair.com/betfairsp/prices/dwbfpricesukwin13072020.csv';
  48.  
  49. begin
  50. download(c,'New.csv');
  51. writeln('press return to finish . . .');
  52. readln;
  53. end.
  54.  
  55.  
Tested Win 10 here.
Good luck.

BobDog

  • Sr. Member
  • ****
  • Posts: 328
Re: How to download a file from the internet by code?
« Reply #7 on: August 13, 2022, 04:14:15 pm »

Here is another Windows method
As before the test site is for a csv file, and the downloaded file is New.csv in the current directory.
(you should change to your site and maybe use New.pdf or something similar)
Code: Pascal  [Select][+][-]
  1.  
  2.  
  3. uses
  4. sysutils;
  5.  
  6.  function  shell(s:pchar):integer ; cdecl external 'msvcrt.dll' name 'system';
  7.  
  8. function filelength(filename:ansistring):longword;
  9. Var F : File of byte;
  10. var L:longword;
  11. begin
  12. Assign (F,filename);
  13.   Reset (F);
  14.   L:=FileSize(F);
  15.   Close (F);
  16.   exit(L);
  17. end;
  18.  
  19. procedure loadfile(var content: ansistring;filename:ansistring);
  20.    Var Fin : File;
  21.    Var x:int32;
  22.    begin
  23.    x:=filelength(filename);
  24.    setlength(content,x);
  25.    Assign (Fin,filename);
  26.    Reset (Fin,x);
  27.    BlockRead (Fin,content[1],1);
  28.    close(fin);
  29. end;
  30.  
  31.  
  32. procedure download(site:ansiString;newfile:ansistring);
  33. var
  34. f,content:ansistring;
  35. begin
  36. content:='';
  37. if fileexists(newfile) then deletefile(newfile);
  38.   If (pos(' ',site) <>0) Then site:=Chr(34)+site+Chr(34);
  39.   f:='type nul > ' +GetCurrentDir +'\'+newfile; // create empty file
  40.   shell (pchar(f));
  41.   shell('bitsadmin /reset');
  42.   shell ('bitsadmin /create mydownload');
  43.   f:='bitsadmin  /transfer mydownload  /download  /priority normal /DYNAMIC ' +  site +' '+ Chr(34) + GetCurrentDir +'\'+ newfile +Chr(34);
  44.   Shell(pchar(f));
  45.   shell ('bitsadmin /complete mydownload');
  46.   if (filelength(newfile)<>0) then loadfile(content,newfile) else exit;
  47.   f:=Chr(34) +GetCurrentDir +  '\'+newfile + Chr(34);
  48.   If (Length(content)>4) Then
  49.     Shell (pchar(f));
  50. End;
  51.  
  52.  
  53. var
  54. site:ansistring= 'https://promo.betfair.com/betfairsp/prices/dwbfpricesukwin13072020.csv';
  55.  
  56. begin
  57. shell('Title Please wait a few seconds until Job completed');
  58. download(site,'New.csv');
  59. writeln('Press return to finish . . .');
  60. readln;
  61. end.
  62.  
  63.  

Nicole

  • Sr. Member
  • ****
  • Posts: 355
Re: How to download a file from the internet by code?
« Reply #8 on: August 16, 2022, 09:35:56 pm »
Thank you for the answer.
I read it several times, because I am new to Lazarus.

Where can I give my pdf file as input?
The file I download is a pdf.


BobDog

  • Sr. Member
  • ****
  • Posts: 328
Re: How to download a file from the internet by code?
« Reply #9 on: August 17, 2022, 06:40:43 am »
I tried the site given by wp, using the bitsadmin method.
site given is 'https://www.ietf.org/rfc/rfc2445.txt'
so
download(site,'New.txt');
and windows opened this text file via notepad
i.e.
Shell (pchar(f))
is the command to open the file at the end of the procedure.
If your download site is for  .pdf then
download(site,'New.pdf');
and windows should open the New.pdf file with your .pdf reader.
That's all my code does.
from pdf to ascii is another story.
Please note my methods are just pascal, and not based on the lazarus ide, I use Geany as an ide/editor.
But the code should run in the Lazarus ide as a simple program.




« Last Edit: August 17, 2022, 06:43:34 am by BobDog »

Nicole

  • Sr. Member
  • ****
  • Posts: 355
Re: How to download a file from the internet by code?
« Reply #10 on: August 17, 2022, 08:46:41 am »
So I thank you both for this very helpful answers and make the thread as solved.

The download of the project is highly recommended!

wp

  • Hero Member
  • *****
  • Posts: 10060
Re: [solved] How to download a file from the internet by code?
« Reply #11 on: August 17, 2022, 07:58:04 pm »
To complete my reply #2 with respect to downloading pdf files into a memo I am attaching a simple solution which delegates the extraction of the text from the pdf file to the tool "pdftotext.exe" (https://www.xpdfreader.com/pdftotext-man.html, binary for Windows included in the attachment).
« Last Edit: August 17, 2022, 08:16:20 pm by wp »

 

TinyPortal © 2005-2018