[solved] How to download a file from the internet by code?

[Nicole]

I have a link to an pdf file in the internet. It is posted every day new.
I need this in my software.

Not sure, but I think with TLS 1.3 on side of the server.

How can I download the file?
In Delphi it was very hard and ceased working after the update from 1.2. to 1.3. (not exactly sure about the version)

[Handoko]

You can download the demo "File Download" in the Networking category in the link below:
https://wiki.freepascal.org/Portal:HowTo_Demos#Networking

[wp]

Sorry this is a very advanced sample, not good for a beginner.

For a beginner I'd recommend the attached simple code which works for most standard downloads on Windows and Linux (probably also on mac, not tested).

[alaa123456789]

hope this help
https://youtu.be/O4Mj76O1-SQ

[Nicole]

@wp: Thank you so much for this project!
It worked at sudden,- although not (now) for me:

Perhaps about the history: I could download a pdf-file by Delphi as file. Then I used by Delphi a command prompt pdf converter which transformed it to ASCII. This ASCII I read by Delphi "by foot" and hoped, the layout will never change.
Under the line I had no work with it, everything was done in the background of my software.

So the first step is done,
I changed the link-address
clicked at the button
and the memo is filled.

Unfortunately, the memo makes no sense.
There are 2 ways out:
Either I do save the download link into a file, not into a stream.
Or what I would prefer (because it would make me independant of this command line tool software): there is a way to convert the pdf info ASCII by Lazarus. There shall be a pdf-tool somewhere, but I am not sure about the use and which is best.

So this is, what the project one gives me, when I try to grab my file:

%PDF-1.3
1 0 obj
[/PDF /Text /ImageB /ImageC /ImageI]
endobj
7 0 obj
<< /Length 10707 /Filter /FlateDecode >> stream
x?}ko?H???
???;ئ?~,?v?nYrK??????R??ܳ?~3?,f?±?ڢ??)?8L?U?????rȋ?k?6??"?Ϳ7?????T?u]gE?5??Y???/?5??.+?1o?l󡱠?6?Ua_`~????1??^|_gC>?cv?9+??uS?ySe7???۪???/?????}???n?3??k???~??c?۲?Mg??}?L{?4??},?!??*?????>{????9??7yE?ؚ7m????ھ@???%????6????v?}??1??????"ļ?Ð?/?Ȫj?e??
v?y?2o?*x?J??Z+??x??CUW]??U׭}m\?@#??|n@??yG??`E?6?}yS?M?My??d*??Φ?8?5??#?L??K//o??W?no׿<d/?F?????q}w???M?Iq?}???|?a|?.?<
?*/?ۛ?p??N?????e??q?X?????oӰ?ڍ+Y?f??mj?ާ?}?.?.??F?;?`???D\??-?bѿ(?UuX??z?????I5???9(묃?u?w9??ƕ?߇e?{h?FSՖ?23M}?i?O?&?Hhjh*M??o?qMe?   ?S??s?5յ?s?v??b?X?4?Nr$??2I\k??4}??]????,?P[?A?с?͊2?L?Oڱ?FSE???Es$<T??֘l??Jr$UA|hTՅٴ3UU??*ϑR?
?Re??8W??*?H+!?ߕ?6B??7?8?$???Ǘ??԰??ƒ???vP?fX?2?{(??S???ѡ???&1?P??(H?(??D?Ky?]??)Q*Q8?E-)lz?(? Q?Gr?n??\*s?6?9v???GJ?????K???B??GP?d?h?@1U?Z]???*~?!?!*UȑT!?Q??*?1"U?#?
SD?
??j?Xa?T!?ň?9?6F*3V???V???Һ??Ë"?XN5??#?#?S???M~B???҄)M?M~

[Nicole]

so in other words, I would need this:

   Memo1.Lines.LoadFrom_aPDF_Stream(stream);

[BobDog]

Windows only.
Used a .csv site to test, please change to your site and your filetype.

Code: Pascal  [Select][+][-]

1. uses
2. sysutils;
3.  
4.  
5.  function  shell(s:pchar):integer ; cdecl external 'msvcrt.dll' name 'system';
6.  
7.  function filelength(filename:ansistring):longword;
8. Var F : File of byte;
9. var L:longword;
10. begin
11. Assign (F,filename);
12.   Reset (F);
13.   L:=FileSize(F);
14.   Close (F);
15.   exit(L);
16. end;
17.  
18.  procedure loadfile(var content: ansistring;filename:ansistring);
19.    Var Fin : File;
20.    Var x:int32;
21.    begin
22.    x:=filelength(filename);
23.    setlength(content,x);
24.    Assign (Fin,filename);
25.    Reset (Fin,x);
26.    BlockRead (Fin,content[1],1);
27.    close(fin);
28. end;
29.  
30. procedure download(site:ansiString;newfile:ansistring);
31. var
32. f:ansistring;
33. content:ansistring='';
34. begin
35. writeln('Please wait -- downloading from:');
36. writeln(site);  
37. if fileexists(GetCurrentDir +'/'+ newfile) then deletefile(GetCurrentDir +'/'+ newfile);
38. f:='powershell Invoke-WebRequest '+site+ '/ -OutFile '+ GetCurrentDir +'/'+ newfile;
39.    If (pos(' ',site) <>0) Then site:=Chr(34)+site+Chr(34);
40.   shell (@f[1]);
41.  if fileexists(newfile) then loadfile(content,newfile);
42.   If (Length(content)<>0) Then
43.     Shell(pchar( Chr(34) +GetCurrentDir +  '\'+newfile + Chr(34)));
44.   end;
45.  
46. var
47. c:ansistring= 'https://promo.betfair.com/betfairsp/prices/dwbfpricesukwin13072020.csv';
48.  
49. begin
50. download(c,'New.csv');
51. writeln('press return to finish . . .');
52. readln;
53. end.
54.  
55.  

Tested Win 10 here.
Good luck.

[BobDog]

Here is another Windows method
As before the test site is for a csv file, and the downloaded file is New.csv in the current directory.
(you should change to your site and maybe use New.pdf or something similar)

Code: Pascal  [Select][+][-]

1.  
2.  
3. uses
4. sysutils;
5.  
6.  function  shell(s:pchar):integer ; cdecl external 'msvcrt.dll' name 'system';
7.  
8. function filelength(filename:ansistring):longword;
9. Var F : File of byte;
10. var L:longword;
11. begin
12. Assign (F,filename);
13.   Reset (F);
14.   L:=FileSize(F);
15.   Close (F);
16.   exit(L);
17. end;
18.  
19. procedure loadfile(var content: ansistring;filename:ansistring);
20.    Var Fin : File;
21.    Var x:int32;
22.    begin
23.    x:=filelength(filename);
24.    setlength(content,x);
25.    Assign (Fin,filename);
26.    Reset (Fin,x);
27.    BlockRead (Fin,content[1],1);
28.    close(fin);
29. end;
30.  
31.  
32. procedure download(site:ansiString;newfile:ansistring);
33. var
34. f,content:ansistring;
35. begin
36. content:='';
37. if fileexists(newfile) then deletefile(newfile);
38.   If (pos(' ',site) <>0) Then site:=Chr(34)+site+Chr(34);
39.   f:='type nul > ' +GetCurrentDir +'\'+newfile; // create empty file
40.   shell (pchar(f));
41.   shell('bitsadmin /reset');
42.   shell ('bitsadmin /create mydownload');
43.   f:='bitsadmin  /transfer mydownload  /download  /priority normal /DYNAMIC ' +  site +' '+ Chr(34) + GetCurrentDir +'\'+ newfile +Chr(34);
44.   Shell(pchar(f));
45.   shell ('bitsadmin /complete mydownload');
46.   if (filelength(newfile)<>0) then loadfile(content,newfile) else exit;
47.   f:=Chr(34) +GetCurrentDir +  '\'+newfile + Chr(34);
48.   If (Length(content)>4) Then
49.     Shell (pchar(f));
50. End;
51.  
52.  
53. var
54. site:ansistring= 'https://promo.betfair.com/betfairsp/prices/dwbfpricesukwin13072020.csv';
55.  
56. begin
57. shell('Title Please wait a few seconds until Job completed');
58. download(site,'New.csv');
59. writeln('Press return to finish . . .');
60. readln;
61. end.
62.  
63.  

[Nicole]

Thank you for the answer.
I read it several times, because I am new to Lazarus.

Where can I give my pdf file as input?
The file I download is a pdf.

[BobDog]

I tried the site given by wp, using the bitsadmin method.
site given is 'https://www.ietf.org/rfc/rfc2445.txt'
so
download(site,'New.txt');
and windows opened this text file via notepad
i.e.
Shell (pchar(f))
is the command to open the file at the end of the procedure.
If your download site is for  .pdf then
download(site,'New.pdf');
and windows should open the New.pdf file with your .pdf reader.
That's all my code does.
from pdf to ascii is another story.
Please note my methods are just pascal, and not based on the lazarus ide, I use Geany as an ide/editor.
But the code should run in the Lazarus ide as a simple program.

[Nicole]

So I thank you both for this very helpful answers and make the thread as solved.

The download of the project is highly recommended!

[wp]

To complete my reply #2 with respect to downloading pdf files into a memo I am attaching a simple solution which delegates the extraction of the text from the pdf file to the tool "pdftotext.exe" (https://www.xpdfreader.com/pdftotext-man.html, binary for Windows included in the attachment).