Indy 10: TCP/HTTPS downloads management OnWork

[torbente]

Hi:
Found a lot of articles related with this using HHTP, but bot a single one using TCP...

Code: Pascal  [Select][+][-]

1. var
2.    ClientChannel : TIdTCPClient;
3.  
4. function GetFile(filename:string):boolean;
5. var
6.   AFileStream : TFileStream;
7. Begin
8. result := false;
9. ClientChannel.Host:=PredefinedHost;
10. ClientChannel.Port:=PredefinedPort;
11. ClientChannel.ConnectTimeout:= 1000;
12. ClientChannel.ReadTimeout:=500;
13. //ClientChannel.OnWork:= ??; Whats going here?
14. TRY
15. ClientChannel.Connect;
16. ClientChannel.IOHandler.WriteLn('GETFILE:'+filename);
17. AFileStream := TFileStream.Create(filename, fmCreate);
18.    TRY
19.    ClientChannel.IOHandler.ReadStream(AFileStream);
20.    result := true;
21.    EXCEPT on E:Exception do
22.       begin
23.       // Report error downloading the file
24.       end;
25.    END{Try};
26. FINALLY
27. AFileStream.Free;
28. ClientChannel.Disconnect();
29. END{try};
30. End;  

This has worked for me for years, but now i need be able to "follow" the progress of the download (maybe a percentage of the completed download?) to be able to abort it (and also in case the user closes the app during a long download)

Thanks in advance.

[Remy Lebeau]

Found a lot of articles related with this using HHTP, but bot a single one using TCP...

The OnWork events work exactly the same way in all of Indy's components.

Code: Pascal  [Select][+][-]

1. //ClientChannel.OnWork:= ??; Whats going here?

An event handler, obviously, for example:

Code: Pascal  [Select][+][-]

1. procedure TSomeClass.OnWorkHandler(ASender: TObject; AWorkMode: TWorkMode; AWorkCount: Int64);
2. begin
3.   ...
4. end;
5.  
6. var InstanceOfSomeClass: TSomeClass;
7. ...
8. ClientChannel.OnWork := InstanceOfSomeClass.OnWorkHandler;

Alternatively, if you don't have an object instance to put the handler into, you can use a standalone procedure, but you will have to tweak the  function signature and then use the System.TMethod record to assign it to the event, eg:

Code: Pascal  [Select][+][-]

1. // note the extra ASelf parameter!...
2. procedure OnWorkHandler(ASelf: Pointer; ASender: TObject; AWorkMode: TWorkMode; AWorkCount: Int64);
3. begin
4.   ...
5. end;
6.  
7. var
8.   M: TMethod;
9. begin
10.   M.Data := nil; // Or whatever you want the ASelf parameter to point at...
11.   M.Code := @OnWorkHandler;
12.   ClientChannel.OnWork := TWorkEvent(M);
13. end;

Or:

Code: Pascal  [Select][+][-]

1. var
2.   Evt: TWorkEvent;
3. begin
4.   TMethod(Evt).Data := nil; // Or whatever you want the ASelf parameter to point at...
5.   TMethod(Evt).Code := @OnWorkHandler;
6.   ClientChannel.OnWork := Evt;
7. end;

Note, you will also need a handler for the OnWorkBegin event, too.  The OnWork event only tells you how many bytes have been transferred so far, but the OnWorkBegin event tells you how many bytes are expected, if known up front:

Code: Pascal  [Select][+][-]

1. procedure TSomeClass.OnWorkBeginHandler(ASender: TObject; AWorkMode: TWorkMode; AWorkCountMax: Int64);
2. begin
3.   ...
4. end;
5.  
6. var InstanceOfSomeClass: TSomeClass;
7. ...
8. ClientChannel.OnWorkBegin := InstanceOfSomeClass.OnWorkBeginHandler;

Or:

Code: Pascal  [Select][+][-]

1. // note the extra ASelf parameter!...
2. procedure OnWorkBeginHandler(ASelf: Pointer; ASender: TObject; AWorkMode: TWorkMode; AWorkCountMax: Int64);
3. begin
4.   ...
5. end;
6.  
7. var
8.   M: TMethod;
9. begin
10.   M.Data := nil; // Or whatever you want the ASelf parameter to point at...
11.   M.Code := @OnWorkBeginHandler;
12.   ClientChannel.OnWorkBegin := TWorkBeginEvent(M);
13. end;

Or:

Code: Pascal  [Select][+][-]

1. var
2.   Evt: TWorkBeginEvent;
3. begin
4.   TMethod(Evt).Data := nil; // Or whatever you want the ASelf parameter to point at...
5.   TMethod(Evt).Code := @OnWorkBeginHandler;
6.   ClientChannel.OnWorkBegin := Evt;
7. end;

Code: Pascal  [Select][+][-]

1. ClientChannel.IOHandler.ReadStream(AFileStream);

The default parameters of ReadStream() are AByteCount=-1 and AReadUntilDisconnect=False.  Under that specific combination, ReadStream() will first attempt to read an integer from the connection, indicating the stream's total byte size, before then reading that many bytes for the actual stream data.  That byte count will be passed to the OnWorkBegin event, and is expected to be transmitted as either a 4-byte Integer or an 8-byte Int64 depending on the IOHandler's LargeStream property.

This has worked for me for years, but now i need be able to "follow" the progress of the download (maybe a percentage of the completed download?) to be able to abort it (and also in case the user closes the app during a long download)

Being able to calculate a percentage requires knowing how many bytes are actually in the file to begin with.  Is your server sending that byte count up front?

[torbente]

I was able to find the way, using a design-time client and assigning the onworkbegin and onwork elements.

Since it was a download from a peer using a TCP connection, it do not have problems.

Now (i believe i could continue here since the post title perfectly fits) i want download (with onwork following) a file from github (so users can get last releases from there directly)

Code: Pascal  [Select][+][-]

1. IdHTTPUpdate: TIdHTTP;
2.  
3. Function GetLastVerZipFile(filename:string):boolean;
4. var
5.   MS: TMemoryStream;
6. Begin
7. MS := TMemoryStream.Create;
8. TRY
9.    TRY
10.    Form1.IdHTTPUpdate.HandleRedirects:=true;
11.    Form1.IdHTTPUpdate.get('https://github.com/owner/repository/releases/download/version/xxx.zip', MS);
12.    MS.SaveToFile(UpdatesDirectory+'filename.zip');
13.    EXCEPT ON E:Exception do
14.       begin
15.       ShowMessage('Error downloading last release: '+E.Message);
16.       end;
17.    END{Try};
18. FINALLY
19. MS.Free;
20. END{try};
21. End;  

Im receiving invalid SSL protocol error. I placed the SSL files sin the same folder than the app but it neither worked. Any idea?

[Remy Lebeau]

Im receiving invalid SSL protocol error.

What is the EXACT error message, verbatim?  And how did you (or, did you, at all?) configure the SSLIOHandler assigned to the TIdHTTP.IOHandler property?

I placed the SSL files sin the same folder than the app but it neither worked.

Are you referring to the two OpenSSL DLLs?  Which version of the DLLs are you using?

[torbente]

  Sorry, forgot to mention it...

What is the EXACT error message, verbatim?

Error downloading last release: Error connecting with SSL.
error:1409442E:SSL routines:ssl3_read_bytes:tlsv1 alert protocol version

And how did you (or, did you, at all?) configure the SSLIOHandler assigned to the TIdHTTP.IOHandler property?

Well, im implemented this...

Code: Pascal  [Select][+][-]

1. IdSSLIOHandler:= TIdSSLIOHandlerSocketOpenSSL.Create;
2. ...
3. Form1.IdHTTPUpdate.IOHandler:=IdSSLIOHandler;

And i was receiving this error:

Error downloading last release: Socket Error # 10060
Connection timed out.

Are you referring to the two OpenSSL DLLs?  Which version of the DLLs are you using?

Yes, ssleay32.dll and libeay32.dll, version 1.0.2

[Remy Lebeau]

Error downloading last release: Error connecting with SSL.
error:1409442E:SSL routines:ssl3_read_bytes:tlsv1 alert protocol version

By default, Indy enables TLS 1.0 only (see this ticket). Chances are, the site you are accessing wants TLS 1.1+ instead.

Well, im implemented this...

Code: Pascal  [Select][+][-]

1. IdSSLIOHandler:= TIdSSLIOHandlerSocketOpenSSL.Create;
2. ...
3. Form1.IdHTTPUpdate.IOHandler:=IdSSLIOHandler;

That is fine (just be sure to call IdSSLIOHandler.Free() when you are done using it, since you are not assigning an Owner to it).

Are you setting the IdSSLIOHandler.SSLOptions.SSLVersions property?  By default, it is set to [sslvTLSv1], try adding [sslvTLSv1_1,sslvTLSv1_2] to it, eg:

Code: Pascal  [Select][+][-]

1. IdSSLIOHandler := TIdSSLIOHandlerSocketOpenSSL.Create(Form1.IdHTTPUpdate);
2.  
3. IdSSLIOHandler.SSLOptions.SSLVersions := [sslvTLSv1,sslvTLSv1_1,sslvTLSv1_2];
4. // or: ...SSLVersions := [sslvTLSv1..sslvTLSv1_2];
5.  
6. // or: ...SSLVersions := ...SSLVersions + [sslvTLSv1_1,sslvTLSv1_2];
7. // or: ...SSLVersions := ...SSLVersions + [sslvTLSv1_1..sslvTLSv1_2];
8.  
9. // or:
10. // ...SSLVersions := [sslvSSLv23]; // enable all supported versions, then...
11. // ...SSLVersions := ...SSLVersions - [sslvSSLv2,sslvSSLv3]; // remove unwanted versions
12.  
13. // or:
14. // ...Method := sslvSSLv23; // enable all supported versions, then...
15. // ...SSLVersions := ...SSLVersions - [sslvSSLv2,sslvSSLv3]; // remove unwanted versions
16.  
17. ...

Yes, ssleay32.dll and libeay32.dll, version 1.0.2

That should be fine then.

[torbente]

That is fine (just be sure to call IdSSLIOHandler.Free() when you are done using it, since you are not assigning an Owner to it).

Yep. I check all the create calls in the code to verify that all are appropiately freed 

Code: Pascal  [Select][+][-]

1. IdSSLIOHandler.SSLOptions.SSLVersions := [sslvTLSv1,sslvTLSv1_1,sslvTLSv1_2];

Worked perfectly! I did was now aware of the TLS thing.
Thanks a lot.