Recent

Author Topic: Download form - which library to use?  (Read 17140 times)

mesiarm

  • New Member
  • *
  • Posts: 45
Download form - which library to use?
« on: June 27, 2013, 04:28:04 pm »
Hi. I want to make some download form with progress bar something like VLC player has  - when you click to download update. How to make it and which library (Synapse, Indy etc.) to use? I have Synapse,but i´m not sure if there is something to use progress bar and download big files (I want to download around 30 MB file).

Rails

  • Guest
Re: Download form - which library to use?
« Reply #1 on: June 27, 2013, 05:33:50 pm »
Use TProgressBar in ComCtrls.


Leledumbo

  • Hero Member
  • *****
  • Posts: 8835
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: Download form - which library to use?
« Reply #2 on: June 27, 2013, 05:43:09 pm »
All available networking libraries (fcl-net, synapse, lnet, indy) are AFAIK capable of doing this.
Synapse: http://synapse.ararat.cz/doku.php/public:howto:ftp_progressbar (applicable to THTTPSend, only change from DSock to Sock to get the socket instance).

mesiarm

  • New Member
  • *
  • Posts: 45
Re: Download form - which library to use?
« Reply #3 on: June 28, 2013, 10:05:27 pm »
thanks.... i tried it i made this program ...I don´t know.. how to continue....

i looked to this
http://www.ararat.cz/synapse/doku.php/public:howto:onstatus
i tried
Code: [Select]
http.Sock.OnStatus := SockPutCallBack(Sender: TObject,HR_ReadCount, value)
I´m not sure with parameters and I don´t know which value to use

Can you explain me a bit working of this program  please?

Code: [Select]
unit Unit1;

{$mode objfpc}{$H+}

interface

uses
  Classes, SysUtils, FileUtil, Forms, Controls, Graphics, Dialogs, ComCtrls,
  StdCtrls, blcksock, synafpc, synsock, synautil, synacode, synaip, httpsend;

type

  { TForm1 }

  TForm1 = class(TForm)
    httpclick: TButton;
    pr: TProgressBar;
    procedure httpclickClick(Sender: TObject);
  private
    { private declarations }
  public
    procedure SockPutCallBack(Sender: TObject;
  Reason: THookSocketReason; const Value: string);
    { public declarations }
  end;

var
  Form1: TForm1;
  CurrentBytes,TotalBytes : integer;
  http : THttpsend;
implementation

{$R *.lfm}



procedure TForm1.SockPutCallBack(Sender: TObject;
  Reason: THookSocketReason; const Value: string);
begin
  case Reason of
    HR_WriteCount: // HR_ReadCount // Value contains number of bytes
      begin
        inc(CurrentBytes, StrToIntDef(Value, 0)); // Increment uploaded bytes
        pr.Position := Round(1000 * (CurrentBytes / TotalBytes));
      end;
    HR_Connect: CurrentBytes := 0; // Reset
    (*
      else
        begin
          v := getEnumName(typeInfo(THookSocketReason),
            integer(Reason)) + ' ' + Value);
          MemoLog.Lines.Add(v); // Show log in a Memo field
        end;
    *)
  end;
end;
procedure TForm1.httpclickClick(Sender: TObject);
begin
  http := THTTPSend.Create;
    try
      http.Sock.OnStatus := SockPutCallBack;
      finally
        http.free;
      end;
end;

end.


Jurassic Pork

  • Hero Member
  • *****
  • Posts: 1290
Re: Download form - which library to use?
« Reply #4 on: June 29, 2013, 02:18:59 am »
hello,
1 - you must get the size of the file which you want to download before downloading, to have the TotalBytes value.
2 - The downloading procedure need to be in a thread else your Form will freeze while download.

here is an example  ( not sure, must be checked) :
Code: [Select]
unit Unit1;

{$mode objfpc}{$H+}

interface

uses
  Classes, SysUtils, FileUtil, Forms, Controls, Graphics,
  Dialogs, StdCtrls, ComCtrls, httpsend,blcksock,typinfo, synamisc;

type
   { THTTPDownload }

  THTTPDownload = class(TThread)
    private
      http : THTTPsend;
      TotalBytes    :  integer;
      CurrentBytes  :  integer;
      fURL: string;
      fFileName: string;
    procedure SocketGetCallBack(Sender: TObject; Reason: THookSocketReason;
    const Value: string);
    public
      constructor Create(URL, FileName: string);
      procedure Execute; override;
  end;

  { TForm1 }

  TForm1 = class(TForm)
    Button1: TButton;
    ELink: TEdit;
    ProgressBar1: TProgressBar;
    procedure Button1Click(Sender: TObject);
  private
    { private declarations }
  public
    { public declarations }
  end;

var
  Form1: TForm1;

implementation

{$R *.lfm}
{ THTTPDownload }

constructor THTTPDownload.Create(URL, FileName: string);
begin
  inherited Create(True);
  fURL:= URL;
  fFileName:= FileName;
end;
procedure THTTPDownload.SocketGetCallBack(Sender: TObject; Reason: THookSocketReason;
  const Value: string);
begin
  if TotalBytes=0 then
    exit;
  case Reason of
    HR_ReadCount:
    begin
      Inc(CurrentBytes, StrToIntDef(Value, 0));
      Form1.ProgressBar1.Position := Round(100 * (CurrentBytes / TotalBytes));
    end;
    HR_Connect: CurrentBytes := 0;
  end;
end;

procedure THTTPDownload.Execute;
const
  MaxRetries=3;
var
  HTTPGetResult: boolean;
  RetryAttempt: integer;
  i:integer;
  s:string;
  httpresult : boolean;
begin
  httpresult:=false;
  RetryAttempt:=1;
  http :=THTTPSend.Create;
  http.Sock.OnStatus := @SocketGetCallBack;
  try
    try
      // try to get the file size
      if http.HTTPMethod('HEAD', fURL) then
         begin
       // search for "Content-Length:" in header
            for i:=0 to http.Headers.Count-1 do
            begin
             s:=UpperCase(http.Headers[i]);
              if pos('CONTENT-LENGTH:',s)>0 then
              TotalBytes := StrToIntDef(copy(s,pos(':',s)+1,length(s)),0);
              end;
          http.Headers.Clear;

         end;
      // Try to get the file
      HTTPGetResult:=http.HTTPMethod('GET', fURL);
      while (HTTPGetResult=false) and (RetryAttempt<MaxRetries) do
      begin
        sleep(500*RetryAttempt);
        HTTPGetResult:=http.HTTPMethod('GET', fURL);
        RetryAttempt:=RetryAttempt+1;
      end;
      // If we have an answer from the server, check if the file
      // was sent to us.
      case http.Resultcode of
        100..299:
          begin
            with TFileStream.Create(fFileName,fmCreate or fmOpenWrite) do
            try
              Seek(0, soFromBeginning);
              CopyFrom(http.Document, 0);
            finally
              Free;
            end;
            httpresult:=true;
          end; //informational, success
        300..399: httpresult:=false; //redirection. Not implemented, but could be.
        400..499: httpresult:=false; //client error; 404 not found etc
        500..599: httpresult:=false; //internal server error
        else httpresult:=false; //unknown code
      end;
    except
      // We don't care for the reason for this error; the download failed.
      httpresult:=false;
    end;
  finally
    http.Free;
  end;
end;

{ TForm1 }


procedure TForm1.Button1Click(Sender: TObject);
  var i:integer;
  s:string;
  download: THTTPDownload;
begin
  Progressbar1.Position := 0;
  download:= THTTPDownload.Create(Elink.Text,'file.dwl');
 // download.OnTerminate:= @DownloadTerminiated;
  download.Resume;
end;

end.

Friendly,  J.P
Jurassic computer : Sinclair ZX81 - Zilog Z80A à 3,25 MHz - RAM 1 Ko - ROM 8 Ko

mesiarm

  • New Member
  • *
  • Posts: 45
Re: Download form - which library to use?
« Reply #5 on: June 30, 2013, 11:29:22 am »
Thank you  :) It works. I edited it bit and I want to ask - How to implement redirect? For example if I want to download from  http://get.videolan.org/vlc/2.0.7/win32/vlc-2.0.7-win32.exe it returns only few Kb file because it won´t redirect ..... And is there chance to even download latest version? maybe download from folder http://get.videolan.org:81/vlc/last/win32/ .... and else .... how can I extract domain from url? I tried more examples with basic pos function  but i didn´t worked

Leledumbo

  • Hero Member
  • *****
  • Posts: 8835
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: Download form - which library to use?
« Reply #6 on: June 30, 2013, 12:18:24 pm »
Quote
How to implement redirect? For example if I want to download from  http://get.videolan.org/vlc/2.0.7/win32/vlc-2.0.7-win32.exe it returns only few Kb file because it won´t redirect
Nothing is magical, so you have to go down to lower level a bit. That page returns html with this fragment:
Quote
<meta http-equiv="refresh" content="5;URL='http://free.nchc.org.tw/vlc/vlc/2.0.7/win32/vlc-2.0.7-win32.exe'" />
The bold part is the real URL that directly gets the file. Scan that meta tag (manual parsing should be enough, full html parsing could be slow).
Quote
And is there chance to even download latest version? maybe download from folder http://get.videolan.org:81/vlc/last/win32/
You know the URL, so just do it. Many repo provide a "symlink" to the latest version, without specific version in the filename, but VLC doesn't seem to follow that so you still have to scan the html. Assuming the filename format doesn't change, you could just look for .exe suffix (there's only one file).
Quote
and else .... how can I extract domain from url? I tried more examples with basic pos function  but i didn´t worked
http://www.freepascal.org/docs-html/fcl/uriparser/index.html

mesiarm

  • New Member
  • *
  • Posts: 45
Re: Download form - which library to use?
« Reply #7 on: July 07, 2013, 08:47:13 pm »
Thank you for advices. I´ll try it, but how to access http directory list, since there is no index.html file? ...and I have one other problem... i have main form which opens updater form as modal(Showmodal) and when i click on button in updater form it start downloading...and in thread in code above I have problem communicating with updater form and it´s progressbar. It raises exception while accesing to progressbar and when I use assigned(updaterform) in button onclick it returns false. It worked when I had it as separate form in test project, but stopped working, when I added it to my project. I dont know, where can be I mistake.... 

Jurassic Pork

  • Hero Member
  • *****
  • Posts: 1290
Re: Download form - which library to use?
« Reply #8 on: July 08, 2013, 05:01:51 am »
hello,
with this code it's OK for me (unit2 contains Updater form modal with a progressbar) :
Code: [Select]
unit Unit2;

{$mode objfpc}{$H+}

interface

uses
  Classes, SysUtils, FileUtil, Forms, Controls, Graphics, Dialogs, ComCtrls,
  httpsend,blcksock,typinfo, synamisc;

type
    { THTTPDownload }

  THTTPDownload = class(TThread)
    private
      http : THTTPsend;
      TotalBytes    :  integer;
      CurrentBytes  :  integer;
      fURL: string;
      fFileName: string;
    procedure SocketGetCallBack(Sender: TObject; Reason: THookSocketReason;
    const Value: string);
     procedure downloadterminated(Sender: TObject);

    public
      constructor Create(URL, FileName: string);
      procedure Execute; override;
  end;

  { TUpdater }

  TUpdater = class(TForm)
    ProgressBar1: TProgressBar;

  private
    { private declarations }
  public
    { public declarations }
  end;

var
  Updater: TUpdater;

implementation

{$R *.lfm}

{ THTTPDownload }

constructor THTTPDownload.Create(URL, FileName: string);
begin
  inherited Create(True);
  fURL:= URL;
  fFileName:= FileName;
  OnTerminate := @downloadterminated;
end;

procedure THTTPDownload.downloadterminated(Sender: TObject);
 begin
 Updater.Close;
 end;
procedure THTTPDownload.SocketGetCallBack(Sender: TObject; Reason: THookSocketReason;
  const Value: string);
begin
  if TotalBytes=0 then
    exit;
  case Reason of
    HR_ReadCount:
    begin
      Inc(CurrentBytes, StrToIntDef(Value, 0));
      Updater.ProgressBar1.Position := Round(100 * (CurrentBytes / TotalBytes));
    end;
    HR_Connect: CurrentBytes := 0;
  end;
end;

procedure THTTPDownload.Execute;
const
  MaxRetries=3;
var
  HTTPGetResult: boolean;
  RetryAttempt: integer;
  i:integer;
  s,Redirstr:string;
  httpresult : boolean;
  TextHttp : TStringList;
begin
  httpresult:=false;
  RetryAttempt:=1;
  Texthttp := TStringList.Create();
  http :=THTTPSend.Create;
  http.Sock.OnStatus := @SocketGetCallBack;
  try
    try
      // try to get the file size
      if http.HTTPMethod('HEAD', fURL) then
         begin
       // search for "Content-Length:" in header
            for i:=0 to http.Headers.Count-1 do
            begin
             s:=UpperCase(http.Headers[i]);
              if pos('CONTENT-LENGTH:',s)>0 then
              TotalBytes := StrToIntDef(copy(s,pos(':',s)+1,length(s)),0);
              end;
          http.Headers.Clear;

         end;
      // Try to get the file
      HTTPGetResult:=http.HTTPMethod('GET', fURL);
      while (HTTPGetResult=false) and (RetryAttempt<MaxRetries) do
      begin
        sleep(500*RetryAttempt);
        HTTPGetResult:=http.HTTPMethod('GET', fURL);
        RetryAttempt:=RetryAttempt+1;
      end;
      // If we have an answer from the server, check if the file
      // was sent to us.
      case http.Resultcode of
        100..299:
          begin
            with TFileStream.Create(fFileName,fmCreate or fmOpenWrite) do
            try
              Seek(0, soFromBeginning);
              CopyFrom(http.Document, 0);
            finally
              Free;
            end;
            httpresult:=true;
          end; //informational, success
        300..399: begin
                  Texthttp.LoadFromStream(http.document);
 //                 RedirStr := HTTPGetResult.
                  httpresult:=false; //redirection. Not implemented, but could be.
                  end;
        400..499: httpresult:=false; //client error; 404 not found etc
        500..599: httpresult:=false; //internal server error
        else httpresult:=false; //unknown code
      end;
    except
      // We don't care for the reason for this error; the download failed.
      httpresult:=false;
    end;
  finally
    http.Free;
  end;
end;
end. 

call from Form1 :
Code: [Select]
procedure TForm1.Button1Click(Sender: TObject);
  var i:integer;
  s:string;
  download: THTTPDownload;
begin
  Updater.Progressbar1.Position := 0;
  download:= THTTPDownload.Create(Elink.Text,'file.dwl');
  download.Resume;
  Updater.ShowModal;
  download.free;
end;

Code not safe (you must add exception management, etc...)

friendly, J.P
« Last Edit: July 08, 2013, 05:14:40 am by Jurassic Pork »
Jurassic computer : Sinclair ZX81 - Zilog Z80A à 3,25 MHz - RAM 1 Ko - ROM 8 Ko

mesiarm

  • New Member
  • *
  • Posts: 45
Re: Download form - which library to use?
« Reply #9 on: July 08, 2013, 08:29:09 pm »
thanks.. :) I would rather control download thread from second form since it´s declared there and I don´t want to move it. I know that there is some problem with forms, because when i have set updater as the main form (right after Application.Initialize) it worked. But normally when I call assigned(updater) in onclick button it returns false ...and raises excteption, when thread tries to comunicate with updater form.      I also tried creating Application.Createform(Tupdater, updater) in main form without success...What could be a problem ? I have also tried updater.Show instead of Showmodal.....

mesiarm

  • New Member
  • *
  • Posts: 45
Re: Download form - which library to use?
« Reply #10 on: July 08, 2013, 08:31:35 pm »
and please how to read the http directory list? since http://get.videolan.org:81/vlc/last/win32/ doesnt have any index.html file.....

Jurassic Pork

  • Hero Member
  • *****
  • Posts: 1290
Re: Download form - which library to use?
« Reply #11 on: July 09, 2013, 02:56:46 am »
for your first problem, please post in attachment or show in a message your source project to see what is wrong (sorry i don't understand how are your forms).
for http directory list , here is an experimental code (not sure than the reg expression is the best one) using regexpr to parse html ( because http directory listing is a html page) .
1 - to use regexpr unit add :
Code: [Select]
uses regexpr;2 - put the html text returned from "http://get.videolan.org:81/vlc/last/win32/"   in a tstringlist :
Code: [Select]
var TextHttp : TStringList;
...
 Texthttp := TStringList.Create();     
...
 Texthttp.LoadFromStream(http.document);

and here is a procedure to extract directory list   with regexpr :
Code: [Select]
procedure ListHTTPFiles(httpText : TstringList);   // J.P  July 2013
var x: integer;
   re: TRegExpr;
    s : string;
begin
  re := TRegExpr.Create;
  // expression to extract href and value from html line like :
  // <li><a href="vlc-2.0.7-win32.7z"> vlc-2.0.7-win32.7z</a></li>
  re.Expression := '<li><a href=\"(.*)\">(.*)</a>.+';
  For x := 0 to httpText.Count -1  do
  begin
  s := httpText.Strings[x];
  if re.Exec(s) then
  ShowMessage('href : ' + re.Match[1] + ' - value : ' + re.Match[2]);
  end;
  re.Free;
end;

called from this line :
Code: [Select]
ListHTTPFiles(Texthttp);
friendly, J.P
« Last Edit: July 09, 2013, 02:58:18 am by Jurassic Pork »
Jurassic computer : Sinclair ZX81 - Zilog Z80A à 3,25 MHz - RAM 1 Ko - ROM 8 Ko

mesiarm

  • New Member
  • *
  • Posts: 45
Re: Download form - which library to use?
« Reply #12 on: July 09, 2013, 08:12:58 pm »
I think it is a bit complicated with my project so I will post a image
Code: [Select]
http://img199.imageshack.us/img199/5716/ddyb.png

mesiarm

  • New Member
  • *
  • Posts: 45
Re: Download form - which library to use?
« Reply #13 on: July 12, 2013, 04:28:54 pm »
i tried your procedure and it works. I have changed it to function and added .exe suffix to regural expression and it works, but i don´t know when call this function, because if I call it at beggining http.document is empty and later it points to url to directory. Should I make second connection or?

Jurassic Pork

  • Hero Member
  • *****
  • Posts: 1290
Re: Download form - which library to use?
« Reply #14 on: July 12, 2013, 04:58:32 pm »
hello,
here is  the code  for   procedure      THTTPDownload.Execute;  ( the game : find where is  ListHTTPFiles function  :P )

Code: [Select]
procedure THTTPDownload.Execute;
const
  MaxRetries=3;
var
  HTTPGetResult: boolean;
  RetryAttempt: integer;
  i:integer;
  s,Redirstr:string;
  httpresult : boolean;
  TextHttp : TStringList;
begin
  httpresult:=false;
  RetryAttempt:=1;
  Texthttp := TStringList.Create();
  http :=THTTPSend.Create;
  http.Sock.OnStatus := @SocketGetCallBack;
  try
    try
      // try to get the file size
      if http.HTTPMethod('HEAD', fURL) then
         begin
       // search for "Content-Length:" in header
            for i:=0 to http.Headers.Count-1 do
            begin
             s:=UpperCase(http.Headers[i]);
              if pos('CONTENT-LENGTH:',s)>0 then
              TotalBytes := StrToIntDef(copy(s,pos(':',s)+1,length(s)),0);
              end;
          http.Headers.Clear;

         end;
      // Try to get the file
      HTTPGetResult:=http.HTTPMethod('GET', fURL);
      while (HTTPGetResult=false) and (RetryAttempt<MaxRetries) do
      begin
        sleep(500*RetryAttempt);
        HTTPGetResult:=http.HTTPMethod('GET', fURL);
        RetryAttempt:=RetryAttempt+1;
      end;
      // If we have an answer from the server, check if the file
      // was sent to us.
      case http.Resultcode of
        100..299:
          begin
              Texthttp.LoadFromStream(http.document);
              ListHTTPFiles(Texthttp);
            with TFileStream.Create(fFileName,fmCreate or fmOpenWrite) do
            try
              Seek(0, soFromBeginning);
              CopyFrom(http.Document, 0);
            finally
              Free;
              Texthttp.free;
            end;
            httpresult:=true;
          end; //informational, success
        300..399: begin
                  Texthttp.LoadFromStream(http.document);
 //                 RedirStr := HTTPGetResult.
                  httpresult:=false; //redirection. Not implemented, but could be.
                  end;
        400..499: httpresult:=false; //client error; 404 not found etc
        500..599: httpresult:=false; //internal server error
        else httpresult:=false; //unknown code
      end;
    except
      // We don't care for the reason for this error; the download failed.
      httpresult:=false;
    end;
  finally
    http.Free;
  end;
end; 

Friendly, J.P
Jurassic computer : Sinclair ZX81 - Zilog Z80A à 3,25 MHz - RAM 1 Ko - ROM 8 Ko

 

TinyPortal © 2005-2018