Recent

Author Topic: [Solved] URLtoStr when web site unavailable  (Read 201 times)

bobonwhidbey

  • Hero Member
  • *****
  • Posts: 540
    • Double Dummy Solver - free download
[Solved] URLtoStr when web site unavailable
« on: July 04, 2022, 06:47:01 pm »
I really like this code (below) that I picked up a few years ago. It works very well. When the web site is unavailable  (perhaps the host of the web site is down for maintenance) this routine tries for several minutes before returning a blank string. My question is - what determines the period of time before the routine gives up and assumes the web site is unavailable? Is this under control of the user's computer? Is there a way for my app to limit this time?

p.s. the user with the long wait was Linux. I'm not sure if this is a Win problem.

Code: Pascal  [Select][+][-]
  1. function URLtoStr(URL: string): string;
  2. var
  3.   NetHandle: HINTERNET;
  4.   UrlHandle: HINTERNET;
  5.   Buffer: array[0..1023] of byte;
  6.   BytesRead: dWord;
  7.   StrBuffer: utf8string;
  8. begin
  9.   strbuffer := URL;
  10.   Result := '';
  11.   NetHandle := InternetOpen('Mozilla/5.0(compatible; WinInet)',
  12.     INTERNET_OPEN_TYPE_PRECONFIG, nil, nil, 0);
  13.  
  14.   // NetHandle valid?
  15.   if Assigned(NetHandle) then
  16.     try
  17.       UrlHandle := InternetOpenUrl(NetHandle, PChar(Url), nil, 0,
  18.         INTERNET_FLAG_RELOAD, 0);
  19.  
  20.       // UrlHandle valid?
  21.       if Assigned(UrlHandle) then
  22.         try
  23.           repeat
  24.             BytesRead := 0;
  25.             InternetReadFile(UrlHandle, @Buffer, SizeOf(Buffer), BytesRead);
  26.             SetString(StrBuffer, pansichar(@Buffer[0]), BytesRead);
  27.             Result := Result + StrBuffer;
  28.           until BytesRead = 0;
  29.         finally
  30.           InternetCloseHandle(UrlHandle);
  31.         end
  32.       // o/w UrlHandle invalid
  33.       else
  34.         ShowMessage('Cannot open URL: ' + Url);
  35.     finally
  36.       InternetCloseHandle(NetHandle);
  37.     end
  38.   // NetHandle invalid
  39.   else
  40.     raise Exception.Create('Unable to initialize WinInet');
  41. end;
  42.  
  43.  
« Last Edit: July 04, 2022, 08:04:23 pm by bobonwhidbey »
Win10 64-bit / Lazarus 32-bit 2.2.2 / FPC 3.2.2

dsiders

  • Hero Member
  • *****
  • Posts: 672
Re: URLtoStr when web site unavailable
« Reply #1 on: July 04, 2022, 07:18:31 pm »
I really like this code (below) that I picked up a few years ago. It works very well. When the web site is unavailable  (perhaps the host of the web site is down for maintenance) this routine tries for several minutes before returning a blank string. My question is - what determines the period of time before the routine gives up and assumes the web site is unavailable? Is this under control of the user's computer? Is there a way for my app to limit this time?

p.s. the user with the long wait was Linux. I'm not sure if this is a Win problem.

Code: Pascal  [Select][+][-]
  1. function URLtoStr(URL: string): string;
  2. var
  3.   NetHandle: HINTERNET;
  4.   UrlHandle: HINTERNET;
  5.   Buffer: array[0..1023] of byte;
  6.   BytesRead: dWord;
  7.   StrBuffer: utf8string;
  8. begin
  9.   strbuffer := URL;
  10.   Result := '';
  11.   NetHandle := InternetOpen('Mozilla/5.0(compatible; WinInet)',
  12.     INTERNET_OPEN_TYPE_PRECONFIG, nil, nil, 0);
  13.  
  14.   // NetHandle valid?
  15.   if Assigned(NetHandle) then
  16.     try
  17.       UrlHandle := InternetOpenUrl(NetHandle, PChar(Url), nil, 0,
  18.         INTERNET_FLAG_RELOAD, 0);
  19.  
  20.       // UrlHandle valid?
  21.       if Assigned(UrlHandle) then
  22.         try
  23.           repeat
  24.             BytesRead := 0;
  25.             InternetReadFile(UrlHandle, @Buffer, SizeOf(Buffer), BytesRead);
  26.             SetString(StrBuffer, pansichar(@Buffer[0]), BytesRead);
  27.             Result := Result + StrBuffer;
  28.           until BytesRead = 0;
  29.         finally
  30.           InternetCloseHandle(UrlHandle);
  31.         end
  32.       // o/w UrlHandle invalid
  33.       else
  34.         ShowMessage('Cannot open URL: ' + Url);
  35.     finally
  36.       InternetCloseHandle(NetHandle);
  37.     end
  38.   // NetHandle invalid
  39.   else
  40.     raise Exception.Create('Unable to initialize WinInet');
  41. end;
  42.  
  43.  

Connection timeout is controlled by InternetSetOption: https://stackoverflow.com/questions/28774137/decrease-internetopenurl-timeout

These routines are imported from wininet.dll and depend on Windows. So they are not cross-platform.
« Last Edit: July 04, 2022, 07:26:22 pm by dsiders »
Preview Lazarus 2.3.0 documentation at: https://dsiders.gitlab.io/lazdocsnext

bobonwhidbey

  • Hero Member
  • *****
  • Posts: 540
    • Double Dummy Solver - free download
Re: URLtoStr when web site unavailable
« Reply #2 on: July 04, 2022, 07:51:27 pm »
i get "illegal qualifier" compilation error for the IntPtr.zero argument.

    InternetSetOption(IntPtr.zero, INTERNET_OPTION_CONNECT_TIMEOUT,
                      @dwTimeOut, SizeOf(dwTimeOut));

That routine is being found in WinINet.pp
Win10 64-bit / Lazarus 32-bit 2.2.2 / FPC 3.2.2

bobonwhidbey

  • Hero Member
  • *****
  • Posts: 540
    • Double Dummy Solver - free download
Re: URLtoStr when web site unavailable
« Reply #3 on: July 04, 2022, 07:59:24 pm »
Once I used NetHandle all seems to be working perfectly. Thank you dsiders.

  NetHandle := InternetOpen('Mozilla/5.0(compatible; WinInet)',
    INTERNET_OPEN_TYPE_PRECONFIG, nil, nil, 0);

  dwTimeOut := 5000; // Timeout in milliseconds, so 5 sec.
//  InternetSetOption(IntPtr.zero, INTERNET_OPTION_CONNECT_TIMEOUT,
  InternetSetOption(NetHandle, INTERNET_OPTION_CONNECT_TIMEOUT,
                    @dwTimeOut, SizeOf(dwTimeOut));
Win10 64-bit / Lazarus 32-bit 2.2.2 / FPC 3.2.2

 

TinyPortal © 2005-2018