Recent

Author Topic: TFPHTTPClient get ERROR 401 (Unauthorized)  (Read 584 times)

DidaJI

  • New Member
  • *
  • Posts: 30
TFPHTTPClient get ERROR 401 (Unauthorized)
« on: August 24, 2020, 10:54:11 am »
Hi,
I have a problem logging in to the device API via TFPHTTPClient.

This works for me from the web browser: http://admin:admin123@192.168.0.39/cgi-bin/record.cgi?action=find&name=Access.

This works for me from python script:
import requests
from requests.auth import HTTPDigestAuth
response = requests.get("http://192.168.0.39/cgi-bin/record.cgi?action=find&name=Access", auth=HTTPDigestAuth("admin", "admin123"))
print (response)

But if I use TFPHTTPClient, it always returns Error 401

For example, I tried this:
url := 'http://admin:admin123@192.168.0.39/cgi-bin/recordFinder.cgi?action=find&name=Access';
htmlResponse := TFPCustomHTTPClient.SimpleGet(url);
 
Or this:
 httpClient := TFPHTTPClient.Create(nil);
  try
   httpClient.AllowRedirect := true;
   httpClient.Password := 'admin123';
   httpClient.UserName := 'admin';
   httpClient.AddHeader('User-Agent','Mozilla/5.0 (compatible; fpweb)');
   FileContents := httpClient.Get('http://192.168.0.39/cgi-bin/recordFinder.cgi?action=find&name=Access');
   theStatusCode := httpClient.ResponseStatusCode;
  finally
   httpClient.Free;
  end;

Or this:
 httpClient := TFPHTTPClient.Create(nil);
 FormData := tstringlist.Create;
  try
   FormData.Values['auth'] := 'HTTPDigestAuth';
   FormData.Values['username'] := 'admin';
   FormData.Values['password'] := 'admin123';
 
   FileContents := httpClient.FormPost('http://192.168.0.39/cgi-bin/recordFinder.cgi?action=find&name=Access', FormData);
   theStatusCode := httpClient.ResponseStatusCode;

  finally
    FormData.Free;
    httpClient.Free;
  end;

And other variants. Unfortunately, everything leads to the result "Error 401".

Would anyone please know how to solve this?

Thank's: Zdenek.
« Last Edit: August 24, 2020, 11:00:57 am by DidaJI »

paweld

  • Full Member
  • ***
  • Posts: 244
Re: TFPHTTPClient get ERROR 401 (Unauthorized)
« Reply #1 on: August 24, 2020, 11:12:55 am »
Code: Pascal  [Select][+][-]
  1.  
  2. uses
  3.   base64;
  4.  
  5. var
  6.   login, pass: String;
  7. begin
  8.   login := 'admin';
  9.   pass := 'admin123';
  10.   httpClient := TFPHTTPClient.Create(nil);
  11.   try
  12.    httpClient.AllowRedirect := true;
  13.    httpClient.RequestHeaders.Add('Authorization: Basic ' + EncodeStringBase64(login + ':' + pass));
  14.    FileContents := httpClient.Get('http://192.168.0.39/cgi-bin/recordFinder.cgi?action=find&name=Access');
  15.    theStatusCode := httpClient.ResponseStatusCode;
  16.   finally
  17.    httpClient.Free;
  18.   end;
  19. end;
Best regards
paweld

DidaJI

  • New Member
  • *
  • Posts: 30
Re: TFPHTTPClient get ERROR 401 (Unauthorized)
« Reply #2 on: August 24, 2020, 11:35:27 am »
To paweld: Thanks, I tried this too and it's still the same.

DidaJI

  • New Member
  • *
  • Posts: 30
Re: TFPHTTPClient get ERROR 401 (Unauthorized)
« Reply #3 on: August 24, 2020, 01:06:56 pm »
So sorry, this: http://admin:admin123@192.168.0.39/cgi-bin/recordFinder.cgi?action=find&name=Access it also doesn't work from a web browser.
I was only able to do this because the browser remembered the login details. If I call it from a browser, then I still have to enter the name and password in the form that appears.

The only thing that works well is this python script:
import requests
from requests.auth import HTTPDigestAuth
response = requests.get("http://192.168.0.39/cgi-bin/record.cgi?action=find&name=Access", auth=HTTPDigestAuth("admin", "admin123"))
print (response)

Would anyone know how to replace this script with TFPHTTPClient?

Thank's: Zdenek.



rvk

  • Hero Member
  • *****
  • Posts: 4327
Re: TFPHTTPClient get ERROR 401 (Unauthorized)
« Reply #4 on: August 24, 2020, 01:27:19 pm »
HTTPDigestAuth should result in the HTTP Basic Authentication that paweld already gave you.

What does the following python3 script give you (adding debug options)?

Code: Python  [Select][+][-]
  1. import requests
  2. from requests.auth import HTTPDigestAuth
  3.  
  4. import logging
  5. try:
  6.     import http.client as http_client
  7. except ImportError:
  8.     # Python 2
  9.     import httplib as http_client
  10. http_client.HTTPConnection.debuglevel = 1
  11. logging.basicConfig()
  12. logging.getLogger().setLevel(logging.DEBUG)
  13. requests_log = logging.getLogger("requests.packages.urllib3")
  14. requests_log.setLevel(logging.DEBUG)
  15. requests_log.propagate = True
  16.  
  17. response = requests.get("http://192.168.0.39/cgi-bin/record.cgi?action=find&name=Access", auth=HTTPDigestAuth("admin", "admin123"   ))
  18. print (response)

trev

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1000
  • Former Delphi 1-7, 10.2 User
Re: TFPHTTPClient get ERROR 401 (Unauthorized)
« Reply #5 on: August 24, 2020, 01:36:08 pm »
Are you sure the device web server implements basic authentication?
o Lazarus v2.1.0 r63871, FPC v3.3.1 r46876, macOS 10.14.6 (with sup update), Xcode 11.3.1
o Lazarus v2.1.0 r61574, FPC v3.3.1 r42318, FreeBSD 12.1 amd64 (VMware Fusion VM)
o FPC 3.0.4, FreeBSD 12.2-STABLE r365646 amd64
o Lazarus v2.1.0 r61574, FPC v3.0.4, Ubuntu 18.04 (Parallels VM)

paweld

  • Full Member
  • ***
  • Posts: 244
Best regards
paweld

rvk

  • Hero Member
  • *****
  • Posts: 4327
Re: TFPHTTPClient get ERROR 401 (Unauthorized)
« Reply #7 on: August 24, 2020, 01:44:52 pm »
https://forum.lazarus.freepascal.org/index.php/topic,15747.msg85028.html#msg85028
Woops. Yes, HTTPBasicAuth is something different from HTTPDigestAuth.
I missed that it was the Digest auth. Logging would have confirmed that.

https://en.wikipedia.org/wiki/Digest_access_authentication

(Bit more complicated but doable)

DidaJI

  • New Member
  • *
  • Posts: 30
Re: TFPHTTPClient get ERROR 401 (Unauthorized)
« Reply #8 on: August 24, 2020, 02:04:50 pm »
To rvk:

the modified script returns the following:
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): 192.168.0.39:80
send: 'GET /cgi-bin/record.cgi?action=find&name=Access HTTP/1.1\r\nHost: 192.168.0.39\r\nConnection: keep-alive\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nUser-Agent: python-requests/2.21.0\r\n\r\n'
reply: 'HTTP/1.1 401 Unauthorized\r\n'
header: WWW-Authenticate: Digest realm="Login to 6a285c4b6cf2c6685d0eee0bbca565d6", qop="auth", nonce="1339093192", opaque="12a495b4a51a1ad37368ffdbf2174e850e759965"
header: Connection: close
header: Set-Cookie:secure; HttpOnly
header: CONTENT-LENGTH: 0
DEBUG:urllib3.connectionpool:http://192.168.0.39:80 "GET /cgi-bin/record.cgi?action=find&name=Access HTTP/1.1" 401 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: 192.168.0.39
send: 'GET /cgi-bin/record.cgi?action=find&name=Access HTTP/1.1\r\nHost: 192.168.0.39\r\nConnection: keep-alive\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nUser-Agent: python-requests/2.21.0\r\nCookie: secure\r\nAuthorization: Digest username="admin", realm="Login to 6a285c4b6cf2c6685d0eee0bbca565d6", nonce="1339093192", uri="/cgi-bin/record.cgi?action=find&name=Access", response="ff0257016ad4988646fe323e7ac0ce9d", opaque="12a495b4a51a1ad37368ffdbf2174e850e759965", qop="auth", nc=00000001, cnonce="4b5442b851a4e032"\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: X-XSS-Protection: 1;mode=block
header: X-Frame-Options: SAMEORIGIN
header: Content-Security-Policy: script-src 'self' 'unsafe-inline' 'unsafe-eval'
header: Strict-Transport-Security: max-age=604800; includeSubDomains
header: Content-type: text/plain;charset=utf-8
header: CONNECTION: close
header: Set-Cookie:secure; HttpOnly
header: CONTENT-LENGTH: 37131
DEBUG:urllib3.connectionpool:http://192.168.0.39:80 "GET /cgi-bin/record.cgi?action=find&name=Access HTTP/1.1" 200 37131
<Response [200]>

rvk

  • Hero Member
  • *****
  • Posts: 4327
Re: TFPHTTPClient get ERROR 401 (Unauthorized)
« Reply #9 on: August 24, 2020, 02:23:56 pm »
the modified script returns the following:
You see that there are to requests done (GET). One is without authentication. You'll get a header back with WWW-Authenticate: Digest and some extra information (realm, qop, nonce and opaque). You'll need that extra information to create a second request with correct authentication in response.

As seen here https://en.wikipedia.org/wiki/Digest_access_authentication

If I can make some time today I can create an example.

rvk

  • Hero Member
  • *****
  • Posts: 4327
Re: TFPHTTPClient get ERROR 401 (Unauthorized)
« Reply #10 on: August 24, 2020, 05:37:18 pm »
Had a few minutes to spare and made a small example for fphttpclient and Digest authentication.
It uses the sample server at https://jigsaw.w3.org/HTTP/
https://jigsaw.w3.org/HTTP/Digest

Drop a TButton and a TMemo on a new project. Double click the button and paste the following code in the editor.
(You'll need the openssl libraries in the .exe directory if they are not available systemwide)

If it works you can change the login, pass and URL for your server and see if it works there too.
(Last text in sample should be "Your browser made it!")

Code: Pascal  [Select][+][-]
  1. uses fphttpclient, URIParser, md5, opensslsockets, dateutils;
  2.  
  3. // sample server with Digest Authentication = https://jigsaw.w3.org/HTTP/Digest/
  4.  
  5. procedure TForm1.Button1Click(Sender: TObject);
  6. var
  7.   login, pass: string;
  8.   HTTPClient: TFPHTTPClient;
  9.   Stream: TStringStream;
  10.   I, J: integer;
  11.   URL: string;
  12.   aStr: string;
  13.   aList: TStringList;
  14.   realm, nonce, qop, opaque, cnonce: string;
  15.   h1, h2, h3, h4: string;
  16.   URI: TURI;
  17.   UriStr: string;
  18. begin
  19.  
  20.   login := 'guest';
  21.   pass := 'guest';
  22.   URL := 'https://jigsaw.w3.org/HTTP/Digest';
  23.   HTTPClient := TFPHTTPClient.Create(nil);
  24.   Stream := TStringStream.Create;
  25.   aList := TStringList.Create;
  26.   try
  27.  
  28.     URI := ParseURI(URL, False);
  29.     UriStr := URI.Path;
  30.     begin
  31.       if (Length(URI.Path) > 0) and ((Length(UriStr) = 0) or (UriStr[Length(UriStr)] <> '/')) then
  32.         UriStr := UriStr + '/';
  33.       UriStr := UriStr + URI.Document;
  34.     end;
  35.     if Length(URI.Params) > 0 then
  36.       UriStr := UriStr + '?' + URI.Params;
  37.     Memo1.Lines.Add(UriStr);
  38.  
  39.     HTTPClient.KeepConnection := true;
  40.     HTTPClient.AllowRedirect := true;
  41.     HTTPClient.HTTPMethod('GET', URL, Stream, [200, 401]);
  42.     if HTTPClient.ResponseStatusCode = 401 then
  43.     begin
  44.  
  45.       for I := 0 to HTTPClient.ResponseHeaders.Count - 1 do
  46.       begin
  47.         if LeftStr(uppercase(HTTPClient.ResponseHeaders.Strings[I]), 24) = 'WWW-AUTHENTICATE: DIGEST' then
  48.         begin
  49.           realm := '';
  50.           nonce := '';
  51.           qop := '';
  52.           opaque := '';
  53.           cnonce := md5Print(md5String(IntToStr(DateTimeToUnix(Now()))));
  54.  
  55.           aList.Clear;
  56.           aList.StrictDelimiter := True;
  57.           aList.Delimiter := ',';
  58.           aList.DelimitedText := trim(Copy(HTTPClient.ResponseHeaders.Strings[I], 25));
  59.           for J := 0 to pred(aList.Count) do
  60.           begin
  61.             aStr := trim(aList.Strings[J]);
  62.             if LeftStr(aStr, 5) = 'realm' then realm := Copy(aStr, 7, Length(aStr)).DeQuotedString(#34);
  63.             if LeftStr(aStr, 5) = 'nonce' then nonce := Copy(aStr, 7, Length(aStr)).DeQuotedString(#34);
  64.             if LeftStr(aStr, 3) = 'qop' then qop := Copy(aStr, 5, Length(aStr)).DeQuotedString(#34);
  65.             if LeftStr(aStr, 6) = 'opaque' then opaque := Copy(aStr, 8, Length(aStr)).DeQuotedString(#34);
  66.           end;
  67.  
  68.           h1 := md5Print(md5String(login + ':' + realm + ':' + pass));
  69.           h2 := md5Print(md5String('GET' + ':' + UriStr));
  70.           if (qop = 'auth') or (qop = 'auth-int') then
  71.             h3 := md5Print(md5String(h1 + ':' + nonce + ':00000001:' + cnonce + ':' + qop + ':' + h2))
  72.           else
  73.             h3 := md5Print(md5String(h1 + ':' + nonce + ':' + h2));
  74.  
  75.           h4 := 'username=' + AnsiQuotedStr(login, #34);
  76.           h4 := h4 + ', realm=' + AnsiQuotedStr(realm, #34);
  77.           h4 := h4 + ', nonce=' + AnsiQuotedStr(nonce, #34);
  78.           h4 := h4 + ', uri=' + AnsiQuotedStr(UriStr, #34);
  79.           if (qop = 'auth') or (qop = 'auth-int') then
  80.           begin
  81.             h4 := h4 + ', qop=' + qop;
  82.             h4 := h4 + ', nc=00000001';
  83.           end;
  84.           h4 := h4 + ', cnonce=' + AnsiQuotedStr(cnonce, #34);
  85.           h4 := h4 + ', response=' + AnsiQuotedStr(h3, #34);
  86.           if opaque <> '' then
  87.             h4 := h4 + ', opaque=' + AnsiQuotedStr(opaque, #34);
  88.  
  89.           HTTPClient.RequestHeaders.Add('Authorization: Digest ' + h4);
  90.  
  91.           Stream.Clear; // clear the previous request in the stream
  92.           HTTPClient.HTTPMethod('GET', URL, Stream, [200]);
  93.  
  94.           Memo1.Clear;
  95.           Memo1.Lines.Add('-----');
  96.           Memo1.Lines.Add(HTTPClient.ResponseHeaders.Text);
  97.           Memo1.Lines.Add('-----');
  98.           Memo1.Lines.Add(Stream.DataString);
  99.  
  100.         end;
  101.       end;
  102.     end;
  103.   finally
  104.     HTTPClient.Free;
  105.     Stream.Free;
  106.     aList.Free;
  107.   end;
  108. end;
« Last Edit: August 24, 2020, 05:39:23 pm by rvk »

DidaJI

  • New Member
  • *
  • Posts: 30
Re: TFPHTTPClient get ERROR 401 (Unauthorized)
« Reply #11 on: August 25, 2020, 07:30:34 am »

It works, thank you very very much !!!

Zdenek.

 

TinyPortal © 2005-2018