[SOLVED] HTTP Authentication

[rvk]

Thanks. Any idea on how to do it for a vBulletin based site?

vBulletin also just works with cookies.
Look at the login site and see where the login-data is posted (in the form tags).

Then look at the link I gave you to a topic here, to login.

You can also use Chrome (press f12 on the inlogscreen) to see what is posted and where you are redirected (under network tab).

Here is a code-example for vBulletin I once made.
https://forum.lazarus.freepascal.org/index.php/topic,49967.msg363705.html#msg363705

Ps. Was that in response to you or is that another P. Curtis?

[Remy Lebeau]

Thanks. Any idea on how to do it for a vBulletin based site?

Again, LOOK AT THE HTML to see how that site handles login authentication.

If it uses an HTML-based webform, then you have to POST the authentication credentials in the HTTP message body, typically in "application/x-www-webform-urlencoded" or "multipart/form-data" format.  The HTML will tell you which.

If it uses HTTP-based authentication, then I suggest you invest in learning how to use your webbrowser's built-in debugger (if it has one) or an external sniffer like Wireshark or Fiddler, to see what the HTTP headers actually look like.  If the site asks for HTTP authentication, it will send an "WWW-Authenticate" header describing the authentication schemes it supports, which the webbrowser can then choose from and send back the credentials in the appropriate format in a subsequent "Authentication" header.

[zamronypj]

I suggest you to look at browser developer tools (for Chrome or Firefox, press F12). While browser developer tool is open , do login manually and inspect what happens behind the scene in developer tools.

If it is form-based login, look at URL at action attribute of form tag. Usually it will be POST method. Inspect request headers and body that browser send to URL for POST method and also inspect response that server returns (usually it returns a cookie in response when login is succesful.

By observing request and response passed between browser and server, you will be able to replicate login with free pascal built in http client (in fact, browser is really just http client, a very sophisticated one)

[pcurtis]

@rvk - Oops, yes it was me. I was probably sidetracked at the time.

Anyway I have just tried it and get an error

---------------
Result = 500
---------------
POST /auth/ajax-login HTTP/1.0
Host: forum.vbulletin.com
Keep-Alive: 300
Connection: keep-alive
User-Agent: Mozilla/4.0 (compatible; Synapse)
Content-Type: application/x-www-form-urlencoded
Content-Length: 72

[rvk]

Anyway I have just tried it and get an error

You can't just use my snippet.
You have to check what kind of login-page is used.
Some use /auth/ajax-login. Others use login.php. And again others use a hashed password instead of passing password itself.
 
I already saw that vBulletin 4.2.2 has a different login-page and procedure from 5.6.4.
So each version can have it's own login-page to where you must post.

What version of vBulletin are you trying to login to?
(usually you can see that at the bottom)

[rvk]

PS. I see from your result that you are trying to login to the forum of vbulletin itself.
If you have an account there it should work.

NOTES: You do need to set the useragent to something else.

Code: Pascal  [Select][+][-]

1.     HTTPSend.Useragent := 'Mozilla/5.0 (Windows NT 10.0; Win64; x64)';

You ALSO need to add ssl_openssl to your uses clause and put the openssl dlls into your .exe directory.
Make sure the bitness of the openssl matches the bitness of your program (i.e. 32bit=32bit, 64bit-64bit)

It should work. I just tested it myself on vBulletin.
But with older vBulletin versions you might need other code. Keep that in mind.

[pcurtis]

OK. I get a HTTP 200. Getting close.

1. How do I know I am logged in?

I see that the site has returned 4 cookies. If I want  to login again do I just save these cookies and then the next time I create the client just reload them? Or are there further steps?

I used the vbulletin web site just because it was in the code.

The site I want to use is older (v3.8.7) it uses /login.php?do=login

Can I be a pest and ask if you know how to prepare the login text?

And a BIG thank you for your help so far.

Thanks.

[rvk]

I think the privacyconsent=1 will make it that you can continue with the login (but I'm not sure).

You just have to save the cookies in HTTPSend.Cookies.Text to a string
and set it back again before doing a next GET of the page you want.

You can see if you are logged in by accessing a page only a logged in person can access (for example the edit profile page).

There are some examples available for the login.php but I'm on mobioe now so can't create an example at the moment. I can try later on this afternoon.

[pcurtis]

Thank you.

[rvk]

Here are the login for both versions. I could only test the last one on v4.2.2. v3.8.7 is really really old (begin 2011) so you might want to warn the administrator about that. There are several exploits for that version and I hope it's completely patched.

For the latest version v5.6.4 (change the highlighted lines):

Code: Pascal  [Select][+][-]

1. uses httpsend, synautil, ssl_openssl;
2.  
3. procedure TForm1.Button1Click(Sender: TObject);
4. var
5.   HTTPSend: THTTPSend;
6.   Cookies: string;
7.   ForumUrl, Username, Password: string;
8.   TextFromDocument: string;
9. begin
10.   Memo1.Lines.Clear;
11.  
12.   ForumUrl := 'https://forum.vbulletin.com'; // <-- CHANGE THIS
13.   Username := 'username'; // <-- CHANGE THIS
14.   Password := 'password'; // <-- CHANGE THIS
15.  
16.   HTTPSend := THTTPSend.Create;
17.   try
18.  
19.     HTTPSend.Useragent := 'Mozilla/5.0 (Windows NT 10.0; Win64; x64)';
20.     HTTPSend.MimeType := 'application/x-www-form-urlencoded; charset=UTF-8';
21.     WriteStrToStream(HTTPSend.Document,
22.       ansistring('username=' + Username + '&password=' + Password +
23.       '&privacyconsent=1&securitytoken=guest'));
24.     HTTPSend.HTTPMethod('POST', ForumUrl + '/auth/ajax-login');
25.     TextFromDocument := ReadStrFromStream(HTTPSend.Document, HTTPSend.Document.Size);
26.     if HTTPSend.ResultCode <> 200 then
27.     begin
28.       Memo1.Lines.Add('---------------');
29.       Memo1.Lines.Add('Loginresult = ' + HTTPSend.ResultCode.ToString + ' ' + HTTPSend.ResultString);
30.       Memo1.Lines.Add('---------------');
31.       exit;
32.     end;
33.  
34.     Cookies := HTTPSend.Cookies.Text; // you can save this for later. It will expire after a while though.
35.     HTTPSend.Clear;
36.     HTTPSend.Cookies.Text := Cookies;
37.  
38.     HTTPSend.HTTPMethod('POST', 'https://forum.vbulletin.com/settings/profile'); // <-- CHANGE THIS
39.     TextFromDocument := ReadStrFromStream(HTTPSend.Document, HTTPSend.Document.Size);
40.     Memo1.Lines.Add('---------------');
41.     Memo1.Lines.Add('Result = ' + HTTPSend.ResultCode.ToString + ' ' + HTTPSend.ResultString);
42.     Memo1.Lines.Add('---------------');
43.     Memo1.Lines.Add(HTTPSend.Headers.Text);
44.     Memo1.Lines.Add('---------------');
45.     Memo1.Lines.Add(TextFromDocument);
46.     Memo1.Lines.Add('---------------');
47.  
48.   finally
49.     HTTPSend.Free;
50.   end;
51. end;

And for v4.2.2 (which is almost identical except for the parameters and url):
(it seems to have vb_login_md5password etc, but apparently they can be empty)

Code: Pascal  [Select][+][-]

1. uses httpsend, synautil, ssl_openssl;
2.  
3. procedure TForm1.Button2Click(Sender: TObject);
4. var
5.   HTTPSend: THTTPSend;
6.   Cookies: string;
7.   ForumUrl, Username, Password: string;
8.   TextFromDocument: string;
9. begin
10.   Memo1.Lines.Clear;
11.  
12.   ForumUrl := 'url_of_main_forum_page/forum.php'; // <-- CHANGE THIS
13.   Username := 'username'; // <-- CHANGE THIS
14.   Password := 'password'; // <-- CHANGE THIS
15.  
16.   HTTPSend := THTTPSend.Create;
17.   try
18.  
19.     HTTPSend.Useragent := 'Mozilla/5.0 (Windows NT 10.0; Win64; x64)';
20.     HTTPSend.MimeType := 'application/x-www-form-urlencoded; charset=UTF-8';
21.     WriteStrToStream(HTTPSend.Document,
22.       ansistring('vb_login_username=' + Username + '&vb_login_password=' + Password +
23.       'securitytoken=guest&do=login&vb_login_md5password=&vb_login_md5password_utf=&cookieuser=1s='));
24.     HTTPSend.HTTPMethod('POST', ForumUrl + '?do=login');
25.     TextFromDocument := ReadStrFromStream(HTTPSend.Document, HTTPSend.Document.Size);
26.     if HTTPSend.ResultCode <> 200 then
27.     begin
28.       Memo1.Lines.Add('---------------');
29.       Memo1.Lines.Add('Loginresult = ' + HTTPSend.ResultCode.ToString + ' ' + HTTPSend.ResultString);
30.       Memo1.Lines.Add('---------------');
31.       exit;
32.     end;
33.  
34.     Cookies := HTTPSend.Cookies.Text; // you can save this for later. It will expire after a while though.
35.     HTTPSend.Clear;
36.     HTTPSend.Cookies.Text := Cookies;
37.  
38.     HTTPSend.HTTPMethod('GET', 'url_of_control_panel_for_testing.php'); // <-- CHANGE THIS
39.     TextFromDocument := ReadStrFromStream(HTTPSend.Document, HTTPSend.Document.Size);
40.     Memo1.Lines.Add('---------------');
41.     Memo1.Lines.Add('Result = ' + HTTPSend.ResultCode.ToString + ' ' + HTTPSend.ResultString);
42.     Memo1.Lines.Add('---------------');
43.     Memo1.Lines.Add(HTTPSend.Headers.Text);
44.     Memo1.Lines.Add('---------------');
45.     Memo1.Lines.Add(TextFromDocument);
46.     Memo1.Lines.Add('---------------');
47.  
48.   finally
49.     HTTPSend.Free;
50.   end;
51.  
52. end;

[Thaddy]

Rik,
I appreciate almost of all your contributions, but plz help me to cut of http.
Use https. Unless you have full control.

[rvk]

I appreciate almost of all your contributions, but plz help me to cut of http.
Use https. Unless you have full control.

I did use https in my examples.

And this isn't about running your own webserver. This is about logging into an existing web forum. So the choice of http(s) isn't an issue.

[pcurtis]

Thanks I'll  take a look.

Going back to my previous post - I can't verify that I am actually logged to vbulletin. I yrirf the following

Code: Pascal  [Select][+][-]

1.     ForumUrl := 'https://forum.vbulletin.com';
2.     Username := myusername';
3.    Password := 'mypassword';
4.  
5.    HTTPSend := THTTPSend.Create;
6.    try
7.      HTTPSend.Useragent := 'Mozilla/5.0 (Windows NT 10.0; Win64; x64)';
8.      HTTPSend.MimeType := 'application/x-www-form-urlencoded';
9.  
10.      WriteStrToStream(HTTPSend.Document,
11.                       ansistring('username=' + Username + '&password=' + Password +
12.                      '&privacyconsent=1&securitytoken=guest'));
13.  
14.      Ok := HTTPSend.HTTPMethod('POST', ForumUrl + '/auth/ajax-login');
15.  
16.      HTTPSend.HTTPMethod('GET', 'https://members.vbulletin.com/');
17.  
18.       SetLength(sTEMP, HTTPSend.Document.Size);
19.       HTTPSend.Document.Read(sTEMP[1], Length(sTEMP));
20.       HTTPSend.Document.SaveToFile('out.html');
21.       Memo1.Lines.Add(HTTPSend.Headers.Text);
22.     finally
23.       HTTPSend.Free;
24.     end;
25.  

The output is

---------------
Result = 200 OK
---------------
HTTP/1.1 200 OK
Date: Tue, 13 Oct 2020 14:54:16 GMT
Content-Type: application/json; charset=UTF-8
Content-Length: 48
Connection: keep-alive
Set-Cookie: __cfduid=d877fc0d36dd653ee8826206f025d250a1602600856; expires=Thu, 12-Nov-20 14:54:16 GMT; path=/; domain=.vbulletin.com; HttpOnly; SameSite=Lax; Secure
Strict-Transport-Security: max-age=31536000
X-Frame-Options: sameorigin
Set-Cookie: vblm_sessionhash=fab382f5a2fb7989892f6c0f9137a632; path=/; domain=.vbulletin.com; secure; HttpOnly
Set-Cookie: vblm_lastvisit=1602600856; expires=Wed, 13-Oct-2021 14:54:16 GMT; Max-Age=31536000; path=/; domain=.vbulletin.com; secure; HttpOnly
Set-Cookie: vblm_lastactivity=1602600856; expires=Wed, 13-Oct-2021 14:54:16 GMT; Max-Age=31536000; path=/; domain=.vbulletin.com; secure; HttpOnly
Expires: Sat, 1 Jan 2000 01:00:00 GMT
Cache-Control: max-age=0,no-cache,no-store,post-check=0,pre-check=0
Pragma: no-cache
X-Frame-Options: sameorigin
Content-Security-Policy: frame-ancestors 'self' https://admin.vbulletin.com/ https://www.vbulletin.com/ https://members.vbulletin.com/ https://testsecureacceptance.cybersource.com/ https://secureacceptance.cybersource.com/ https://ssl.kaptcha.com/'; script-src * blob: 'unsafe-inline' 'unsafe-eval' ; object-src *
Last-Modified: Tue, 13 Oct 2020 14:54:16 GMT
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Vary: Accept-Encoding
CF-Cache-Status: DYNAMIC
cf-request-id: 05c40d81bc000027a0b3269200000001
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report?lkg-colo=31&lkg-time=1602600856"}],"group":"cf-nel","max_age":604800}
NEL: {"report_to":"cf-nel","max_age":604800}
Server: cloudflare
CF-RAY: 5e19e515fe6027a0-PRG


---------------
{"errors":[["strikes","\/lostpw"]],"userid":"0"}

If you notice the text

{"errors":[["strikes","\/lostpw"]],"userid":"0"}

at the end.

What stupid mistake am I making?

[rvk]

For checking if you are logged in you can look at the code I posted.
There I try to access the /settings/profile
When not logged in you get an errir in html text.

The first result should give a newtoken in json.
You get an error with strikes.
I'm not sure what that means and if you are logged in at that point.

What does the /settings/profile show after this call (as shown in my example)?

Edit:
Ah, you removed that json result from your post.
Then you can continue accessing the forum and see if it works.

[rvk]

So wait 15 minutes.

Use Login "Strikes" System
Setting this to No disables the system that prevents a user (with a specific IP address) from logging in to an account after entering an incorrect password 5 times.

After the first failed login attempt, the user receives the following message:

You have used 1 out of 5 login attempts. After all 5 have been used, you will be unable to log in for 15 minutes.

After the fifth failed login attempt, the user receives the following message:

Wrong username or password. You have used up your failed login quota! Please wait 15 minutes before trying again. Don't forget that the password is case sensitive.

After the fifth failed attempt, the user is locked out for fifteen minutes and and email will be sent to the email address associated with the account alerting them about the login failure.