* * *

Author Topic: [SOLVED] IdHTTP and BAD REQUEST  (Read 1684 times)

DanishMale

  • New member
  • *
  • Posts: 40
[SOLVED] IdHTTP and BAD REQUEST
« on: May 06, 2017, 01:01:59 am »
I've got a problem which drives me nuts....

I have googled and searched for a couple of days now and still not found any solution which solves my problem >:( :o

I keep getting the error HTTP/1.1 - 400 Bad Request

My code is as follows:

Code: Pascal  [Select]
  1. function GetRemoteData(web_url: String): String;
  2. var
  3.   HTTP1: TIdHTTP;
  4.   StrList, Data: TStringList;
  5. begin
  6.  try
  7.    try
  8.    HTTP1 := TIdHTTP.Create(nil);
  9.    HTTP1.Request.UserAgent:= 'APP Name/'+GetAppVer(GetAppPath+'emcd.exe')+ ' +http://app url';
  10.    HTTP1.Request.CharSet := 'utf-8';
  11.    if (POS('per_page',web_url) > 0) then
  12.    begin
  13.      HTTP1.Request.ContentType := 'application/x-www-form-urlencoded';
  14.      HTTP1.IOHandler := TIdSSLIOHandlerSocketOpenSSL.Create(nil);
  15.      HTTP1.Request.Accept := 'application/vnd.discogs.v2.plaintext+json';
  16.      StrList := TStringList.Create;
  17.      SplitText('&', web_url, StrList);
  18.      StrList[0] := StringReplace(StrList[0],' ','%20',[rfReplaceAll]);
  19.      Data := TStringList.Create;
  20.      Data.Values['q'] := copy(StrList[0],POS('=',StrList[0])+1, Length(StrList[0]));
  21.      Data.Values['key'] := copy(StrList[1],POS('=',StrList[1])+1, Length(StrList[1]));
  22.      Data.Values['secret'] := copy(StrList[2],POS('=',StrList[2])+1, Length(StrList[2]));
  23.      Data.Values['per_page'] := copy(StrList[3],POS('=',StrList[3])+1, Length(StrList[3]));
  24.      Data.Values['page'] := copy(StrList[4],POS('=',StrList[4])+1, Length(StrList[4]));
  25.      StrList.Destroy;
  26.      Result := HTTP1.Post('https://api.discogs.com/database/search',data);
  27.      Data.Destroy;
  28.    end
  29.    else
  30.    begin
  31.      HTTP1.Request.ContentType := 'text/html';
  32.      Result := HTTP1.Get(web_url, IndyTextEncoding_UTF8);
  33.    end;
  34.  
  35.    except
  36.      on Error: Exception do
  37.         ShowMessage('My Errorcatcher: ' +Error.ClassName+' - '+Error.Message);
  38.    end;
  39.  finally
  40.    HTTP1.Free;
  41.  end;
  42. end;
  43.  

The error only occurs when I try to POST (e.g using the part where "per_page" is the query and the main url is "hardcoded" hence it is static, any other request works perfectly using the GET (e.g. everything w/o "per_page")

I have the correct SSL dlls installed (hence they are reuse from another program)

I hope some you intelligent guys out there can look thru and point out my mistake  :D
« Last Edit: May 07, 2017, 01:23:56 pm by DanishMale »
Lazarus 1.6 x64 | FPC 3.0.0 | Windows 10 x64 | MySQL Community Server (GPL) 5.6 x64 | MariaDB 10.1.21 x64 | Firebird 3.0.1 x64 | SQLite 3.16.2 x64|MS SQL Server 2012 x64 | PostgresSQL 9.6.1 x64 | Oracle 12.1.0.2 SE x64 |Windows Server 2008 x64

Leledumbo

  • Hero Member
  • *****
  • Posts: 7742
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: IdHTTP and BAD REQUEST
« Reply #1 on: May 06, 2017, 08:32:05 am »
HTTP 4XX is ALWAYS your fault (invalid credentials, invalid request method or parameter, etc.), in which without any valid and up to date (as in: the one the server is running) reference to discogs API, I can't tell much. I don't usually return empty data in 4XX HTTP response for API, naturally at least a reason should be given (probably with fix suggestion), but it all comes down to the API developer.

DanishMale

  • New member
  • *
  • Posts: 40
Re: [SOLVED] IdHTTP and BAD REQUEST
« Reply #2 on: May 07, 2017, 01:25:44 pm »
I don't know what happened ... today I switched back to one of my previous solutions and now it works as expected ....


Thx for your time and answer Leledumbo :)
Lazarus 1.6 x64 | FPC 3.0.0 | Windows 10 x64 | MySQL Community Server (GPL) 5.6 x64 | MariaDB 10.1.21 x64 | Firebird 3.0.1 x64 | SQLite 3.16.2 x64|MS SQL Server 2012 x64 | PostgresSQL 9.6.1 x64 | Oracle 12.1.0.2 SE x64 |Windows Server 2008 x64

Remy Lebeau

  • Sr. Member
  • ****
  • Posts: 349
    • Lebeau Software
Re: [SOLVED] IdHTTP and BAD REQUEST
« Reply #3 on: May 09, 2017, 08:38:52 pm »
Code: Pascal  [Select]
  1. SplitText('&', web_url, StrList);

What is the actual value of web_url?

Code: Pascal  [Select]
  1. StrList[0] := StringReplace(StrList[0],' ','%20',[rfReplaceAll]);

Why are you doing that?  And only for the first string and not the other strings?

In "application/x-www-form-urlencoded", spaces are encoded as "+", not as "%20".  And besides, TIdHTTP.Post() already encodes spaces for you.  What you are doing is problematic if any "%" characters belonging to urlencoded sequences end up in your TStringList strings, because TIdHTTP.Post() will re-encode them as "%25" instead of leaving them as "%", thus corrupting the data the server sees.  Unless you disable the hoForceEncodeParams flag in the TIdHTTP.HTTPOptions property (it is enabled by default).

Code: Pascal  [Select]
  1. Data.Values['q'] := copy(StrList[0],POS('=',StrList[0])+1, Length(StrList[0]));
  2. Data.Values['key'] := copy(StrList[1],POS('=',StrList[1])+1, Length(StrList[1]));
  3. Data.Values['secret'] := copy(StrList[2],POS('=',StrList[2])+1, Length(StrList[2]));
  4. Data.Values['per_page'] := copy(StrList[3],POS('=',StrList[3])+1, Length(StrList[3]));
  5. Data.Values['page'] := copy(StrList[4],POS('=',StrList[4])+1, Length(StrList[4]));

You are assuming the StrList values are in the same order as the Data values.  I would do it more dynamically instead:

Code: [Select]
for I := 0 to StrList.Count-1 do
begin
  J := Pos('=', StrList[I]);
  Data.Values[Copy(StrList[I], 1, J-1)] := Copy(StrList[I], J+1, MaxInt);
end;

Or simpler:

Code: [Select]
for I := 0 to StrList.Count-1 do
begin
  Data.Values[StrList.Names[I]] := StrList.ValueFromIndex[I];
end;

Or, since the strings are clearly already in "name=value" format, just add them as-is instead:

Code: [Select]
for I := 0 to StrList.Count-1 do
  Data.Add(StrList[I]);

Or simpler:

Code: [Select]
Data.Assign(StrList);
In which case, you don't need two separate TStringLists at all.  Get rid of Data and post StrList as-is instead:

Code: [Select]
Result := HTTP1.Post('https://api.discogs.com/database/search', StrList);
Or, if web_url is already in "application/x-www-form-urlencoded" format to begin with (such as if are extracting it from the query portion of a URL), then just post it as-is without parsing it at all:

Code: [Select]
Strm := TStringStream.Create(web_url);
Result := HTTP1.Post('https://api.discogs.com/database/search', Strm);
Strm.Destroy;

The error only occurs when I try to POST

That usually means you are corrupting the data you are posting.

any other request works perfectly using the GET (e.g. everything w/o "per_page")

Sure, because you are not sending any data to the server, you are requesting data from the server instead.
« Last Edit: May 09, 2017, 08:44:11 pm by Remy Lebeau »
Remy Lebeau
Lebeau Software - Owner, Developer
Internet Direct (Indy) open source project - Admin, Developer

DanishMale

  • New member
  • *
  • Posts: 40
Re: [SOLVED] IdHTTP and BAD REQUEST
« Reply #4 on: May 12, 2017, 03:31:06 am »
@Remy Lebeau:

SplitText & StringReplace : The actual value of web_url is static and ONLY the first part is dynamic (StrList[0])

Data.Values are ALWAYS in the same order due to it's hardcoded

I am grateful for your suggestions how to improve my coding and your fantastic explanation on this issue. You pointed out something I should have remembered (e.g. diff between GET and POST) :D  O:-)

Thanks again to all for their great contributions  ;D  :D
Lazarus 1.6 x64 | FPC 3.0.0 | Windows 10 x64 | MySQL Community Server (GPL) 5.6 x64 | MariaDB 10.1.21 x64 | Firebird 3.0.1 x64 | SQLite 3.16.2 x64|MS SQL Server 2012 x64 | PostgresSQL 9.6.1 x64 | Oracle 12.1.0.2 SE x64 |Windows Server 2008 x64

 

Recent

Get Lazarus at SourceForge.net. Fast, secure and Free Open Source software downloads Open Hub project report for Lazarus