[SOLVED] Error in most basic TFPHTTPClient usage

[guest58172]

Code: Pascal  [Select][+][-]

1. program Project1;
2.  
3. uses fphttpclient;
4.  
5. begin
6.   try
7.     TFPHTTPClient.SimpleGet('https://api.github.com/repos/octocat/Hello-World');
8.   except
9.     writeln('oops');
10.   end;
11. end.

but if you go to the url, you'll see that a JSON is well generated:

https://api.github.com/repos/octocat/Hello-World

What's wrong here ?

[Phil]

Server is returning error 403.

-Phil

[molly]

try this, and you're quit possibly up for the next error ;-)

Code: [Select]

program simple;

{$MODE OBJFPC}{$H+}

uses
  fphttpclient, SysUtils;


procedure test;
begin
  try
    TFPHTTPClient.SimpleGet('https://api.github.com:443/repos/octocat/Hello-World');
  except
    on E: Exception do
      writeLn('An exception was raised: ' + E.Message);
  end;
end;

begin
  Test;
end.


[guest58172]

Yes but for example libcurl succeeds to bring me the answer.

[molly]

Yes but for example libcurl succeeds to bring me the answer.

Then print out the exception, as shown. I bet you did not even noticed _all_ the changes in my code 

[guest58172]

I didn't see the port but there's always  a 403, with or without a port in the uri.

[Phil]

Try Synapse or Indy or something else and see if that makes a difference.

I tried on Mac with Cocoa's NSMutableURLRequest class and it returned the JSON fine.

-Phil

[molly]

And.... what does error 403 mean ?

Yes but for example libcurl succeeds to bring me the answer.

Then let libcurl be your guide and let it show you what it does to get that result.

I don't believe SimpleGet() is going to cut it.

[guest58172]

I just tried with libcurl in D and I would prefer to use the FCL client for a ObjPascal program, it was just as matter of comparison.

By the way it looks like Github API sucks a bit with authorizations, but I still don't see why libcurl handle them automagically while the FCL web client not...especially since I use on to get something on another site that uses the https protocol too.

[d.ioannidis]

Hi,

  in GitHub API https://developer.github.com/v3/#user-agent-required says "requests MUST include a valid User-Agent header".

AFAIK, FPHTTPClient doesn't provide such header by default.

The following snippet works ...

Code: Pascal  [Select][+][-]

1. program Project1;
2.  
3. {$MODE OBJFPC}{$H+}{$J-}
4.  
5. uses
6.   fphttpclient;
7.  
8. var
9.   cli: TFPHTTPClient;
10.  
11. begin
12.   cli := TFPHTTPClient.Create(nil);
13.   cli.AddHeader('User-Agent','Mozilla/5.0 (compatible; fpweb)');
14.   try
15.     try
16.       Cli.Get('https://api.github.com/repos/octocat/Hello-World');
17.     except
18.       writeln('oops');
19.       readln;
20.     end;
21.   finally
22.     cli.free
23.   end;
24. end.

regards,

[guest58172]

Thx much for reading the doc for me.

[molly]

By the way it looks like Github API sucks a bit with authorizations, but I still don't see why libcurl handle them automagically while the FCL web client not.

And how many code does libcurl need for that... then compare that to your one-liner.

Do you really expect .simpleget() f.e. to automatically redirect, handle client certificates, play animations etc. for you ?

That's why i said use curl as your guide and let it do some verbose logging to see what is required to retrieve that url. Then figure out if SimpleGet() is able to do that (as i said, i don't believe it is). so ditch the simpleget() and use TFPHTTPClient exactly in the same way that curl does. In case that is failing, then there is valid reason to suspect TFPHTTPClient failure/bug (or missing feature).

edit: ah, ic Dimitrios already figured out the exact reason for you 

[d.ioannidis]

Thx much for reading the doc for me.

It's an API. Shouldn't at least had a quick look how to use it ? 

( Couldn't believe that fphttpclient failed in such basic operation .... )

[Phil]

in GitHub API https://developer.github.com/v3/#user-agent-required says "requests MUST include a valid User-Agent header".

Good to know, although that strikes me as an unusual requirement. I've used TFPHTTPClient with dozens of sites and have never run into that requirement before. Perhaps it has more to do with them logging who's retrieving what:

"We request that you use your GitHub username, or the name of your application, for the User-Agent header value. This allows us to contact you if there are problems."

[Phil]

( Couldn't believe that fphttpclient failed in such basic operation .... )

Yes, BBasile should probably file a bug report. Looking at Synapse, it appears it use "Mozilla/4.0 (compatible; Synapse)" as default agent. And Indy appears to use "Mozilla/3.0 (compatible; Indy Library)". So it seems reasonable to expect fcl-web to pass a default agent header.

I assume Apple's NSMutableURLRequest class does that, perhaps identifying itself as Safari, since I didn't need to set any headers for it to work correctly.

-Phil