[SOLVED] Error 302 when attempting to fetch Wikipedia page

[maurobio]

Dear ALL,

I have a simple test program which fetches a Wikipedia article for a given scientific name and presents the article summary:

Code: Pascal  [Select][+][-]

1. program wiki_test;
2.  
3. {$APPTYPE CONSOLE}
4. {$mode objfpc}{$H+}
5.  
6. uses  Classes, TypInfo,SysUtils, fphttpclient, fpjson, jsonparser, opensslsockets;
7.  
8. const
9.     url = 'https://en.wikipedia.org/api/rest_v1/page/summary/';
10.     searchName = 'Vicia faba'; // <--- This works
11.         searchName = 'Loxodonta africana'; // <--- This does NOT work
12.  
13. var
14.   jData: TJSONData;
15.   jItem, jItems: TJSONData;
16.   i: integer;
17.  
18. begin
19.   WriteLn('Searching Wikipedia for: ' + searchName);
20.   // Get the page summary
21.   jData := GetJSON(TFPHTTPClient.SimpleGet(url + StringReplace(searchName, ' ', '_', [rfReplaceAll])));
22.   WriteLn(jData.FindPath('extract').AsString);
23.   WriteLn;
24.   jData.Free;
25. end.  

For some species (eg., Vicia faba) it works fine, but for others (eg., Loxodonta africana), the program throws an exception: "EHTTPClient: Unexpected response status code: 302".

According to here (https://blog.airbrake.io/blog/http-errors/302-found), this error arises when "the requested resource has been temporarily moved to a different URI", which suggests that HTTPClient is not handling the (somewhat strange?) redirection of pages performed by Wikipedia.

Could someone  help me to sort out - and hopefully solve - this problem?

Thanks in advance!

With warmest regards,

[TRon]

According to here (https://blog.airbrake.io/blog/http-errors/302-found), this error arises when "the requested resource has been temporarily moved to a different URI", which suggests that HTTPClient is not handling the (somewhat strange?) redirection of pages performed by Wikipedia.

Could someone  help me to sort out - and hopefully solve - this problem?

That can not be realized with simpleget. You'd need to create the httpclient object and set the allowredirect option, see also https://wiki.freepascal.org/fphttpclient#Download_a_file_via_HTTP_protocol (4th example)

[maurobio]

Hi, @TRon!

Great! Based on the example you have pointed out, I amended my code and it now works fine!

Here it is:

Code: Pascal  [Select][+][-]

1. program wiki_test;
2.  
3. {$APPTYPE CONSOLE}
4. {$mode objfpc}{$H+}
5.  
6. uses  Classes, TypInfo,SysUtils, fphttpclient, fpjson, jsonparser, openssl, opensslsockets;
7.  
8. const
9.     url = 'https://en.wikipedia.org/api/rest_v1/page/summary/';
10.     searchName = 'Vicia faba'; // <--- This works
11.         searchName = 'Loxodonta africana'; // <--- This ALSO works
12.  
13. var
14.   Client: TFPHttpClient;
15.   jData: TJSONData;
16.  
17. begin
18.   InitSSLInterface;    
19.   WriteLn('Searching Wikipedia for: ' + searchName);
20.   Client := TFPHttpClient.Create(nil);
21.   try
22.     { Allow redirections }
23.     Client.AllowRedirect := True;
24.     jData := GetJSON(Client.Get(url + StringReplace(searchName, ' ', '_', [rfReplaceAll])));
25.   finally
26.     Client.Free;
27.   end;
28.   { Get the page summary }
29.   WriteLn(jData.FindPath('extract').AsString);
30.   WriteLn;
31.   jData.Free;
32. end.  
33.  

Thank you very much!

With warmest regards,