AResponseInfo.ContentLength := 99999; <------ NO, NO, NO!!
Absolutely DO NOT do that. The ContentLength must be set to the correct number of bytes, not characters, that will be sent:
AResponseInfo.ContentLength := IndyTextEncoding_UTF8.GetByteCount(strJSON);
Or set to -1 (the default) to let TIdHTTPServer calculate the value for you:
AResponseInfo.ContentLength := -1;
AResponseInfo.ResponseNo := 200;
AResponseInfo.ContentType := 'text/html';
AResponseInfo.CharSet := 'utf-8';
AResponseInfo.ContentText := strNotFound;
Why are returning a
success response code (200) if the requested JSON cannot be retrieved? I would suggest either 404 or even 5xx instead.
FIdHTTP.Connect('localhost', 8080);
Do not call Connect() directly on a TIdHTTP. Its various request methods, like Get() and Post(), will call Connect() internally for you when needed. The correct way to use TIdHTTP in this situation is to use the TIdHTTP.Get() method:
strJSON := FIdHTTP.Get('http://localhost:8080/pays');
Which is also all the more reason NOT to have the server returns a 2xx responsse code if the JSON is not available. It should return an error response code so Get() can raise an exception accordingly.
I have the following problems client-side
a) Browsers
i) the text is not rendered correctly (see screenshot)
The screenshot actually shows the JSON text is encoded as UTF-8. The browser is simply interpreting the UTF-8 bytes as ISO-8859-1/Latin-1 when displaying the bytes. For instance, the Unicode character 'é' is encoded in UTF-8 as bytes $C3 $A9, which are the characters 'é' in ISO-8859-1.
That is a browser issue, not a TIdHTTPServer issue. TIdHTTPServer should be sending a 'Content-Type: application/json; charset=utf-8' response header. The browser is likely just ignoring the charset attribute, since technically the official MIME registration for JSON does not include a charset attribute. Per JSON's
MIME registration), the default byte encoding for JSON text is UTF-8, but the browser is likely ignoring that as well. It is likely seeing the media type as a generic 'application/...' and displaying the raw bytes as-is, rather than seeing the media type is specifically 'application/json' and displaying it as UTF-8.
If you include a 'Content-Disposition' response header, you can force a browser to save the JSON to a file instead of display it as text. Then you can verify the encoding of the file using any hex/text viewer that supports UTF-8.
AResponseInfo.ContentDisposition := 'attachment; filename=pays.json';
ii) the entire JSON string is not sent unless AResponseInfo.ContentLength := 99999. I chose a large number to be able to send long strings
As we have discussed on the Embarcadero forum, I
guarantee you that is NOT how TIdHTTPServer operates. Something else is going on to mess with the transmission. TIdHTTPServer does not utilize the ContentLength property when sending the ContentText. Setting the ContentLength property in that case merely provides a value for the 'Content-Length' response header, nothing more. See the source code in the TIdHTTPResponseInfo.WriteHeader() and TIdHTTPResponseInfo.WriteContent() and methods in IdCustomHTTPServer.pas.
In discussions with Remy Lebeau, he said, and it makes perfect sense, that setting AResponseInfo.ContentLength to a predefined number will FORCE clients to keep reading from the socket until those bytes arrive, or a timeout/error occurs.
That is true, per requirement of the HTTP spec (see
RFC 2616 Section 4.4 Message Length).
Remy assures me that it MUST be called before responses are sent
Yes, and TIdHTTPResponseInfo.WriteContent() calls WriteHeader() if it has not already been called:
procedure TIdHTTPResponseInfo.WriteContent;
begin
if not HeaderHasBeenWritten then begin
WriteHeader;
end;
...
end;
TIdCustomHTTPServer.DoExecute() automatically calls WriteHeader() and WriteContent() as need, after relevant event handlers have been called to prepare the response properties.
but I still don't understand why the JSON responses are not being rendered properly in the browser
Actually, it is, just not the way you are expecting. Remember, the 'application/...' media type is for, well, application data, not text data. The browser is displaying a textual representation of something that is technically binary data, not text data. And there is no official 'text/...' media type for JSON (see
What is the correct JSON content type?).
& not being sent completely in browsers and indy clients.
That, I do not know yet. We have not uncovered the real culprit yet.
Use a packet sniffer, like Wireshark or Fiddler, to verify the response is being sent correctly, and in the correct timing. Maybe something sitting in between the client and TIdHTTPServer is hindering the transmission.
Use a debugger to step into the server code to see what it is really doing while processing your request and make sure it is preparing the response correctly.