Recent

Author Topic: JPEG loading color difference FPReadJPEG  (Read 3128 times)

ttomas

  • Full Member
  • ***
  • Posts: 245
JPEG loading color difference FPReadJPEG
« on: November 20, 2020, 02:58:17 pm »
I'm developing image resize web service with BGRABitmap and my customers complain of color difference in resized images. There is not a problem in BGRA lib or resize. Source image is Jpeg and color difference is with loading with FPReadJPEG. I have the same problem with FPMemoryImage, just load image and save to bmp or jpeg, result image have color difference. For testing I load image with libjpegturbo decompressor and the image is also bed. Original jpeg image is displayed correctly in Chrome browser and MS Photos Viewer in W10. Loading same image with MS Paint, LazPaint, IrfanView display incorrect colors.
Someone have this problem with jpeg image, any hint how to solve the problem.

This is the link of original jpeg image
http://halkyon.com/download/p11010_4org.jpg

And this is the same image loaded and saved with Quality 100, same color diff if I save to bmp
http://halkyon.com/download/06-Q100.jpg

I can't change format of source images i suspect in format of source jpeg, but if browser and MS Photos Viewer can open correctly image why not FreePascal application.
« Last Edit: November 20, 2020, 03:04:06 pm by ttomas »

lainz

  • Hero Member
  • *****
  • Posts: 4468
    • https://lainz.github.io/
Re: JPEG loading color difference FPReadJPEG
« Reply #1 on: November 20, 2020, 03:18:53 pm »
Hi, maybe it's the color profile.

wp

  • Hero Member
  • *****
  • Posts: 11912
Re: JPEG loading color difference FPReadJPEG
« Reply #2 on: November 20, 2020, 03:43:14 pm »
Looking at the EXIF data it can be found that the original image contains a ColorSpace tag value of 65535 which means "uncalibrated". Most of the jpg images that I have looked at, however, have a ColorSpace of 1 which means sRGB (except for the images of a friend who is an ambitious photographer and usually works with raw format; here the ColorSpace is uncalibrated again).

Please check the other images that your customers report to have false colors whether the ColorSpace is uncalibrated as well. Many image processing programs display EXIF (e.g. IrfanView mentioned by you).
« Last Edit: November 20, 2020, 03:55:53 pm by wp »

ttomas

  • Full Member
  • ***
  • Posts: 245
Re: JPEG loading color difference FPReadJPEG
« Reply #3 on: November 20, 2020, 04:18:53 pm »
Opening the image in Photoshop ask me to use custom color space or use default.
Is there a embedded custom pallet color space in jpeg format? How to use this color space? What library use Chrome to read jpeg?
@wp Yes all images are the same format (catalog of products for web shop)
IrfanView have the same problem, displayed colors are not correct. Only browser Firefox, Chrome, Edge and MS Photo Viewer and Windows File Explorer thumbnail display correct colors.
« Last Edit: November 20, 2020, 04:20:48 pm by ttomas »

wp

  • Hero Member
  • *****
  • Posts: 11912
Re: JPEG loading color difference FPReadJPEG
« Reply #4 on: November 20, 2020, 05:13:00 pm »
I cannot answer these questions, maybe ask on the Lazarus Mailinglist where the dev who wrote the jpeg unit may be around (in fact, I don't know who he is...).

Googling for "Delphi jpeg colorspace" I came across the NativeJpg library which looks very complete: https://www.simdesign.nl/nativejpg.html. Unfortunately it is a commercial license (although not expensive), but maybe you can contact the author and ask him to load your two images and check whether they look different. And whether his software works for Lazarus, of course.

circular

  • Hero Member
  • *****
  • Posts: 4217
    • Personal webpage
Re: JPEG loading color difference FPReadJPEG
« Reply #5 on: November 20, 2020, 05:37:36 pm »
If the colorspace is undefined, there may not be correct/incorrect colors.

The second image seems darker, which is something you get when converting from sRGB to RGB.

If you call ConvertFromLinearRGB function of TBGRABitmap, do you get the same colors as at the beginning?
Conscience is the debugger of the mind

ttomas

  • Full Member
  • ***
  • Posts: 245
Re: JPEG loading color difference FPReadJPEG
« Reply #6 on: November 20, 2020, 07:03:58 pm »
Opening the image with Gimp on Manjaro display this dialog. Any options Keep or Convert to Adobe RGB or GEMP build in sRGB open image with correct colors.
@circular My customers have the products and they see the difference in colors. If Gimp and other app can show the correct color must be the way for freepascal to load correct colors.

circular

  • Hero Member
  • *****
  • Posts: 4217
    • Personal webpage
Re: JPEG loading color difference FPReadJPEG
« Reply #7 on: November 21, 2020, 02:30:38 pm »
I don't think you understood what I said. Correct means that it is fully defined. The EXIF mentioned by wp seem to be undefined.

Though from the Gimp dialog, it seems that the source colorspace is AdobeRGB colorspace.

Doing the following gives something quite close. There is a little difference, maybe the color matrix for AdobeRGB isn't quite right.
Code: Pascal  [Select][+][-]
  1. uses BGRABitmap, BGRABitmapTypes;
  2. var
  3.   bmp: TBGRABitmap;
  4.   p: PBGRAPixel;
  5.   i: Integer;
  6. begin
  7.   bmp := TBGRABitmap.Create(ExtractFilePath(Application.ExeName)+'p11010_4org.jpg');
  8.   p := bmp.Data;
  9.   for i := 0 to bmp.NbPixels-1 do
  10.   begin
  11.     p^ := TAdobeRGBA.New(p^.red, p^.green, p^.blue, p^.alpha);
  12.     inc(p);
  13.   end;
  14.   bmp.SaveToFile(ExtractFilePath(Application.ExeName)+'p11010_4org_fromadobergb.jpg');
  15.   bmp.Free;
  16. end;  

So if it is possible to detect that the file is in AdobeRGB, that could be a way to fix it.
Conscience is the debugger of the mind

ttomas

  • Full Member
  • ***
  • Posts: 245
Re: JPEG loading color difference FPReadJPEG
« Reply #8 on: November 21, 2020, 04:13:55 pm »
@circular Thanks a lot, I don't know of this TAdobeRGBA function. Source image is in Adobe RGB 98 profile. This info is in ICC Profile in jpeg and now I need a way to read this info from jpeg header. Delphi/FPC jpeg reader don't read this info. Just to inform LazPaint also need to check profile of jpeg image and correct colors when editing like Gimp, Photoshop, etc. Any Adobe RGB profile jpeg is loaded incorrect.

circular

  • Hero Member
  • *****
  • Posts: 4217
    • Personal webpage
Re: JPEG loading color difference FPReadJPEG
« Reply #9 on: November 21, 2020, 06:51:26 pm »
I am interested as well if someone finds a way to identify this. I could then add a patch in BGRABitmap (and so to LazPaint).
Conscience is the debugger of the mind

ttomas

  • Full Member
  • ***
  • Posts: 245
Re: JPEG loading color difference FPReadJPEG
« Reply #10 on: November 21, 2020, 09:53:56 pm »
I research jpeg icc data and find this nice tool
https://github.com/wp-xyz/exif_spy
It is sample in
https://github.com/cutec-chris/dexif
All in Lazarus
If you build exifSpy you can look at APP2 marker, if exists inside you have ICC_PROFILE. Look at FindMarker function. APP2 marker is $E2 and you need to search for $FF$E2 in jpeg file. Next in stream you have 2 bytes for length of ICC_PROFILE and you can extract icc_profile from jpeg. Now next step is to parse and read icc_profile.

circular

  • Hero Member
  • *****
  • Posts: 4217
    • Personal webpage
Re: JPEG loading color difference FPReadJPEG
« Reply #11 on: November 22, 2020, 01:24:21 pm »
What does this ICC profile look like? Is is text or binary?
Conscience is the debugger of the mind

lucamar

  • Hero Member
  • *****
  • Posts: 4219
Re: JPEG loading color difference FPReadJPEG
« Reply #12 on: November 22, 2020, 01:53:33 pm »
Turbo Pascal 3 CP/M - Amstrad PCW 8256 (512 KB !!!) :P
Lazarus/FPC 2.0.8/3.0.4 & 2.0.12/3.2.0 - 32/64 bits on:
(K|L|X)Ubuntu 12..18, Windows XP, 7, 10 and various DOSes.

ttomas

  • Full Member
  • ***
  • Posts: 245
Re: JPEG loading color difference FPReadJPEG
« Reply #13 on: November 22, 2020, 11:31:46 pm »
I can now detect Adobe RGB (1998) jpeg image and apply color correction.
TAdobeRGBA.New is not ideal there is still color difference.
I search for jpeg markers to find APP2 marker. Inside is ICC_PROFILE and pars icc tags for desc (description) tag to find "Adobe RGB (1998)". This is just detection of adobe rgb image.
Real implementation will be to read icc_profile and convert image to sRGB using embedded profile as default. If no profile exist in jpeg sRGB is default.
I attach source for loading jpeg and auto correct adobe rgb. There is $define to enable libjpeg-turbo for loading.
Did someone use libjpeg-turbo in multithreaded app?
I need to use this on ubuntu server with mORMot method based service.
Thanks all for your help!
Code: Pascal  [Select][+][-]
  1. unit hkJpegLoad;
  2.  
  3. {.$define libjpeg-turbo}
  4.  
  5. {$mode objfpc}{$H+}
  6.  
  7. interface
  8.  
  9. uses
  10.   Classes, SysUtils,
  11.   {$ifdef libjpeg-turbo}
  12.   turbojpeg, ctypes,
  13.   {$else}
  14.   FPReadJPEG,
  15.   {$EndIf}
  16.   BGRABitmap,
  17.   BGRABitmapTypes;
  18.  
  19. function LoadJpegFromFile(const aFileName: string): TBGRABitmap;
  20. function LoadJpegFromMemStream(aStream: TMemoryStream): TBGRABitmap;
  21.  
  22. implementation
  23.  
  24. function IsAdobeRGB(aStream: TMemoryStream): boolean;
  25. var pCnt, pSize: Int64;
  26.     p, picc, pdesc: PByte;
  27.     pw: PWord;
  28.     pdw: PDWord;
  29.     len, offset, tagcount: integer;
  30.     pFoundApp2: boolean;
  31.     s, sdesc: ansistring;
  32. begin
  33.   Result := False;
  34.   // Find APP2 marker $FF $E2
  35.  
  36.   // Check valid Jpeg file must start with $FF $D8
  37.   p := aStream.Memory;
  38.   if p^ <> $FF then exit;
  39.   inc(p);
  40.   if p^ <> $D8 then exit;
  41.   pFoundApp2 := False;
  42.   pCnt := 2;
  43.   pSize := aStream.Size;
  44.   inc(p);
  45.   while (pCnt < pSize) do begin
  46.     if p^ <> $FF then
  47.        exit;
  48.     inc(p);
  49.     if p^ = $DA then begin
  50.        // $DA SOS marker stop loop, app2 not found
  51.        exit;
  52.     end;
  53.     if p^ = $E2 then begin
  54.        pFoundApp2 := True;
  55.        break;
  56.     end;
  57.     inc(p);
  58.     pw := PWord(p);
  59.     len := BEToN(pw^);
  60.     inc(p, len);
  61.     pCnt := pCnt + 2 + len;
  62.   end;
  63.   if not pFoundApp2 then
  64.      exit;
  65.   inc(p);
  66.   pw := PWord(p);
  67.   len := BEToN(pw^);  // length of app2 data
  68.   inc(p,2); // point to ICC_PROFILE#0 12-bytes
  69.   inc(p, 12);
  70.   // implemented only 65535 length icc_profile
  71.   if p^<>1 then
  72.      exit;  // first chunk
  73.   inc(p);
  74.   if p^<>1 then
  75.      exit;  // total num of chunks
  76.   inc(p);
  77.   pdw := PDWord(p);
  78.   len := BEToN(pdw^);
  79.   // Now we have ICC_Profile pointing in p with length=len
  80.   picc := p;
  81.   // icc_parsing looking for description tag desc with value "Adobe RGB (1998)"
  82.   inc(p, $80); // go to tag count in tag table
  83.   pdw := PDWord(p);
  84.   tagcount := BEToN(pdw^); // Number of tags;
  85.   inc(p,4);
  86.   SetLength(s, 4);
  87.   while tagcount>0 do begin
  88.     Move(p^, s[1], 4);
  89.     if s='desc' then begin
  90.        inc(p, 4);
  91.        pdw := PDWord(p);
  92.        offset := BEToN(pdw^);
  93.        inc(p, 4);
  94.        pdw := PDWord(p);
  95.        len := BEToN(pdw^);
  96.        pdesc := picc + offset;
  97.        inc(pdesc, 8); // skip desc and 4 $0
  98.        pdw := PDWord(pdesc);
  99.        len := BEToN(pdw^);
  100.        inc(pdesc, 4);
  101.        SetLength(sdesc, len);
  102.        Move(pdesc^, sdesc[1], len);
  103.        if AnsiStrComp(PChar(sdesc), PChar('Adobe RGB (1998)'))=0 then begin
  104.           Result := True;
  105.           break;
  106.        end;
  107.     end;
  108.     inc(p, 12);
  109.     dec(tagcount);
  110.   end;
  111. end;
  112.  
  113. function LoadJpegFromFile(const aFileName: string): TBGRABitmap;
  114. var ms: TMemoryStream;
  115. begin
  116.   ms := TMemoryStream.Create;
  117.   try
  118.     ms.LoadFromFile(aFileName);
  119.     Result := LoadJpegFromMemStream(ms);
  120.   finally
  121.     ms.Free;
  122.   end;
  123. end;
  124.  
  125. function LoadJpegFromMemStream(aStream: TMemoryStream): TBGRABitmap;
  126. var b: TBGRABitmap;
  127.     p: PBGRAPixel;
  128.     i: Integer;
  129.  
  130.   {$ifdef libjpeg-turbo}
  131.   function LoadTurbo: TBGRABitmap;
  132.   var jpegDecompressor: tjhandle;
  133.       jpegWidth,jpegHeight:  cint;
  134.       jpegSubSamp:           cint;
  135.   begin
  136.     jpegDecompressor := tjInitDecompress();
  137.     tjDecompressHeader2(jpegDecompressor, PCUChar(aStream.Memory), aStream.Size, @jpegWidth, @jpegHeight, @jpegSubsamp);
  138.     Result := TBGRABitmap.Create(jpegWidth, jpegHeight);
  139.     tjDecompress2(jpegDecompressor, PCUChar(aStream.Memory), aStream.Size, pcuchar(Result.Data), jpegWidth, 0, jpegHeight, cint(TJPF_BGRA), {TJFLAG_FASTDCT}TJFLAG_ACCURATEDCT);
  140.     tjDestroy(jpegDecompressor);
  141.   end;
  142.   {$else}
  143.   function LoadBGRA: TBGRABitmap;
  144.   var pReader: TFPReaderJPEG;
  145.   begin
  146.     pReader := TFPReaderJPEG.Create;
  147.     try
  148.       pReader.Performance:=jpBestQuality;
  149.       Result := TBGRABitmap.Create;
  150.       Result.LoadFromStream(aStream, pReader, []);
  151.     finally
  152.       pReader.Free;
  153.     end;
  154.   end;
  155.   {$endif}
  156. begin
  157.   {$ifdef libjpeg-turbo}
  158.   b := LoadTurbo;
  159.   {$else}
  160.   b := LoadBGRA;
  161.   {$EndIf}
  162.   if IsAdobeRGB(aStream) then begin
  163.     p := b.Data;
  164.     for i := 0 to b.NbPixels-1 do
  165.     begin
  166.       p^ := TAdobeRGBA.New(p^.red, p^.green, p^.blue);
  167.       inc(p);
  168.     end;
  169.   end;
  170.   Result := b;
  171. end;
  172.  
  173. end.
  174.  

circular

  • Hero Member
  • *****
  • Posts: 4217
    • Personal webpage
Re: JPEG loading color difference FPReadJPEG
« Reply #14 on: November 23, 2020, 12:15:26 am »
Cool. That's a good starting point.  :)

If the color profile is just applied to the RGBA byte values, this can be an opportunity to fix TAdobeRGBA to actually match the colorspace, as that's what it is supposed to be. I've used matrices that are almost the same (to 3 or 4 significant digits) as the one in Adobe documentation. Same thing for the gamma value. See AdobeRGBAToXYZA function in extendedcolorspaces.inc.

There is probably something else to take into account. Changing the reference white to D65 doesn't do much either even though that's probably better as it avoids a chromatic adaptation:
Code: Pascal  [Select][+][-]
  1. SetReferenceWhite(ReferenceWhite2D65); //before using TAdobeRGBA
Conscience is the debugger of the mind

 

TinyPortal © 2005-2018