I modified the demo that I posted for
https://forum.lazarus.freepascal.org/index.php/topic,50444.msg368175.html#msg368175 such that the Graphics32 library is not needed. It does not use BGRABitmap either, just the regular LCL routines in one case, and in the other case the high-speed Intel Jpeg Library (IJL) which, however, is only for Windows and for 32 bit applications.
Your image loads within 1.2s with the standard LCL jpeg reader, but within 0.2 s using the IJL dll.
If you cannot use IJL you could try the Vampyre Imaging Library (available in OPM) or FreeImage (google for it). Both of them are cross-platform, and, as I remember from an old Delphi project, faster than the Delphi jpeg reader, although not as fast as IJL. And both of them can read a variety of formats.
Another reason why picasa appears to be fast could be that it pre-loads the next image from the image directory in the background while the previously loaded image is viewed.