JPG fast loading

[user5]

Hi folks. I've searched the web and past postings of this forum looking for the fastest way to load graphics and the best that I've come up with is the code below which loads 450 240x180 jpgs in slightly less than 3 sec, which is pretty good. Does anyone know of something even better?

 var AJpg: TJpegImage;

 AJpg := TJpegImage.Create;

 AJpg.LoadFromFile('c:\dir1\pix1.jpg');
 Image1.picture.Bitmap.Assign(AJpg);

Thanks

[user5]

I did some simple math on the stuff in the previous message and it turns out that the code shown loads a 6KB .jpg in .006 sec!

That's pretty darn fast.

All praise to Lazarus...

[derek.john.evans]

EDIT: Updated the code to include a standard TBitmap loader.

This reminds me of a task Ive been putting off for ages. Years ago, VisualCLX patched FreeCLX to use OleLoadPicture to load JPEG's, which was a lot faster back then.

The patch can be found here: (Search for OleLoadPicture)
http://unvclx.sourceforge.net/archive/index.php?version=v3.10&show=25&hl=1

I was unable to get OleLoadPicture working in Lazarus, but I have got OleLoadPictureFile working.

Here is the unit file. You can switch between a Lazarus picture load, OleAut32 picture load via a boolean, soo....

I'd be interested in what the difference is, and should the Win32 LCL contain a similar patch?

Code: Pascal  [Select][+][-]

1. unit OleAut32PictureLoad;
2.  
3. {$MODE DELPHI}
4.  
5. interface
6.  
7. uses Graphics, SysUtils;
8.  
9. procedure BitmapLoadFromFile(const ABitmap: TBitmap; const AFileName: TFileName;
10.   const AUseOle: Boolean);
11. procedure PictureLoadFromFile(const APicture: TPicture; const AFileName: TFileName;
12.   const AUseOle: Boolean);
13.  
14. implementation
15.  
16. uses
17.   ActiveX, Classes, Forms, Types;
18.  
19. function OleLoadPictureFile(varFileName: olevariant; var lpdispPicture: IDispatch): HResult;
20.   stdcall external oleaut32dll Name 'OleLoadPictureFile';
21.  
22. const
23.   HIMETRIC_INCH = 2540;
24.  
25. procedure BitmapLoadFromFile(const ABitmap: TBitmap; const AFileName: TFileName;
26.   const AUseOle: Boolean);
27. var
28.   LDispatch: IDispatch;
29.   LPicture: IPicture;
30.   LMetricX: OLE_XSIZE_HIMETRIC;
31.   LMetricY: OLE_YSIZE_HIMETRIC;
32. begin
33.   LDispatch := nil;
34.   if AUseOle and (OleLoadPictureFile(AFileName, LDispatch) = S_OK) then
35.   begin
36.     LPicture := LDispatch as IPicture;
37.     LPicture.get_Width(LMetricX);
38.     LPicture.get_Height(LMetricY);
39.     ABitmap.SetSize(LMetricX * Screen.PixelsPerInch div HIMETRIC_INCH,
40.       LMetricY * Screen.PixelsPerInch div HIMETRIC_INCH);
41.     LPicture.Render(wireHDC(ABitmap.Canvas.Handle), 0, 0, ABitmap.Width,
42.       ABitmap.Height, 0, LMetricY, LMetricX, -LMetricY, PRect(nil)^);
43.   end else begin
44.     ABitmap.LoadFromFile(AFileName);
45.   end;
46. end;
47.  
48. procedure PictureLoadFromFile(const APicture: TPicture; const AFileName: TFileName;
49.   const AUseOle: Boolean);
50. begin
51.   if AUseOle then
52.   begin
53.     BitmapLoadFromFile(APicture.Bitmap, AFileName, True);
54.   end else begin
55.     APicture.LoadFromFile(AFileName);
56.   end;
57. end;
58.  
59. end.  
60.  

[derek.john.evans]

I just did some testing: (Tests are on a Intel Core 2 Duo 3.0 Windows XP. So, it might be different on Windows 7/8/Vista)

179 JPEG images. Total size 104 MB.

OleAut32 (IPicture) = 41 seconds
Lazarus TPicture = 55 Seconds

EDIT:
TBitmap.Assign(TJpegImage) also came in at 55 seconds (actually 56 seconds)
TPicture.Bitmap.Assign(TJpegImage) also 55 seconds.

Which isn't a big difference, but, if you are loading 100 pixel width thumbs, it takes 33 seconds.

Code: Pascal  [Select][+][-]

1. procedure BitmapThumbLoadFromFile(const ABitmap: TBitmap; const AFileName: TFileName;
2.   const AWidth: Integer);
3. var
4.   LDispatch: IDispatch;
5.   LPicture: IPicture;
6.   LMetricX: OLE_XSIZE_HIMETRIC;
7.   LMetricY: OLE_YSIZE_HIMETRIC;
8. begin
9.   LDispatch := nil;
10.   if OleLoadPictureFile(AFileName, LDispatch) = S_OK then
11.   begin
12.     LPicture := LDispatch as IPicture;
13.     LPicture.get_Width(LMetricX);
14.     LPicture.get_Height(LMetricY);
15.     ABitmap.SetSize(AWidth, AWidth * LMetricY div LMetricX);
16.     LPicture.Render(wireHDC(ABitmap.Canvas.Handle), 0, 0, ABitmap.Width,
17.       ABitmap.Height, 0, LMetricY, LMetricX, -LMetricY, PRect(nil)^);
18.   end else begin
19.     raise Exception.Create('Unable to load thumb.');
20.   end;
21. end;
22.  

An alternative is to inherit from TJPEGImage to get access to the TJPEGScale (Defined in FPReadJPEG)

Code: Pascal  [Select][+][-]

1. TJPEGScale = (jsFullSize, jsHalf, jsQuarter, jsEighth);
2.  

Which is fast, but the resulting quality can be unpredictable depending on the JPEG. What I've done in the past for a thumbnail view, is to use the fastest possible method for a draft view, so either TJPEGScale=jsEighth or maybe the OleAut32 stuff, then run a timer/thread to load improved thumbs.

Code: Pascal  [Select][+][-]

1. type
2.  
3.   TJPEGImageEx = class(TJPEGImage)
4.   strict private
5.     FScale: TJPEGScale;
6.   protected
7.     procedure InitializeReader(AImage: TLazIntfImage; AReader: TFPCustomImageReader); override;
8.   public
9.     property Scale: TJPEGScale read FScale write FScale;
10.   end;
11.  
12. procedure TJPEGImageEx.InitializeReader(AImage: TLazIntfImage; AReader: TFPCustomImageReader);
13. begin
14.   (AReader as TFPReaderJpeg).Scale := FScale;
15.   inherited InitializeReader(AImage, AReader);
16. end;  
17.  

Hope all that helps. Thats about as far as I went with JPEG loading. I know INTEL has a JPEG library, which I haven't tried: (Here is a doc that compares it to Microsoft's IPicture.

http://www.drdobbs.com/inside-intels-jpeg-library/184405097

*** EDIT ****

Curiosity got the better of me. So I looked for some Delphi INTEL JPEG Library code.

After hacking some code from here, it seems the Intel Jpeg Lib loads the same images in 12 seconds!

http://bouchez.info/myjpeg.html

Here is my hackup version of MyJpeg. There are graphics libraries that support IJL which seems like it would be a good thing to investigate.

www.wascal.net/bin/LazMyJpeg.zip

[user5]

    derek.john.evan, thank you for your extremely interesting, extensive and potentially very exciting message on loading .jpgs quickly. I'm a little tired at the moment but later I'll certainly study what you posted and yes, it could be very helpful to me. It looks like you went to some effort and I'm sure that I and some other folks appreciate it.

    I say your note is exciting because you may have given me the key I needed to be able to reduce the size of the program that I'm working on and to raise its playback performance level. Maybe, maybe not. I don't know yet. To explain as briefly as possible, my program currently has several methods of playing a project that will be made into a custom .mpg or .avi. One method uses MPlayer and while audio synchronization and loading speed are good, it can't display transparencies (as far as I know), it adds 30 Mb to the program size and exhibits some slight color shifts.

    The other method loads the individual frames one after another from up to seven imported videos or animated .gifs into seven or less TImages for display while the program captures them into the project movie being made. The great advantage of this method is that transparencies are allowed, there are no color shifts, the self-adjusting sync is good and you basically see what you'll get in the finished movie.
    The only drawback (up to now) is the fact that a project with more than two or three imported movies plays a bit slow because the program can't load all those images quite fast enough.
    The thing is though, it almost is fast enough and if I can just increase the loading speed overall by a few hundredths of a second then everything will be peachy keen and I can do without MPlayer and its 30 Mb.
    When I get a chance I may try out your suggested code and let ya know what the results are.

    This program makes clips up to five minutes long which are then joined to any longer movie that may be under construction.

    From the info that you gave, it appears that you're using uncompressed .jpgs that are quite large. The .jpgs that I'll be using will never be larger than 8 Kb each, with a maximum size of 528x386. That's pretty small storage space for a large picture and it helps with loading speeds.

    Thanks again for your input. I enjoy sharing information too when I think I've got something useful to others, which isn't often for me because this forum has some sharp folks who may already know what I might say.


   

[user5]

     I haven't seen anything that is faster than TJpegImage. derek.john.evans, I couldn't get any of the stuff that you posted to work correctly, which is probably because I'm doing something wrong. I downloaded Vampyre but I haven't used it yet.

    For now, I'm going to have two methods of playback in my program; One using MPlayer (MPlayerControl) and the other one using TJpegImage. The second method works pretty well though it's load-sensitive and plays slightly slower than it should if there are a lot of videos (up to seven) in the project or the computer being used is old or if other programs are running at the same time. 
    I may try to use multithreading in conjuntion with TJpegImage; When it's time to display say, seven frames for capturing, the program (if possible) could start some threads to load six of the TImages at relatively the same time, followed by one more loading of the seventh TImage (To ensure that the code execution goes from 'parallel' execution back to linear execution).

    If any of you guys are aware of anything that you know is faster than TJpegImage then I'd sure like to hear about it, but thanks in any event.

[derek.john.evans]

Just to clarify. Are you saying you are pre converting selected movie frames to JPEG's which are then used to display in a movie frame navigation tool? ie: a preview

Or, are are you converting the entire movie to JPEG's for editing, then joining them back together once editing is done?

Are you using something like FFmpeg to extract all the JPEG's?

Im unsure what your program actually is 

[derek.john.evans]

Ok, if you are doing what I think you are doing (and I might be completely wrong), the only solution that works here (2.8 P4 WinXP) is adjusting TJPEGScale before loading the JPEG.

I used FFmpeg to convert a 7 minute movie to JPEG's at 24 FPS. The result is 10101 files which is about 7 * 60 * 24 = 10080. The frame size is 852 x 480, which is larger than what you say your max is (528x386). I then wrote a JPEG cache (50 slots), and a simple threaded buffer which reads 24 frames ahead of the frame position.

The code chugs at Scale := jsFullSize but runs smoothly at jsHalf. I can get 4 movies running at jsQuarter and 8 movies using jsEighth.

Performance := jpBestSpeed may help, but in my tests I cant see any significant speed increase.

Here is the JPEGImageEx unit which gives you access to Scale. (EDIT: Performance is already avaliable)

Code: Pascal  [Select][+][-]

1. unit JPEGImageEx;
2.  
3. {$MODE DELPHI}
4.  
5. interface
6.  
7. uses
8.   FPimage, FPReadJPEG, Graphics, IntfGraphics;
9.  
10. type
11.   TJPEGImageEx = class(TJPEGImage)
12.   strict private
13.     FScale: TJPEGScale;
14.   protected
15.     procedure InitializeReader(AImage: TLazIntfImage; AReader: TFPCustomImageReader); override;
16.   public
17.     property Scale: TJPEGScale read FScale write FScale;
18.   end;
19.  
20. implementation
21.  
22. procedure TJPEGImageEx.InitializeReader(AImage: TLazIntfImage; AReader: TFPCustomImageReader);
23. begin
24.   (AReader as TFPReaderJpeg).Scale := FScale;
25.   inherited InitializeReader(AImage, AReader);
26. end;
27.  
28. end.  
29.  

This is the code I used to test a cache/buffer idea. You may find ideas in there, but I wouldn't use the code. Not thread safe and not fully tested.

Code: Pascal  [Select][+][-]

1. unit JpegMovie;
2.  
3. {$MODE DELPHI}
4.  
5. interface
6.  
7. uses
8.   Classes, Contnrs, FPReadJPEG, Graphics,
9.   JPEGImageEx, Math, SysUtils;
10.  
11. type
12.  
13.   TJPEGMovieFrame = class(TJPEGImageEx)
14.   private
15.     FFileName: TFileName;
16.     procedure SetFileName(const AFileName: TFileName);
17.   end;
18.  
19.   TJPEGMovieCache = class(TObjectList)
20.   private
21.     FMaxSize: Integer;
22.   public
23.     constructor Create(const AMaxSize: Integer);
24.     function LoadFromFile(const AFileName: TFileName): TJPEGMovieFrame;
25.   end;
26.  
27.   TJPEGMovieBuffer = class(TThread)
28.   strict private
29.     FMoviePath: String;
30.     FFrameCache: TJPEGMovieCache;
31.     FFrameCount: Integer;
32.     FFrames: array of TJPEGMovieFrame;
33.     FPosition: Integer;
34.   protected
35.     procedure Execute; override;
36.     function GetFrameFileName(const AIndex: Integer): TFileName;
37.   public
38.     constructor Create(const AMoviePath: String);
39.     destructor Destroy; override;
40.   public
41.     function GetFrame(const APosition: Integer): TJPEGImageEx;
42.   end;
43.  
44. implementation
45.  
46. procedure TJPEGMovieFrame.SetFileName(const AFileName: TFileName);
47. begin
48.   FFileName := AFileName;
49.   try
50.     Scale := jsQuarter;
51.     Performance := jpBestSpeed;
52.     LoadFromFile(AFileName);
53.   except
54.   end;
55. end;
56.  
57. constructor TJPEGMovieCache.Create(const AMaxSize: Integer);
58. begin
59.   inherited Create(True);
60.   FMaxSize := AMaxSize;
61. end;
62.  
63. function TJPEGMovieCache.LoadFromFile(const AFileName: TFileName): TJPEGMovieFrame;
64. var
65.   LIndex: Integer;
66. begin
67.   for LIndex := 0 to Count - 1 do
68.   begin
69.     Result := Items[LIndex] as TJPEGMovieFrame;
70.     if SameFileName(Result.FFileName, AFileName) then
71.     begin
72.       Move(LIndex, 0);
73.       Exit;
74.     end;
75.   end;
76.   Result := TJPEGMovieFrame.Create;
77.   Insert(0, Result);
78.   if Count > FMaxSize then
79.   begin
80.     Delete(Count - 1);
81.   end;
82.   Result.SetFileName(AFileName);
83. end;
84.  
85. constructor TJPEGMovieBuffer.Create(const AMoviePath: String);
86. begin
87.   FMoviePath := AMoviePath;
88.   FFrameCache := TJPEGMovieCache.Create(50);
89.   FFrameCount := 0;
90.   while FileExists(GetFrameFileName(FFrameCount + 1)) do
91.   begin
92.     Inc(FFrameCount);
93.   end;
94.   SetLength(FFrames, FFrameCount);
95.   inherited Create(True);
96. end;
97.  
98. destructor TJPEGMovieBuffer.Destroy;
99. begin
100.   Terminate;
101.   WaitFor;
102.   inherited Destroy;
103.   FreeAndNil(FFrameCache);
104. end;
105.  
106. function TJPEGMovieBuffer.GetFrameFileName(const AIndex: Integer): TFileName;
107. begin
108.   Result := FMoviePath + IntToStr(AIndex) + '.jpg';
109. end;
110.  
111. procedure TJPEGMovieBuffer.Execute;
112. var
113.   LIndex: Integer;
114. begin
115.   while not Terminated do
116.   begin
117.     for LIndex := Max(FPosition, 0) to Min(FPosition + 24, High(FFrames)) do
118.     begin
119.       FFrames[LIndex] := FFrameCache.LoadFromFile(GetFrameFileName(LIndex));
120.     end;
121.     Suspend;
122.   end;
123. end;
124.  
125. function TJPEGMovieBuffer.GetFrame(const APosition: Integer): TJPEGImageEx;
126. begin
127.   FPosition := APosition;
128.   if InRange(APosition, 0, High(FFrames)) then
129.   begin
130.     Resume;
131.     Result := FFrames[APosition];
132.   end else begin
133.     Result := nil;
134.   end;
135. end;
136.  
137. end.
138.  

[user5]

    Sorry I didn't respond sooner. I've been a bit tied up. The attached zip file contains a picture, a text file and movie file that pretty much show what the program does. That's my voice in the movie.
    The program is about 85% finished. Yes, it plays and records from intact videos as well as individual frames.
    You seem like you really know your beans and I'm studying the stuff you posted.
    Say, are you familiar with multithreading and would you look at some multithreading code that I need help on? If not, then I can post my multithreading query in this forum's General section. I know that this forum's archive has a lot of stuff on multithreading but a lot of it is over my head. I'm pretty new to multithreading but perhaps it's not what I need. Thanks again.
 

[derek.john.evans]

Mmmm. That sounds like a lot of movie/image compositing. Its hard to say what I'd do without knowing what has been done.

I'd implement a couple of object resource caches for TBitmaps and TJPEGImages. I'd then have a thread for each movie which renders frames ahead of the movie frame position and stores the results in an array of TBitmap (which is the size of the number of frames in the movie).

The idea is to set the movie position in the main thread, and the threads will run in the background filling in the TBitmap array. You grab the TBitmap for the current movie position and render.

I guess using caches removes the need to allocate/free objects, which simplifies the relationship between the main thread and the processing threads.

Hope that helps. It might be completely wrong for what you want. Also, I think if your code is working fine, but just cant handle more projects, then quality reduction and frame skipping are solutions.

[user5]

When a movie is being made by capturing a specific area of the screen in my program, the user at that point can choose the rate at which the frames are processed, as well as whether that processing is done from disk or memory but the idividual frames are in fact stored on disk. I'm amazed that loading jpgs from disk is actually very fast, almost as fast as memory!

You are quite correct that all those frames can take up a lot of disk space, too much to be stored in memory. A five minute clip (The maximum length that may be joined to a longer movie that's under construction) at 30 fps = (9000 frames * 8 Kb) = 72 Mg. If the project has the maximum number of imported videos (7) then that's 504 Mb temporary storage space in addition to an occasional full lenght converted copy of the original import. I'll probably lower the maximum clip size to 3 minutes.

Each clip is like a scene in a movie. Most computers can temporarily spare 302 Mb of space but the program also checks each step of the way to make sure that there's enough disk space and the amount of needed space won't always be the maximum. The program also does some tricks to keep that space as small as possible. This frame stuff is pretty recent and I may make it be simply another option for the user.

MPlayerControl is NOT load sensitive while this new frame stuff is, but the frame method is a more perfect playback as long as the computer processor isn't over burdened.

You sound like you know about multithreading so I'll post the best functioning multithreading code that I currently have and describe what I'm trying to do with it and what I need from it. Sorry that the movie I attached was so short but I had a hard time keeping within this forum's maximum file attachment size limits.

All praise to Lazarus of course and thanks so much for helping me. All the rest of the folks here have been helping me for a long time. There are some really sharp folks here. Jeez

[user5]

derek.john.evan, your suggestion about threading is what I planned to do and your ideas about processing the captured bitmaps is similar to what the program already currently does, if the user decides to use memory instead of disk for processing. We are thinking along the same lines. The program uses a ram disk instead of an array but both methods use live memory. The program gives the user a wide and convenient range of options to get the most speed as possible. Of course, one downside to using a ram disk is that its reserved memory is not available for computer processing and it's also not very dynamic but I'm going to leave it the way it is. I need to move on here if I'm going to have the prototype ready in five months.

I might post my message about multithreading in this forum's General section because it's about more than graphics and I might reach more folks there. If I do, then I'll mention this thread. Since you have tried so much to help me, as a courtesy to you I've attached to this message some screenshots of the most important windows of this program. The Custom page is the main activity page. The other screenshots are self-explanatory.

The screenshots were made while testing the program on an older 512 Mb Dell computer. It runs well on my faster computer too. I'm designing the program so that it will run on any Windows computer. Might adapt it for other OS systems later.

[user5]

Help! I got all excited about possibly using dspack for Lazarus from github.com/TheBlackSheep/DSPack-Lazarus but when I try to install the component I get an error message directing me to the Package Graph window which tells me that I need the package pl_win_directx, which led me to CodeTyphoon. Never heard of it. Do I download and install CodeTyphoon and then try again?

I'm sorry that I'm always needing help but I would sure appreciate some here. derek.john.evan, I know that I've already presumed upon you greatly but are you familiar with dspack for Lazarus? Thanx.

[user5]

Well, I just found out that the zip file for CodeTyphoon is almost 600 Mb so unless there's something I don't know then I guess this won't work out.

[derek.john.evans]

avra posted information about these links.

https://bitbucket.org/avra/ct2laz
http://forum.lazarus.freepascal.org/index.php?topic=28897.0

http://www.pilotlogic.com/sitejoom/index.php/forum/installation-setup/2963-sourcefilemanager-pas

Direct links to Typhon components:
http://www.pilotlogic.com/codetyphon/current/src/typhon_src.7z
http://www.pilotlogic.com/codetyphon/current/src/CodeOcean.7z