rayMedia

[Guva]

adding headers and compiling library for linux - dinamic only. included in raylib,so.550 ( windows is coming soon).

What is ramedia?

Clean and user-friendly extension for raylib that adds seamless audio and video streaming support via the FFmpeg libav* libraries. It enables easy integration of multimedia content into raylib applications, providing direct access to video textures and audio streams, with support for seeking and looping.

autor repo
https://github.com/cloudofoz/raylib-media/

[Guva]

Well, I took out ray media and raygizmo separately.
For linux, linking is static. I haven't compiled it for windows yet.

Code: Pascal  [Select][+][-]

1. unit raymedia;
2.  
3. {$mode objfpc}{$H+}
4. {$packrecords c}
5. {$ALIGN 8}
6. {$MINENUMSIZE 4}
7. // Include configuration file
8. {$I ../raylib.inc}
9.  
10.  
11. interface
12.  
13. uses
14.   raylib;
15.  
16. {$IFNDEF RAY_STATIC}
17. const
18. {$ifdef linux}
19.   cDllName = 'librmedia.so';
20. {$endif}
21. {$endif}
22.  
23. type
24.   PMediaContext = ^TMediaContext;
25.   TMediaContext = pointer; //record end;
26.  
27.   PMediaStream = ^TMediaStream;
28.   TMediaStream = record
29.     videoTexture: TTexture;      // Current video frame texture (if available)
30.     audioStream: TAudioStream;   // Audio stream for playback (if available)
31.     ctx: PMediaContext;  //????         // Internal use only
32.   end;
33.  
34.   TMediaProperties = record
35.     durationSec: double;   // Media duration in seconds
36.     avgFPS: single;        // Average video FPS
37.     hasVideo: boolean;     // True if video is present
38.     hasAudio: boolean;     // True if audio is present
39.   end;
40.  
41. type
42.   TReadFn = function(userData: Pointer; buffer: PByte; bufferSize: Integer): Integer; cdecl;
43.   TSeekFn = function(userData: Pointer; offset: Int64; whence: Integer): Int64; cdecl;
44.  
45.   PMediaStreamReader = ^TMediaStreamReader;
46.   TMediaStreamReader = record
47.     readFn: TReadFn;
48.     seekFn: TSeekFn;
49.     userData: Pointer;
50.   end;
51.  
52.   //Flags for LoadMediaEx() to customize media loading.
53.   PMediaLoadFlag = ^TMediaLoadFlag;
54.   TMediaLoadFlag = Integer;
55.   const
56.     MEDIA_LOAD_AV           = TMediaLoadFlag(0);      // Load audio and video (default)
57.     MEDIA_LOAD_NO_AUDIO     = TMediaLoadFlag(1 shl 1); // Do not load audio
58.     MEDIA_LOAD_NO_VIDEO     = TMediaLoadFlag(1 shl 2); // Do not load video
59.     MEDIA_FLAG_LOOP         = TMediaLoadFlag(1 shl 3); // Loop playback
60.     MEDIA_FLAG_NO_AUTOPLAY  = TMediaLoadFlag(1 shl 4); // Load without starting playback
61.  
62. type
63.   PMediaState = ^TMediaState;
64.   TMediaState = Integer;
65.   const
66.     MEDIA_STATE_INVALID     = TMediaState(-1);     // Not loaded or initialized
67.     MEDIA_STATE_STOPPED     = TMediaState(0);      // Stopped
68.     MEDIA_STATE_PAUSED      = TMediaState(1);      // Paused
69.     MEDIA_STATE_PLAYING     = TMediaState(2);      // Playing (call UpdateMedia() each frame)
70.  
71. type
72.   PMediaConfigFlag = ^TMediaConfigFlag;
73.   TMediaConfigFlag = Integer;
74.   const
75.     MEDIA_VIDEO_QUEUE           = TMediaConfigFlag(0);  // Video packet queue capacity
76.     MEDIA_AUDIO_QUEUE           = TMediaConfigFlag(1);  // Audio packet queue capacity
77.     MEDIA_AUDIO_DECODED_BUFFER  = TMediaConfigFlag(2);  // Maximum decoded audio buffer size
78.     MEDIA_AUDIO_STREAM_BUFFER   = TMediaConfigFlag(3);  // Audio stream buffer size
79.     MEDIA_AUDIO_FORMAT          = TMediaConfigFlag(4);  // Output audio format (refer to MediaAudioFormat)
80.     MEDIA_AUDIO_CHANNELS        = TMediaConfigFlag(5);  // Number of audio channels
81.     MEDIA_VIDEO_MAX_DELAY       = TMediaConfigFlag(6);  // Maximum delay (ms) before discarding a video packet
82.     MEDIA_AUDIO_MAX_DELAY       = TMediaConfigFlag(7);  // Maximum delay (ms) before discarding an audio packet
83.     MEDIA_AUDIO_UPDATE          = TMediaConfigFlag(8);  // Max bytes uploaded to AudioStream per frame
84.  
85. type
86.   PMediaAudioFormat = ^TMediaAudioFormat;
87.   TMediaAudioFormat = Integer;
88.   const
89.     AUDIO_FMT_U8  = TMediaAudioFormat(0);  // Unsigned 8-bit
90.     AUDIO_FMT_S16 = TMediaAudioFormat(1);  // Signed 16-bit (default)
91.     AUDIO_FMT_S32 = TMediaAudioFormat(2);  // Signed 32-bit
92.     AUDIO_FMT_FLT = TMediaAudioFormat(3);  // Float
93.     AUDIO_FMT_DBL = TMediaAudioFormat(4);  // Double
94.  
95.   type
96.     TMediaStreamIOResult = (
97.       MEDIA_IO_EOF     = -541478725, // Конец потока достигнут (соответствует AVERROR_EOF)
98.       MEDIA_IO_INVALID = -22         // Некорректный вызов или операция (соответствует AVERROR(EINVAL))
99.     );
100.  
101. function LoadMedia(const fileName: PChar): TMediaStream; cdecl; external {$IFNDEF RAY_STATIC}cDllName{$ENDIF} name 'LoadMedia';
102. function LoadMediaEx(const fileName: PChar; flags: Integer): TMediaStream; cdecl; external {$IFNDEF RAY_STATIC}cDllName{$ENDIF} name 'LoadMediaEx';
103. function LoadMediaFromStream(streamReader: TMediaStreamReader; flags: integer): TMediaStream; cdecl; external {$IFNDEF RAY_STATIC}cDllName{$ENDIF} name 'LoadMediaFromStream';
104. function IsMediaValid(media: TMediaStream): Boolean; cdecl; external {$IFNDEF RAY_STATIC}cDllName{$ENDIF} name 'IsMediaValid';
105. function GetMediaProperties(media: TMediaStream): TMediaProperties; cdecl; external {$IFNDEF RAY_STATIC}cDllName{$ENDIF} name 'GetMediaProperties';
106. function UpdateMedia(media: PMediaStream): Boolean; cdecl; external {$IFNDEF RAY_STATIC}cDllName{$ENDIF} name 'UpdateMedia';
107. function UpdateMediaEx(media: PMediaStream; deltaTime: Double): Boolean; cdecl; external {$IFNDEF RAY_STATIC}cDllName{$ENDIF} name 'UpdateMediaEx';
108. function GetMediaState(media: TMediaStream): Integer; cdecl; external {$IFNDEF RAY_STATIC}cDllName{$ENDIF} name 'GetMediaState';
109. function GetMediaPosition(media: TMediaStream): Double; cdecl; external {$IFNDEF RAY_STATIC}cDllName{$ENDIF} name 'GetMediaPosition';
110. function SetMediaPosition(media: TMediaStream; timeSec: Double): Boolean; cdecl; external {$IFNDEF RAY_STATIC}cDllName{$ENDIF} name 'SetMediaPosition';
111. function SetMediaLooping(media: TMediaStream; loopPlay: Boolean): Boolean; cdecl; external {$IFNDEF RAY_STATIC}cDllName{$ENDIF} name 'SetMediaLooping';
112. function SetMediaFlag(flag: Integer; value: Integer): Integer; cdecl; external {$IFNDEF RAY_STATIC}cDllName{$ENDIF} name 'SetMediaFlag';
113. function GetMediaFlag(flag: Integer): Integer; cdecl; external {$IFNDEF RAY_STATIC}cDllName{$ENDIF} name 'GetMediaFlag';
114. procedure UnloadMedia(media: PMediaStream); cdecl; external {$IFNDEF RAY_STATIC}cDllName{$ENDIF} name 'UnloadMedia';
115.  
116. implementation
117.  {$IFDEF linux}
118.  {$IFDEF RAY_STATIC}
119.   {$linklib c}
120.   {$linklib m}
121.   {$linklib dl}
122.   {$linklib pthread}
123.   {$linklib librmedia.a}
124.   {$linklib libavcodec}
125.   {$linklib libavformat}
126.   {$linklib libavutil}
127.   {$linklib libswresample}
128.   {$linklib libswscale}
129.  {$endif}
130. {$endif}
131. end.
132.  

Code: Pascal  [Select][+][-]

1. program project1;
2.  
3. {$mode objfpc}{$H+}
4.  
5. uses
6.  cthreads,
7.  Classes, SysUtils, CustApp, raylib, raymedia;
8.  
9. type
10.   { TRayApplication }
11.   TRayApplication = class(TCustomApplication)
12.   protected
13.     procedure DoRun; override;
14.   public
15.     constructor Create(TheOwner: TComponent); override;
16.     destructor Destroy; override;
17.   end;
18.  
19.   const AppTitle = 'raylib - basic window';
20.   var videoMedia : TMediaStream;
21.  
22. { TRayApplication }
23.  
24. constructor TRayApplication.Create(TheOwner: TComponent);
25. begin
26.   inherited Create(TheOwner);
27.  
28.   InitWindow(800, 600, AppTitle); // for window settings, look at example - window flags
29.   InitAudioDevice();
30.   SetWindowState( FLAG_VSYNC_HINT or FLAG_MSAA_4X_HINT or FLAG_WINDOW_RESIZABLE);
31.   SetMediaLooping(videoMedia, true);
32.   videoMedia  := LoadMedia('123.mp4');
33.    // Verify if the media has loaded correctly
34.     if (not IsMediaValid(videoMedia))  then
35.     begin
36.         TraceLog(LOG_ERROR, 'Failed to load media file: %s', '123.mp4');
37.         CloseAudioDevice();
38.         CloseWindow();
39.     end;
40.      // Set the media to play in a continuous loop
41.     SetMediaLooping(videoMedia, true);
42.  
43. end;
44.  
45. procedure TRayApplication.DoRun;
46. var srcRect, dstRect: TRectangle;
47. begin
48.  
49.   while (not WindowShouldClose) do // Detect window close button or ESC key
50.   begin
51.  // Update the media stream based on frame timing
52.       UpdateMedia(@videoMedia);
53.     // Draw
54.     BeginDrawing();
55.       ClearBackground(RAYWHITE);
56.       DrawFPS(10,10);
57.       // Calculate the coordinates to center the video in the window
58.       srcRect.Create( 0, 0, videomedia.videoTexture.width, videomedia.videoTexture.height);
59.       dstRect.Create(0,0, GetScreenWidth, GetScreenHeight);
60.        DrawTexturePro(Videomedia.videoTexture,
61.        srcRect, dstRect, Vector2Create( 0, 0 ), 0.0, WHITE);
62.     EndDrawing();
63.   end;
64.  
65.   // Stop program loop
66.   Terminate;
67. end;
68.  
69. destructor TRayApplication.Destroy;
70. begin
71.   UnloadMedia(@Videomedia);
72.   // De-Initialization
73.   CloseWindow(); // Close window and OpenGL context
74.  
75.   // Show trace log messages (LOG_DEBUG, LOG_INFO, LOG_WARNING, LOG_ERROR...)
76.   TraceLog(LOG_INFO, 'your first window is close and destroy');
77.  
78.   inherited Destroy;
79. end;
80.  
81. var
82.   Application: TRayApplication;
83. begin
84.   Application:=TRayApplication.Create(nil);
85.   Application.Title:=AppTitle;
86.   Application.Run;
87.   Application.Free;
88. end.
89.  
90.  

[Guva]

I finally managed to compile and achieve stable performance in windows.
Now it remains to do all this in a separate plug-in package.

[Roland57]

Thanks for sharing. By the way, very impressed by all your work around raylib!

I can compile your demo here (Linux Mageia 64), but the program crashes and says that my MP4 file is invalid (if I understand correctly).

Code: Text  [Select][+][-]

1. INFO: Initializing raylib 5.6-dev
2. INFO: Platform backend: DESKTOP (GLFW)
3. INFO: Supported raylib modules:
4. INFO:     > rcore:..... loaded (mandatory)
5. INFO:     > rlgl:...... loaded (mandatory)
6. INFO:     > rshapes:... loaded (optional)
7. INFO:     > rtextures:. loaded (optional)
8. INFO:     > rtext:..... loaded (optional)
9. INFO:     > rmodels:... loaded (optional)
10. INFO:     > raudio:.... loaded (optional)
11. INFO: DISPLAY: Device initialized successfully
12. INFO:     > Display size: 1366 x 768
13. INFO:     > Screen size:  800 x 600
14. INFO:     > Render size:  800 x 600
15. INFO:     > Viewport offsets: 0, 0
16. INFO: GLAD: OpenGL extensions loaded successfully
17. INFO: GL: Supported extensions count: 221
18. INFO: GL: OpenGL device information:
19. INFO:     > Vendor:   Intel
20. INFO:     > Renderer: Mesa Intel(R) HD Graphics 5500 (BDW GT2)
21. INFO:     > Version:  4.6 (Core Profile) Mesa 24.2.8
22. INFO:     > GLSL:     4.60
23. INFO: GL: VAO extension detected, VAO functions loaded successfully
24. INFO: GL: NPOT textures extension detected, full NPOT textures supported
25. INFO: GL: DXT compressed textures supported
26. INFO: GL: ETC2/EAC compressed textures supported
27. INFO: PLATFORM: DESKTOP (GLFW - X11): Initialized successfully
28. INFO: TEXTURE: [ID 1] Texture loaded successfully (1x1 | R8G8B8A8 | 1 mipmaps)
29. INFO: TEXTURE: [ID 1] Default texture loaded successfully
30. INFO: SHADER: [ID 1] Vertex shader compiled successfully
31. INFO: SHADER: [ID 2] Fragment shader compiled successfully
32. INFO: SHADER: [ID 3] Program shader loaded successfully
33. INFO: SHADER: [ID 3] Default shader loaded successfully
34. INFO: RLGL: Render batch vertex buffers loaded successfully in RAM (CPU)
35. INFO: RLGL: Render batch vertex buffers loaded successfully in VRAM (GPU)
36. INFO: RLGL: Default OpenGL state initialized successfully
37. INFO: TEXTURE: [ID 2] Texture loaded successfully (128x128 | GRAY_ALPHA | 1 mipmaps)
38. INFO: FONT: Default font loaded successfully (224 glyphs)
39. INFO: SYSTEM: Working Directory: /home/roland/Documents/raylib/librmedia
40. INFO: AUDIO: Device initialized successfully
41. INFO:     > Backend:       miniaudio | PulseAudio
42. INFO:     > Format:        32-bit IEEE Floating Point -> 16-bit Signed Integer
43. INFO:     > Channels:      2 -> 2
44. INFO:     > Sample rate:   48000 -> 48000
45. INFO:     > Periods size:  3600
46. WARNING: WINDOW: MSAA can only be configured before window initialization
47. WARNING: MEDIA: Trying to set the looping mode of an invalid media.
48. INFO: TEXTURE: [ID 2] Unloaded texture data from VRAM (GPU)
49. INFO: SHADER: [ID 3] Default shader unloaded successfully
50. INFO: TEXTURE: [ID 1] Default texture unloaded successfully
51. INFO: Window closed successfully
52. INFO: your first window is close and destroy

[Guva]

@Roland57 Thanks for testing !

I can compile your demo here (Linux Mageia 64), but the program crashes and says that my MP4 file is invalid (if I understand correctly).

mageia has a pretty old ffmpeg.
You need version 7.1 to work.

[cdbc]

...Okidoki, mine is version 6.1.1 on PCLinuxOS, which is family to Mageia...
But nice to know.
Regards Benny

[Roland57]

mageia has a pretty old ffmpeg.
You need version 7.1 to work.

Indeed!

Code: Text  [Select][+][-]

1. [roland@localhost librmedia]$ ffmpeg -version
2. ffmpeg version 5.1.6 Copyright (c) 2000-2024 the FFmpeg developers

Thanks for the information. I will update and retry.