Lazarus

Programming => Graphics and Multimedia => Audio and Video => Topic started by: Phoenix on February 28, 2017, 01:40:59 pm

Title: [SOLVED] UOS: play from TMemoryStream
Post by: Phoenix on February 28, 2017, 01:40:59 pm
With UOS there a way to run a sound directly from a TMemoryStream without using the net or going to save a file.
I only saw
uos_AddFromFile (PlayerIndex1, (pchar (SoundFilename)));
but a function of the type
ms: TMemoryStream
uos_AddFromMStream (PlayerIndex1, ms);
or by passing the file extension if needed
uos_AddFromMStream (PlayerIndex1, ms, ext);
it's possible

Thanks, any help is appreciated  :)
Title: Re: UOS: play from TMemoryStream
Post by: Fred vS on February 28, 2017, 10:52:32 pm
Hello.

It is possible to first load file into memory with uos_AddFromFile().

Then you may use  uos_PlayNoFree(), it will keep the file into memory even after uos_Stop().

Please, ask it to uos forum (http://uos.2369694.n4.nabble.com), you will receive more infos.

Fre;D
Title: Re: UOS: play from TMemoryStream
Post by: molly on March 01, 2017, 04:31:34 am
@Fred vS
The answer to Phoenix's question is no.

You can not create a sound 'in memory' and then play it with UOS. You are always required to store it to disk first and then load it again using uos_AddFromFile().

TS probably want to do this, f.e. creating a in memory wav file, then play that. In that situation it seems a bit redundant to store things to disk first.

I only had a quick glance over the sources of UOS but hypothetically speaking does UOS not offer the possibility to (in the end) supply the data using a Pascal stream (whether it be a filestream or a memorystream) ?
Title: Re: UOS: play from TMemoryStream
Post by: Phoenix on March 01, 2017, 10:13:42 am
@molly exactly, I have an encrypted data structure where I fetch the files and decode them. Everything happens in memory. So to me it is fundamental the use of TMemoryStream without writing anything. uos_AddFromFile () I can not use it. I saw that in "uos_libsndfile.pas" in addition to "sf_open ()" there is also "sf_open_virtual" I do not understand how to exploit it  %).
However also I follow the advice of Fred vS for the forum.
Thank you!!  :)
Title: Re: UOS: play from TMemoryStream
Post by: molly on March 01, 2017, 10:37:12 am
@Phoenix:
Well spotted  :)

See here (http://www.mega-nerd.com/libsndfile/api.html#open_virtual) for the api call documentation:
Quote
Code: [Select]
SNDFILE* sf_open_virtual (SF_VIRTUAL_IO *sfvirtual, int mode, SF_INFO *sfinfo, void *user_data) ;
Opens a soundfile from a virtual file I/O context which is provided by the caller. This is usually used to interface libsndfile to a stream or buffer based system. Apart from the sfvirtual and the user_data parameters this function behaves like sf_open.

With a lot of details added in the original documentation. Problem is, i also have no idea (*) how to exploit it (nor do i currently have the time to look into it).

(*) well the function itself opens up some possible scenario's but any implementation should also fit/match Fred's original work.

edit:
In the mean while you could perhaps have a look at the virtualio test example (https://github.com/erikd/libsndfile/blob/master/tests/virtual_io_test.c) that accompanies libsndfile.
Title: Re: UOS: play from TMemoryStream
Post by: Phoenix on March 01, 2017, 11:03:21 am
@molly thanks, I already posted on the forum Fred. Maybe it can find it interesting as a starting point for a new library functionality. For now I hope to find a solution  O:-)

(sorry for my English)
Title: Re: UOS: play from TMemoryStream
Post by: Phoenix on March 01, 2017, 11:36:16 am
@molly I saw the sample file and it is a good starting point.
Thank you!!  :)
Title: Re: UOS: play from TMemoryStream
Post by: Fred vS on March 01, 2017, 07:00:28 pm
Quote
You can not create a sound 'in memory' and then play it with UOS.

Huh, did you try AddFromFileIntoMemory() form last commit 610ed5c..d12dca7 ?
This decodes a audio file and stores the result into a memory buffer.
That memory buffer will be used to play the sound.

Quote
, I have an encrypted data structure where I fetch the files and decode them. Everything happens in memory.
What kind of data ?
PCM already decoded?
Or all the raw audio file?

If PCM is already decoded, uos can do it.
If the file is stored into memory, this is not (AFAIK) something for uos but more a fpc challenge.

By the way, if your audio stream is big, you will need access to disk (virtual ram).

Fre;D
Title: Re: UOS: play from TMemoryStream
Post by: molly on March 01, 2017, 07:15:42 pm
Quote
You can not create a sound 'in memory' and then play it with UOS.

Huh, did you try AddFromFileIntoMemory() form last commit 610ed5c..d12dca7 ?
This decodes a audio file and stores the result into a memory buffer.
That memory buffer will be used to play the sound.
Language barrier, i presume  :)

You can't do it without the (current) absolute necessity of storing your data (that is already a valid wav file in memory) to a file on disk first. The you need to load it again from file in order to play it from memory again.

Can you understand the redundancy better now ?  :)

Quote
If the file is stored into memory, this is not (AFAIK) something for uos but more a fpc challenge.
Challenge accepted and finished  :P

I can load and save to a Free Pascal TMemoryStream using the sf_open_virtual() function from libsndfile.

Now... are you up for the challenge to integrate this functionality in the uos library ?  ;-)
Title: Re: UOS: play from TMemoryStream
Post by: Fred vS on March 01, 2017, 07:21:07 pm
Quote
Can you understand the redundancy better now ?  :)

Huh, not perfectly.  :-[

Quote
I can load and save to a Free Pascal TMemoryStream using the sf_open_virtual() function from libsndfile.
Now... are you up for the challenge to integrate this functionality in the uos library ?  ;-)

Haaaa, that is what I love, constructive comment with solution.
And I will be very happy to integrate your challenge in uos and you in the list of great contributors.

Fre;D
Title: Re: UOS: play from TMemoryStream
Post by: molly on March 01, 2017, 07:26:10 pm
@Fred vS
PM send

Huh, not perfectly.  :-[
No worries,. i try again.

Sometimes you have some mix (e.g. multiple channels mixed into one wav file that is created on the fly into memory) that you created yourself.

It is really an very inefficient method to first store that memory data to disk in a file and, then directly load it again (remember the perfect valid wav data is still in memory, because i just saved it to disk), then clear the memory that my wav data just used in order to make room, then load it from the file that i just saved to disk..... which delivers me the same wav data in memory again (the memory that i just cleared).

And, who knows (i am not that familiar with uos)... Perhaps it is possible to implement support for never ending streams this way, e.g. using buffered memory streams with a backend that create music on the fly (as said i have no idea if the libraries used in uos can handle such implementation)

edit:
Hmz, since there seem to be a language barrier, the above wouldn't help much either i guess. Maybe it helps with showing some code ?

Code: Pascal  [Select][+][-]
  1. program silly;
  2.  
  3. {$MODE OBJFPC}{$H+}
  4.  
  5. uses
  6.   Classes;
  7.  
  8. var
  9.   A : String = 'This is some data i would like to use';
  10.  
  11. Procedure OldAndPointless;
  12. var
  13.   S : TMemoryStream;
  14.   T : String;
  15. begin
  16.   S := TMemoryStream.Create;
  17.   S.WriteAnsiString(A);
  18.   S.SaveToFile('pointless.txt');
  19.   S.Free;
  20.  
  21.   S := TMemoryStream.Create;
  22.   S.LoadFromFile('pointless.txt');
  23.   S.Position := 0;
  24.   T := S.ReadAnsiString;
  25.   S.Free;
  26.  
  27.   WriteLn(T);  
  28. end;
  29.  
  30. procedure NewAndEfficient;
  31. var
  32.   S : TMemoryStream;
  33.   T : String;
  34. begin
  35.   S := TMemoryStream.Create;
  36.   S.WriteAnsiString(A);
  37.   S.Position := 0;
  38.   T := S.ReadAnsiString;
  39.   S.Free;
  40.  
  41.   WriteLn(T);  
  42. end;
  43.  
  44. begin
  45.   WriteLn('old:');
  46.   OldAndPointless;
  47.   WriteLn('new:');
  48.   NewAndEfficient;
  49. end.
  50.  

I'm probably quilty of the code above being bit biased but, i do hope yo get the gist of it.
Title: Re: UOS: play from TMemoryStream
Post by: Phoenix on March 01, 2017, 10:47:14 pm
@molly
Thanks for your time !!  :)
@Fred vS
Quote
What kind of data ?
Can be for example a Flac, Ogg, .. (those available through libsndfile)
Quote
PCM already decoded?
Or all the raw audio file?
A source to give the idea for test
Code: Pascal  [Select][+][-]
  1. Const
  2.  ext = '.flac';
  3.  path = 'c:\test'+ext; //!!! ONLY FOR SIMULATE
  4. Var
  5.  ms: TMemoryStream;
  6. begin
  7.  ms:= TMemoryStream.Create;
  8.  try
  9.   ms.LoadFromFile(path);
  10.  
  11.   (*  
  12.        WARNING:
  13.  
  14.        Now forget the const PATH: you have ONLY "ms" and "ext"
  15.        for execute sound
  16.   *)
  17.  
  18.  finally
  19.   ms.Free;
  20.  end;
  21. end;
  22.  

I saw the "AddFromFileIntoMemory ()" method just added but the problem is always the same that I have NO PATH. The audio files are ready to use it only in TMemoryStream.  ;D

As well as in my case, this function may be useful for those who use a game engine and use a unique data structure that contains all the sounds of the game.  :)

Thanks!!  :)
Title: Re: UOS: play from TMemoryStream
Post by: Fred vS on March 02, 2017, 12:07:49 am
Quote
And, who knows (i am not that familiar with uos)... Perhaps it is possible to implement support for never ending streams this way, e.g. using buffered memory streams with a backend that create music on the fly (as said i have no idea if the libraries used in uos can handle such implementation)

This is already implemented in uos.
Take a look at Multi-Input demo.
It gives the tips for a multi-channels layout.

Quote
A source to give the idea for test
Huh, what I see, is something for fpc, so it will be usable not only for flac, ogg or wav but also mp3, opus and custom format.
Something like: 

Code: Pascal  [Select][+][-]
  1. path := ConvertMemoryStreamIntoVirtualFile(ms);

Something more easy should be to store into the memory stream a buffer of float (array of float).
Then, yes uos can deal with it.

Fre;D

Title: Re: UOS: play from TMemoryStream
Post by: Fred vS on March 02, 2017, 01:59:35 pm
Hello.

Added in last commit d35cb28  AddFromMemory(...) to play from memory stream.

Fre;D
Title: Re: UOS: play from TMemoryStream
Post by: marcov on March 02, 2017, 02:25:07 pm
Quote
A source to give the idea for test
Huh, what I see, is something for fpc, so it will be usable not only for flac, ogg or wav but also mp3, opus and custom format.
Something like: 

Code: Pascal  [Select][+][-]
  1. path := ConvertMemoryStreamIntoVirtualFile(ms);

Something more easy should be to store into the memory stream a buffer of float (array of float).
Then, yes uos can deal with it.

Fre;D

The virtual file is not a general concept, and probably only applies to UOS or libraries that it uses. As such it doesn't belong in FPC, but in UOS or the header/libraries that it uses.

A different reason for needed memorystreams btw is the case of when the additional media are in resources. (I do that, but for pictures, not audio)
Title: Re: UOS: play from TMemoryStream
Post by: Fred vS on March 02, 2017, 02:35:06 pm
Quote
he virtual file is not a general concept

And it is also a concept that I will not use.

It seems to me something for hackers (you know those people who prefer to use their brain to steal vs to create).

uos_AddFromMemory(...) uses a array of float as memory stream.

Up to the user to create that input from what he want.

Fre;D
Title: Re: UOS: play from TMemoryStream
Post by: molly on March 02, 2017, 03:12:52 pm
Quote
he virtual file is not a general concept

And it is also a concept that I will not use.

It seems to me something for hackers (you know those people who prefer to use their brain to steal vs to create).

uos_AddFromMemory(...) uses a array of float as memory stream.

Up to the user to create that input from what he want.

Fre;D
I am a bit flabbergasted by that response.

I'm sure it is all due to the language barrier but, for sure there isn't anything hackish about a library (such as libsndfile) offering functionality to implement a virtual backend that can be used to load and save data.

Libraries like libsndfile provide such feature in order to let their functionality integrated independent of the programming language being used.

That allows for better integration in the programming language, in case things can not be done using the 'normal' offered functions to accomplish the task.

In that regards it is nothing more then adding events to a form. If you don't implement the event then you do not use that functionality if you do use the event then you've added support for it. f.e. handling mouse-clicks.

FPC makes heavy use of streams, simply because it allows a simple more or less uniform way whether a user wishes to load something from a file, memory, the internet or whatever the stream retrieve it data from.

Components/classes like zipstream, structured storage and resource streams are a good example of allowing the user to store their data in whatever stream format and offer a uniform way to extract data from those streams.

Quote
uos_AddFromMemory(...) uses a array of float as memory stream.
While that is a good start, it still does not allow for the user to use what he/she likes the most about FPC: working with streams.

Besides that, in case the data is to be 'decoded' first, then the end-user is not allowed to just point to the memory location of the 'encoded' stream.

Therefor a function like uos_addFromMemory() will still not allow the end-user to use streams.

User would have to extract the data from the stream first, store it into memory and then offer it to this function (just as inefficient as the previous writeln exampe i showed).

btw: Yes, that might be an array of float (that is, in case the sound format expects to consist out of floats, otherwise it's another type).

The PM i sended Fred vS is a simple implementation of the virtual functions offered by libsndfile and uses FPC streams to load from and save to. libsndfile itself does not care where the data is actually coming from.

To Fred vS i would like to say:

UOS is your library and you are the maintainer. Either you would like to support FPC streams or not. No hard feelings either way.

The only thing i did is responding to TS question in which TS expressed his question if it was possible to load data from a Stream.

The virtual functions of libsndfile allows to do so. The only thing i do not know if UOS itself is able to make use of that (in theory it should be able to).
Title: Re: UOS: play from TMemoryStream
Post by: Phoenix on March 02, 2017, 04:56:27 pm
Hello @Fred vS,

function uos_AddFromMemory (PlayerIndex: cint32; MemoryBuffer: PDArFloat; OutputIndex: cint32;
   SampleFormat: cint32; SampleRate: cint32; Channels: cint32; FramesCount: cint32): cint32;

It did not accept as a parameter an TMemoryStream but will be useful.
I try to help by implementing a method for using sf_open_virtual that solves the problem.
This is my implementation:  :D

Code: Pascal  [Select][+][-]
  1. Type
  2.  PMemoryStream = ^TMemoryStream;
  3.  sf_count_t = Int64;
  4.  
  5.  //pm_get_filelen = ^tm_get_filelen;
  6.  tm_get_filelen =
  7.   function (pms: PMemoryStream): sf_count_t;
  8.  //pm_seek = ^tm_seek;
  9.  tm_seek =
  10.   function (offset: sf_count_t; whence: Integer; pms: PMemoryStream): sf_count_t;
  11.  //pm_read = ^tm_read;
  12.  tm_read =
  13.   function (buf: Pointer; count: sf_count_t; pms: PMemoryStream): sf_count_t;
  14.  //pm_write = ^tm_write;
  15.  tm_write =
  16.   function (const buf: Pointer; count: sf_count_t; pms: PMemoryStream): sf_count_t;
  17.  //pm_tell = ^tm_tell;
  18.  tm_tell =
  19.   function (pms: PMemoryStream): sf_count_t;
  20.  
  21.  TSF_VIRTUAL = packed record
  22.   sf_vio_get_filelen  : tm_get_filelen;
  23.   sf_vio_seek         : tm_seek;
  24.   sf_vio_read         : tm_read;
  25.   sf_vio_write        : tm_write;
  26.   sf_vio_tell         : tm_tell;
  27.  end;
  28.  
  29. function m_get_filelen(pms: PMemoryStream): sf_count_t;
  30. begin
  31.  Result:= pms^.Size;
  32. end;
  33.  
  34. function m_seek(offset: sf_count_t; whence: Integer; pms: PMemoryStream): sf_count_t;
  35. Const
  36.  SEEK_SET = 0;
  37.  SEEK_CUR = 1;
  38.  SEEK_END = 2;
  39. begin
  40.  case whence of
  41.   SEEK_SET: Result:= pms^.Seek(offset, soFromBeginning);
  42.   SEEK_CUR: Result:= pms^.Seek(offset, soFromCurrent);
  43.   SEEK_END: Result:= pms^.Seek(offset, soFromEnd);
  44.  end;
  45. end;
  46.  
  47. function m_read(buf: Pointer; count: sf_count_t; pms: PMemoryStream): sf_count_t;
  48. begin
  49.  Result:= pms^.Read(buf^,count);
  50. end;
  51.  
  52. function m_write(const buf: Pointer; count: sf_count_t; pms: PMemoryStream): sf_count_t;
  53. begin
  54.  Result:= pms^.Write(buf^,count);
  55. end;
  56.  
  57. function m_tell(pms: PMemoryStream): sf_count_t;
  58. begin
  59.  Result:= pms^.Position;
  60. end;
  61.  
  62. procedure TSSoundUOS.TestLibSndFile;
  63. Var
  64.  sfInfo: TSF_INFO;
  65.  sfVirtual: TSF_VIRTUAL;
  66.  
  67.  procedure ShowInfo;
  68.  Var
  69.   sl: TStringList;
  70.  begin
  71.   sl:= TStringList.Create;
  72.   try
  73.    with sfInfo do
  74.    begin
  75.     sl.Add('frames: '+IntToStr(frames));
  76.     sl.Add('samplerate: '+IntToStr(samplerate));
  77.     sl.Add('channels: '+IntToStr(channels));
  78.     sl.Add('format: '+IntToStr(format));
  79.     sl.Add('sections: '+IntToStr(sections));
  80.     sl.Add('seekable: '+IntToStr(seekable));
  81.    end;
  82.    ShowMessage(sl.Text);
  83.   finally
  84.    sl.Free;
  85.   end;
  86.  end;
  87.  
  88. Const
  89.  path_test = 'c:\test.flac'; //for simulate
  90. Var
  91.  ms: TMemoryStream;
  92. begin
  93.  FillChar(sfInfo, SizeOf(sfinfo), 0);
  94.  sf_open(path_test, SFM_READ, sfInfo);
  95.  ShowInfo;
  96.  
  97.  with sfVirtual do
  98.  begin
  99.   sf_vio_get_filelen  := m_get_filelen;
  100.   sf_vio_seek         := m_seek;
  101.   sf_vio_read         := m_read;
  102.   sf_vio_write        := m_write;
  103.   sf_vio_tell         := m_tell;
  104.  end;
  105.  
  106.  FillChar(sfInfo, SizeOf(sfinfo), 0);
  107.  ms:= TMemoryStream.Create;
  108.  try
  109.   ms.LoadFromFile(path_test);
  110.   ms.Position:= 0;
  111.   sf_open_virtual(@sfVirtual, SFM_READ, @sfInfo, @ms);
  112.   ShowInfo;
  113.  finally
  114.   ms.Free;
  115.  end;
  116. end;  
  117.  

This work on Windows 10 64bit and Lazarus 1.6.2 FPC 3.0.0 64bit  ;D

Now create this

function Tuos_Player.AddFromMStream(ms: TMemoryStream; OutputIndex: cint32;
  SampleFormat: cint32 ; FramesCount: cint32 ): cint32;

Replace the code that uses sf_open with sf_open_virtual

Code: Pascal  [Select][+][-]
  1. Streamin [x] .Data.HandleSt: = sf_open (FileName, SFM_READ, sfinfo);
  2.  
  3. Streamin [x] .Data.HandleSt: = sf_open_virtual(@sfVirtual, SFM_READ, @sfInfo, @ms);
  4.  

is it solve for libsndfile, you only have to test all  :)

Title: Re: UOS: play from TMemoryStream
Post by: molly on March 02, 2017, 05:03:19 pm
This is my implementation:  :D
Shame as that is (almost) exactly what i sended Fred Vs in my PM to him as well ;-)

Quote
Now create this

function Tuos_Player.AddFromMStream(ms: TMemoryStream; OutputIndex: cint32;
  SampleFormat: cint32 ; FramesCount: cint32 ): cint32;

Replace the code that uses sf_open with sf_open_virtual

Code: Pascal  [Select][+][-]
  1. Streamin [x] .Data.HandleSt: = sf_open (FileName, SFM_READ, sfinfo);
  2.  
  3. Streamin [x] .Data.HandleSt: = sf_open_virtual(@sfVirtual, SFM_READ, @sfInfo, @ms);
  4.  

is it solve for libsndfile, you only have to test all  :)
Would that indeed be enough for UOS to be able to add streaming support in a safe manner ?
Title: Re: UOS: play from TMemoryStream
Post by: Fred vS on March 02, 2017, 07:06:10 pm
If you want to use a memorystream, it is up to you.
But, AFAIK, a memorystream needs data.

Then, only use AddFromMemory(data_of_memorystream).

Why do you want to work with streams?
uos does the seek(), read() in the main loop.

In uos there is a utility: 
Code: Pascal  [Select][+][-]
  1. bufferdata := FiletoMemory(filename);

Then store that bufferdata into ressource.

After this, you may use:
uos_AddFromMemory (bufferdata,...);

function uos_AddFromMemory (PlayerIndex: cint32; MemoryBuffer: PDArFloat; OutputIndex: cint32;
   SampleFormat: cint32; SampleRate: cint32; Channels: cint32; FramesCount: cint32): cint32;

It did not accept as a parameter an TMemoryStream but will be useful.

OK, I will add a demo in uos to show how to do  ;D

Fre;D
Title: Re: UOS: play from TMemoryStream
Post by: molly on March 02, 2017, 07:19:23 pm
If you want to use a memorystream, it is up to you.
Meaning that you won't consider adding support for streams ?

A decision i personally can't understand at all nor have i seen any valid reason(s) to not support streams but, it is fine with me if that is to be the choice.
Title: Re: UOS: play from TMemoryStream
Post by: Fred vS on March 02, 2017, 07:32:59 pm
If you want to use a memorystream, it is up to you.
Meaning that you won't consider adding support for streams ?

What are you taking?
uos is full off streams.
OK, I will add a function with TMemoryStream as input.

Thanks for all.

Fre;D
Title: Re: UOS: play from TMemoryStream
Post by: Phoenix on March 03, 2017, 10:40:22 am
Hello,

@molly
After all, we used the same sample in c as a starting point.  ;)

I have tested the changes and indeed the sound works perfectly, but at the end of the sound crashes. Perhaps I created a bug or change is not enough. Even using PlayNoFree and without the Free TMemoryStream not solve.

Aspect of the implementation of Free vS but I was curious to try.  :)

@Fred vS
Thanks to the decision to add the method!!  :D

Thank you all!!
Title: Re: UOS: play from TMemoryStream
Post by: Fred vS on March 03, 2017, 01:26:20 pm
Quote
OK, I will add a demo in uos to show how to do  ;D

Hello.

Added ConsolePlayMemory.pas in last commit  d35cb28..0746826 to show how to use memory buffer as input.
Note that with this added in ressource you will not need to load audio libraries anymore.

uos will do everything alone (only portaudio is needed).

Fre;D
Title: Re: UOS: play from TMemoryStream
Post by: Phoenix on March 06, 2017, 10:31:16 am
Thanks for add  :)
Title: Re: UOS: play from TMemoryStream
Post by: Phoenix on March 12, 2017, 10:59:23 am
The solution has been found !!!  :D :D :D
For those interested look
http://uos.2369694.n4.nabble.com/Load-sound-from-TMemoryStream-td28.html
TinyPortal © 2005-2018