[SOLVED] Capture sound from line in or microphone

[Fred vS]

Should not the size of the buffer be higher?

8192 is the default size of the input buffer (FramesCount * nb channels).
A uos_player is a independent thread that has a main loop who fill one or more input buffers then sent the result to one or more outputs buffers.

If you want to adjust the size of the input of uos_AddFromDevIn(), use the custom parameters:

Code: Pascal  [Select][+][-]

1. function uos_AddFromDevIn(PlayerIndex: cint32; Device: cint32; Latency: CDouble;
2.   SampleRate: CDouble; OutputIndex: cint32;
3.   SampleFormat: cint32; FramesCount : cint32; ChunkCount: cint32): cint32;
4. // Add a Input from Device Input with custom parameters
5. // PlayerIndex : Index of a existing Player
6. // Device ( -1 is default Input device )
7. // Latency  ( -1 is latency suggested )
8. // SampleRate : delault : -1 (44100)
9. // OutputIndex : Output index of used output// -1: all output, -2: no output, other cint32 refer to a existing OutputIndex  (if multi-output then OutName = name of each output separeted by ';')
10. // SampleFormat : default : -1 (2:Int16) (0: Float32, 1:Int32, 2:Int16)
11. // FramesCount : default : -1 (4096)
12. // ChunkCount : default : -1 (= 512)
13.  

The size of the buffer = FramesCount * Nb Channels. (how short how better latency but too short gives problems you already meet)

And should In1Index always be 0?

If you have only one input, yes.
In1Index is the index of the new added input into the input-array of the player.

Code: Pascal  [Select][+][-]

1. In1Index := uos_AddFromDevIn(PlayerIndex1);

Using uos_AddFromDevIn() to set In1Index is used also to check if all went ok.
If the result = -1, something went wrong adding the input.

You may add many other input into the same player, like mic, audio-file, synth-wave, buffer, etc.

Code: Pascal  [Select][+][-]

1. In2Index := uos_AddFromSynth(PlayerIndex1);
2. In3Index := uos_AddFromFile(PlayerIndex1, afile1);
3. In4Index := uos_AddFromFile(PlayerIndex1, afile2);
4. In5Index := uos_AddFromFile(PlayerIndex1, afile3);
5. ...

Each input has his own index and buffer of samples.
Those input-buffers can be changed before to sent to the output using DSP methods directly to the buffer.

[Fred vS]

Hello CM360.

I did test your demo and you were very close to the goal! 
All that was missing was the conversion of the input int16 samples to float32 compatible with your widget.
Int16 has a range of integer from -32768 to +32767 and float32 range of float from -1.0 to +1.0 (and that is what your widget likes).
There is the uos converter:

Code: Pascal  [Select][+][-]

1. function CvInt16ToFloat32(Inbuf: TDArFloat): TDArFloat; inline;
2. var
3.   x: cint32;
4.   arfl: TDArFloat;
5.   ps: PDArShort;
6. begin
7.   setlength(arfl,length(Inbuf));
8.   ps := @inbuf;
9.   for x := 0 to high(Inbuf) do
10.     arfl[x] := ps^[x] / 32768;
11.   Result := arfl;
12. end;
13.  

[EDIT] Will make it public in uos_flat.pas in next commit with uos_Int16ToFloat32().

And then in code:

Code: Pascal  [Select][+][-]

1. procedure Tform1.AddBuffer;
2. var
3.   mybuffertemp :array of single;
4.   i: integer;
5. begin
6.    mybuffertemp :=  CvInt16ToFloat32(uos_InputGetBuffer(PlayerIndex1, In1Index));

In attachment your updated project.
Note that you will need to add the /lib directory with the libraries needed.

[CM630]

Hi
Thanks
It seems working now!
There is some timing issue (3500 s have passed, 500 s are shown on the chart), but I suppose that TAChart cannot update quickly enough and some memory buffers are missed.

Code: Pascal  [Select][+][-]

1. procedure Tform1.AddBuffer;
2. var
3.   mybuffertemp :array of single;
4.   i: integer;
5. begin
6.    mybuffertemp :=  CvInt16ToFloat32(uos_InputGetBuffer(PlayerIndex1, In1Index));
7.    Label3.Caption := IntToStr((GetTickCount64 - starttime) div 1000) + ' s ; ' + IntToStr  (Length(thebuffer)) + ';  ' + IntToStr(length(mybuffertemp)) + '; '  + IntToStr(InputIndex1);
8.    for i:=0 to high(mybuffertemp) do
9.    begin
10.      if i mod 2 = 0 then
11.      begin
12.        Chart1LineSeriesLeft.AddXY(SampleIndex/44100,double(mybuffertemp[i]));
13.        inc(SampleIndex);
14.      end; //if
15.    end; //for
16. end;

AddXY is not the best way. I will fiddle around.
What I noticed is that when I use the Bluetooth headphones, after half a minute the AddBuffer stops being triggered, while when using an inbuilt microphone there is not such an issue.
Also, is there some selector for the source of the selector?
And what is supposed to happen when the audio source device is gone? I switch off the BT headphones, the AddBuffer stops being triggered, but no error is shown, is there a method for detection?

[Fred vS]

Hello CM630

I am very happy that it works finally for you.
To get the list of audio devices, take a look at DeviceInfos example.
In uos_AddFromDevIn() you can adjust the device to use with the "Device" parameter.
Also if you have problems to update the buffer, try with Latency = 0.8 and FramesCount = 8192 (or more).

Code: Pascal  [Select][+][-]

1. function uos_AddFromDevIn(PlayerIndex: cint32; Device: cint32; Latency: CDouble;
2.   SampleRate: CDouble; OutputIndex: cint32;
3.   SampleFormat: cint32; FramesCount : cint32; ChunkCount: cint32): cint32;
4. // Add a Input from Device Input with custom parameters
5. // PlayerIndex : Index of a existing Player
6. // Device ( -1 is default Input device )
7. // Latency  ( -1 is latency suggested )
8. // SampleRate : delault : -1 (44100)
9. // OutputIndex : Output index of used output// -1: all output, -2: no output, other cint32 refer to a existing OutputIndex  (if multi-output then OutName = name of each output separeted by ';')
10. // SampleFormat : default : -1 (2:Int16) (0: Float32, 1:Int32, 2:Int16)
11. // FramesCount : default : -1 (4096)
12. // ChunkCount : default : -1 (= 512)
13. //  result :  Output Index in array

And what is supposed to happen when the audio source device is gone?

I dont know, I dont have a usb or bluetooth audio device. 
But I suspect that it is the same like for usb memory stick, first stop the player and then remove the device.

PS: I did some commit in uos source following your suggestion (check for default device).

[CM630]

...
To get the list of audio devices, take a look at DeviceInfos example.

Hi Fred vS,I added a combobox and some code, to get a list of the input devices (maybe it will work only in Windows).
Something odd happens - the name of some of the devices are truncated (by uosDeviceInfos[].DeviceName):
Microphone Array (Realtek(R) Audio) is truncated to Microphone Array (Realtek(R) Au.
Headset (MUSICSOUND Hands-Free AG Audio) is truncated to Headset (MUSICSOUND Hands-Free.

Code: Pascal  [Select][+][-]

1. /////////////////// Demo how to use United Openlib of Sound ////////////////////
2.  
3.  
4.  
5.  
6. unit main_di;
7.  
8.  
9. {$mode objfpc}{$H+}
10.  
11.  
12. interface
13.  
14.  
15. uses
16.   uos_flat,
17.   Forms,
18.   Dialogs,
19.   SysUtils,
20.   Graphics,
21.   StdCtrls,
22.   ExtCtrls,
23.   Grids,
24.   Classes;
25.  
26.  
27. type
28.   tRecorderData =  record
29.     DevID: integer;
30.     DevName: String;
31.     ChanIn: integer;
32.     SRate: Double;
33.     DefaultDevIn : Boolean;
34.   end;
35.  
36.  
37.   tRecorders = array of tRecorderData;
38.  
39.  
40.   { TForm1 }
41.   TForm1 = class(TForm)
42.     Button1: TButton;
43.     Button2: TButton;
44.     ComboBox1: TComboBox;
45.     Edit1: TEdit;
46.     Label1: TLabel;
47.     Label2: TLabel;
48.     Label3: TLabel;
49.     Label4: TLabel;
50.     Label5: TLabel;
51.     PaintBox1: TPaintBox;
52.     StringGrid1: TStringGrid;
53.     procedure Button1Click(Sender: TObject);
54.     procedure Button2Click(Sender: TObject);
55.     procedure FormActivate(Sender: TObject);
56.     procedure FormCreate(Sender: TObject);
57.     procedure FormDestroy(Sender: TObject);
58.     procedure PaintBox1Paint(Sender: TObject);
59.     procedure CheckInfos();
60.   private
61.     { private declarations }
62.   public
63.     { public declarations }
64.   end;
65.  
66.  
67.  
68.  
69. procedure uos_logo();
70.  
71.  
72. var
73.   Form1: TForm1;
74.   BufferBMP: TBitmap;
75.   Recorders : tRecorders;
76.  
77.  
78. implementation
79.  
80.  
81. {$R *.lfm}
82.  
83.  
84. { TForm1 }
85.  
86.  
87.  
88.  
89. procedure TForm1.FormActivate(Sender: TObject);
90. {$IFDEF Darwin}
91. var
92.   opath: string;
93.             {$ENDIF}
94. begin
95.   uos_logo();
96.       {$IFDEF Windows}
97.      {$if defined(cpu64)}
98.   edit1.Text := application.Location + 'lib\Windows\64bit\LibPortaudio-64.dll';
99. {$else}
100.   edit1.Text := application.Location + 'lib\Windows\32bit\LibPortaudio-32.dll';
101.    {$endif}
102.  {$ENDIF}
103.  
104.  
105.    {$IFDEF Darwin}
106.    {$IFDEF CPU32}
107.   opath := application.Location;
108.   opath := copy(opath, 1, Pos('/uos', opath) - 1);
109.   edit1.Text := opath + '/lib/Mac/32bit/LibPortaudio-32.dylib';
110.    {$ENDIF}
111.     {$IFDEF CPU64}
112.    opath := application.Location;
113.   opath := copy(opath, 1, Pos('/uos', opath) - 1);
114.   Edit1.Text := opath + '/lib/Mac/64bit/LibPortaudio-64.dylib';
115.     {$ENDIF}
116.     {$ENDIF}
117.  
118.  
119.  {$if defined(CPUAMD64) and defined(openbsd) }
120.  edit1.Text := application.Location + 'lib/OpenBSD/64bit/LibPortaudio-64.so';
121.  {$ENDIF}
122.  
123.  
124.   {$if defined(CPUAMD64) and defined(linux) }
125.   edit1.Text := application.Location + 'lib/Linux/64bit/LibPortaudio-64.so';
126.   {$endif}
127.  
128.  
129. {$if defined(cpu86) and defined(linux)}
130.   edit1.Text := application.Location + 'lib/Linux/32bit/LibPortaudio-32.so';
131. {$endif}
132.  
133.  
134.   {$if defined(linux) and defined(cpuarm)}
135.   edit1.Text := application.Location + 'lib/Linux/arm_raspberrypi/libportaudio-arm.so';
136. {$endif}
137.  
138.  
139.  {$if defined(linux) and defined(cpuaarch64)}
140.   edit1.Text := ordir + 'lib/Linux/aarch64_raspberrypi/libportaudio_aarch64.so';
141.   {$ENDIF}
142.  
143.  
144.  {$IFDEF freebsd}
145.     {$if defined(cpu64)}
146.   edit1.Text := application.Location + 'lib/FreeBSD/64bit/libportaudio-64.so';
147. {$else}
148.   edit1.Text := application.Location + 'lib/FreeBSD/32bit/libportaudio-32.so';
149. {$endif}
150.  
151.  
152.             {$ENDIF}
153.   //////////////////////////////////////////////////////////////////////////
154.  
155.  
156. end;
157.  
158.  
159. procedure TForm1.PaintBox1Paint(Sender: TObject);
160. begin
161.   PaintBox1.Canvas.Draw(0, 0, BufferBMP);
162. end;
163.  
164.  
165.  
166.  
167.  
168.  
169. procedure TForm1.CheckInfos();
170. var
171.   x: integer;
172.   RecIndex: integer = 0;
173. begin
174.   SetLength (Recorders,0);
175.  
176.  
177.   uos_GetInfoDevice();
178.  
179.  
180.   label2.Caption := 'Devices Count = ' + IntToStr(uosDeviceCount);
181.  
182.  
183.   label3.Caption := 'Default Device IN = ' + IntToStr(uosDefaultDeviceIN);
184.  
185.  
186.   label4.Caption := 'Default Device OUT = ' + IntToStr(uosDefaultDeviceOUT);
187.  
188.  
189.   stringgrid1.rowcount := uosDeviceCount + 1;
190.  
191.  
192.  
193.  
194.   x := 1;
195.  
196.  
197.   while x < uosDeviceCount + 1 do
198.   begin
199.     if (lowercase(uosDeviceInfos[x - 1].DeviceType) = 'in') and (lowercase(uosDeviceInfos[x - 1].HostAPIName) = 'mme') and (lowercase(uosDeviceInfos[x - 1].DeviceName)<>'microsoft sound mapper - input') then
200.     begin
201.       SetLength (Recorders,length(Recorders)+1);
202.       Recorders[RecIndex].DevID := uosDeviceInfos[x - 1].DeviceNum;
203.       Recorders[RecIndex].DevName := uosDeviceInfos[x - 1].DeviceName;
204.       Recorders[RecIndex].ChanIn := uosDeviceInfos[x - 1].ChannelsIn;
205.       Recorders[RecIndex].SRate := uosDeviceInfos[x - 1].SampleRate;
206.       Recorders[RecIndex].DefaultDevIn := uosDeviceInfos[x - 1].DefaultDevIn;
207.       inc (RecIndex);
208.     end;
209.  
210.  
211.     stringgrid1.Cells[0, x] := IntToStr(uosDeviceInfos[x - 1].DeviceNum);
212.     stringgrid1.Cells[1, x] := uosDeviceInfos[x - 1].DeviceName;
213.     if uosDeviceInfos[x - 1].DefaultDevIn = True then
214.       stringgrid1.Cells[2, x] := 'Yes'
215.     else
216.       stringgrid1.Cells[2, x] := 'No';
217.  
218.  
219.     if uosDeviceInfos[x - 1].DefaultDevOut = True then
220.       stringgrid1.Cells[3, x] := 'Yes'
221.     else
222.       stringgrid1.Cells[3, x] := 'No';
223.  
224.  
225.     stringgrid1.Cells[4, x]  := IntToStr(uosDeviceInfos[x - 1].ChannelsIn);
226.     stringgrid1.Cells[5, x]  := IntToStr(uosDeviceInfos[x - 1].ChannelsOut);
227.     stringgrid1.Cells[6, x]  := floattostrf(uosDeviceInfos[x - 1].SampleRate, ffFixed, 15, 0);
228.     stringgrid1.Cells[7, x]  := floattostrf(uosDeviceInfos[x - 1].LatencyHighIn, ffFixed, 15,;
229.     stringgrid1.Cells[8, x]  := floattostrf(uosDeviceInfos[x - 1].LatencyHighOut,
230.       ffFixed, 15,;
231.     stringgrid1.Cells[9, x]  := floattostrf(uosDeviceInfos[x - 1].LatencyLowIn, ffFixed, 15,;
232.     stringgrid1.Cells[10, x] :=
233.       floattostrf(uosDeviceInfos[x - 1].LatencyLowOut, ffFixed, 15,;
234.     stringgrid1.Cells[11, x] := uosDeviceInfos[x - 1].HostAPIName;
235.     stringgrid1.Cells[12, x] := uosDeviceInfos[x - 1].DeviceType;
236.  
237.  
238.     Inc(x);
239.   end;
240.  
241.  
242.   if Length(Recorders) > 0 then
243.   begin
244.     ComboBox1.Clear;
245.     for x := 0 to High(Recorders) do
246.       begin
247.         ComboBox1.Items.Add (Recorders[x].DevName);
248.         if Recorders[x].DefaultDevIn = True then
249.           ComboBox1.ItemIndex := x;
250.       end;//for
251.   end;
252.  
253.  
254. end;
255.  
256.  
257.  
258.  
259. procedure TForm1.Button1Click(Sender: TObject);
260. begin
261.   // Load the library
262.   //function  uos_loadlib(PortAudioFileName, SndFileFileName, Mpg123FileName, Mp4ffFileName, FaadFileName, opusfilefilename: PChar) : LongInt;
263.  
264.  
265.   if uos_LoadLib(PChar(edit1.Text), nil, nil, nil, nil, nil) = 0 then
266.   begin
267.     form1.hide;
268.     button1.Caption := 'PortAudio is loaded...';
269.     button1.Enabled := False;
270.     edit1.ReadOnly  := True;
271.  
272.  
273.     CheckInfos();
274.     form1.Height   := 688;
275.     form1.Position := poScreenCenter;
276.     form1.Show;
277.   end
278.   else
279.   begin
280.     if uosLoadResult.PAloaderror = 1 then
281.       MessageDlg(edit1.Text + ' do not exist...', mtWarning, [mbYes], 0);
282.     if uosLoadResult.PAloaderror = 2 then
283.       MessageDlg(edit1.Text + ' do not load...', mtWarning, [mbYes], 0);
284.   end;
285. end;
286.  
287.  
288. procedure TForm1.Button2Click(Sender: TObject);
289. begin
290.   CheckInfos();
291. end;
292.  
293.  
294.  
295.  
296. procedure uos_logo();
297. var
298.   xpos, ypos: integer;
299.   ratio: double;
300. begin
301.   xpos      := 0;
302.   ypos      := 0;
303.   ratio     := 1;
304.   BufferBMP := TBitmap.Create;
305.   with form1 do
306.   begin
307.     form1.PaintBox1.Parent.DoubleBuffered := True;
308.     PaintBox1.Height := round(ratio * 116);
309.     PaintBox1.Width  := round(ratio * 100);
310.     BufferBMP.Height := PaintBox1.Height;
311.     BufferBMP.Width  := PaintBox1.Width;
312.     BufferBMP.Canvas.AntialiasingMode := amOn;
313.     BufferBMP.Canvas.Pen.Width := round(ratio * 6);
314.     BufferBMP.Canvas.brush.Color := clmoneygreen;
315.     BufferBMP.Canvas.FillRect(0, 0, PaintBox1.Width, PaintBox1.Height);
316.     BufferBMP.Canvas.Pen.Color := clblack;
317.     BufferBMP.Canvas.brush.Color := $70FF70;
318.     BufferBMP.Canvas.Ellipse(round(ratio * (22) + xpos),
319.       round(ratio * (30) + ypos), round(ratio * (72) + xpos),
320.       round(ratio * (80) + ypos));
321.     BufferBMP.Canvas.brush.Color := clmoneygreen;
322.     BufferBMP.Canvas.Arc(round(ratio * (34) + xpos), round(ratio * ( + ypos),
323.       round(ratio * (58) + xpos), round(ratio * (32) + ypos), round(ratio * (58) + xpos),
324.       round(ratio * (20) + ypos), round(ratio * (46) + xpos),
325.       round(ratio * (32) + xpos));
326.     BufferBMP.Canvas.Arc(round(ratio * (34) + xpos), round(ratio * (32) + ypos),
327.       round(ratio * (58) + xpos), round(ratio * (60) + ypos), round(ratio * (34) + xpos),
328.       round(ratio * (48) + ypos), round(ratio * (46) + xpos),
329.       round(ratio * (32) + ypos));
330.     BufferBMP.Canvas.Arc(round(ratio * (-28) + xpos), round(ratio * (18) + ypos),
331.       round(ratio * (23) + xpos), round(ratio * (80) + ypos), round(ratio * (20) + xpos),
332.       round(ratio * (50) + ypos), round(ratio * (3) + xpos), round(ratio * (38) + ypos));
333.     BufferBMP.Canvas.Arc(round(ratio * (70) + xpos), round(ratio * (18) + ypos),
334.       round(ratio * (122) + xpos), round(ratio * (80) + ypos),
335.       round(ratio * (90 - xpos)),
336.       round(ratio * (38) + ypos), round(ratio * (72) + xpos),
337.       round(ratio * (50) + ypos));
338.     BufferBMP.Canvas.Font.Name := 'Arial';
339.     BufferBMP.Canvas.Font.Size := round(ratio * 10);
340.     BufferBMP.Canvas.TextOut(round(ratio * (4) + xpos),
341.       round(ratio * (83) + ypos), 'United Openlib');
342.     BufferBMP.Canvas.Font.Size := round(ratio * 7);
343.     BufferBMP.Canvas.TextOut(round(ratio * (20) + xpos),
344.       round(ratio * (101) + ypos), 'of');
345.     BufferBMP.Canvas.Font.Size := round(ratio * 10);
346.     BufferBMP.Canvas.TextOut(round(ratio * (32) + xpos),
347.       round(ratio * (98) + ypos), 'Sound');
348.   end;
349. end;
350.  
351.  
352. procedure TForm1.FormCreate(Sender: TObject);
353. begin
354.   Form1.Height := 126;
355. end;
356.  
357.  
358. procedure TForm1.FormDestroy(Sender: TObject);
359. begin
360.   if button1.Enabled = False then
361.     uos_free;
362. end;
363.  
364.  
365. end.

[Fred vS]

Hello CM630.
Are the names of the devices also truncated in the grid of the "original" DeviceInfos demo?
If it is ok, then maybe there is a property in LCL combobox to expand.
Sorry, I dont have a Windows machine to test.
Also I dont know good how Lazarus components work.
[EDIT] Tested here on Linux GTK2 and the name in combobox is not truncated.

[CM630]

The names are truncated in the StringGrind (your original app) and also in the ComboBox.
I have dug trough the registry for Microphone Array (Realtek(R) Audio, I did not find a truncated version of it, that is why I think there is something wrong in uosDeviceInfos[].DeviceName.
I will try to see what uosDeviceInfos does, maybe I will find something there.

[Fred vS]

The names are truncated in the StringGrind (your original app) and also in the ComboBox.
I have dug trough the registry for Microphone Array (Realtek(R) Audio, I did not find a truncated version of it, that is why I think there is something wrong in uosDeviceInfos[].DeviceName.
I will try to see what uosDeviceInfos does, maybe I will find something there.

For Windows there is a limitation of the Wave API only having 32 chars in a struct for the device name.
https://portaudio.music.columbia.narkive.com/fyHpcZW3/mme-device-names-are-truncated

Here how to get the complete name using DirectX in Pascal:
https://stackoverflow.com/questions/1449162/get-the-full-name-of-a-wavein-device

[CM630]

In the link you have provided is a description of how to do it without DX.
Anyway, that is another issue.

I have attached a working example (the so and dll files shall be added).

Working is not the exact word - it has problems with Bluetooth and some data is lost while adding data to TAChart, because I am not using the best data populating method for that (Waveform graph populates TAChart better, so it seems to work fine with it).

Once again, thanks to everyone for the assistance!