Recent

Author Topic: Midi Packet Size is different between 32carbon and 64 bit cocoa  (Read 1001 times)

josh

  • Hero Member
  • *****
  • Posts: 754
Hi
I have been battling with getting midi working with Cocoa.
Code that works fine on Carbon; give incorrect data error on Cocoa.

I have tracked down a problem; that I think is causing it; but do not know the best way to correct what I see as a BUG in the Cocoa Implementation of CoreMidi

A midi Data Packet is a record
timestamp:uInt64;
length:uInt16;
data:array[0..255] of byte

The record structure has a size of 268bytes on carbon; where it shows the correct information; but on cocoa the exact same structure give a size of 272bytes.
A similar issue exists with MidiPacketList which has 272(carbon) and 280 (cocoa).

Could this be a memory allignment issue?

Anyone have an idea of how to fix; as core midi and other mac frameworks exists; it is possible that the same could exists within these aswell.

Hope the above makes sense.

« Last Edit: June 19, 2019, 11:32:18 pm by josh »
Development Installation Lazarus 1.3, FPC 2.7.1,Windows 7/8 32/64, OSX, *nix

Test Environment Lazarus & FPC Trunk on Windows and OSX (Cocoa Mainly on OSX). Testing also Crosscompile windows to OSX.. 
Any posts made from 2015 will be based on Lazarus Trunk.

engkin

  • Hero Member
  • *****
  • Posts: 2513
Re: Midi Packet Size is different between 32carbon and 64 bit cocoa
« Reply #1 on: June 19, 2019, 11:41:57 pm »
use packed record.

josh

  • Hero Member
  • *****
  • Posts: 754
Re: Midi Packet Size is different between 32carbon and 64 bit cocoa
« Reply #2 on: June 20, 2019, 12:21:06 am »
Hi

I have tried using packed
Code: [Select]
MIDIPacket = packed record
timeStamp: MIDITimeStamp;
length: UInt16;
data: array [0..255] of Byte;
end;                               

But the size is still 272 bytes.
Development Installation Lazarus 1.3, FPC 2.7.1,Windows 7/8 32/64, OSX, *nix

Test Environment Lazarus & FPC Trunk on Windows and OSX (Cocoa Mainly on OSX). Testing also Crosscompile windows to OSX.. 
Any posts made from 2015 will be based on Lazarus Trunk.

engkin

  • Hero Member
  • *****
  • Posts: 2513
Re: Midi Packet Size is different between 32carbon and 64 bit cocoa
« Reply #3 on: June 20, 2019, 12:25:42 am »
Your first post shows:
timestamp:uInt64;

Your last post shows:
timeStamp: MIDITimeStamp;

Is MIDITimeStamp of uInt64 type? if not, replace it with uInt64.

engkin

  • Hero Member
  • *****
  • Posts: 2513
Re: Midi Packet Size is different between 32carbon and 64 bit cocoa
« Reply #4 on: June 20, 2019, 12:27:19 am »
Wait!!

engkin

  • Hero Member
  • *****
  • Posts: 2513
Re: Midi Packet Size is different between 32carbon and 64 bit cocoa
« Reply #5 on: June 20, 2019, 12:32:25 am »
That was not packed, sorry. Try this:
Code: Pascal  [Select]
  1. {$push}
  2. {$PackRecords 4}
  3. type
  4.   TMIDIPacket = record
  5.     timestamp:uInt64;
  6.     length:uInt16;
  7.     data:array[0..255] of byte;
  8.   end;
  9. {$pop}

This should be 268 bytes.
Packed would be 266 bytes.

josh

  • Hero Member
  • *****
  • Posts: 754
Re: Midi Packet Size is different between 32carbon and 64 bit cocoa
« Reply #6 on: June 20, 2019, 01:03:49 am »
hi

Code: [Select]
{$push}
{$PackRecords 4}
MIDIPacket =  record
timeStamp: UInt64;
length: UInt16;
data: array [0..255] of Byte;
end;
{$pop}       

The above code has no effect on cocoa; it is still 272bytes...
Development Installation Lazarus 1.3, FPC 2.7.1,Windows 7/8 32/64, OSX, *nix

Test Environment Lazarus & FPC Trunk on Windows and OSX (Cocoa Mainly on OSX). Testing also Crosscompile windows to OSX.. 
Any posts made from 2015 will be based on Lazarus Trunk.

engkin

  • Hero Member
  • *****
  • Posts: 2513
Re: Midi Packet Size is different between 32carbon and 64 bit cocoa
« Reply #7 on: June 20, 2019, 03:18:07 am »
Probably your target is 64bit. Back to packed records:
Code: Pascal  [Select]
  1. type
  2.   MIDIPacket = packed record
  3.     timestamp:uInt64;
  4.     length:uInt16;
  5.     padding:array[0..1] of byte;
  6.     data:array[0..255] of byte;
  7.   end;
« Last Edit: June 20, 2019, 03:26:25 am by engkin »

Thaddy

  • Hero Member
  • *****
  • Posts: 9187
Re: Midi Packet Size is different between 32carbon and 64 bit cocoa
« Reply #8 on: June 20, 2019, 07:28:51 am »
The last is correct indeed. Midi streams need packed records and {$packrecords 1} not 4! There can not be or should not be any slack spacing.
This has nothing to do with alignment, but with reading/writing the stream correctly. Stored data should always be packed and Midi is no exception.

IOW: byte alignment of the members. The in-memory record start itself will always be aligned to current alignment so there is no speed disadvantage in any way.
« Last Edit: June 20, 2019, 07:31:44 am by Thaddy »
also related to equus asinus.

josh

  • Hero Member
  • *****
  • Posts: 754
Re: Midi Packet Size is different between 32carbon and 64 bit cocoa
« Reply #9 on: June 20, 2019, 11:23:26 am »
Hi

Something odd is going on, on my end.

If I create a midipacket type I my application unit; with pack record 4, then the structure size is correct at 268.

If I use unit MACOSALL; which uses MIDISERVICES definition, then the size is incorrect.
If I modify the MIDIServies definition it still is not correct.

Now if I make a syntax error in the MIDIservices definition; expecting a compile error; it does not and still compiles. So I then done a clean all build; and still it compiles .

So question now is how can I modify midiservices.

I have attached a project that show the issue, you can comment out the define statement to use either macosall definition or its own.

Hopefully someone can replicate.

« Last Edit: June 20, 2019, 11:38:15 am by josh »
Development Installation Lazarus 1.3, FPC 2.7.1,Windows 7/8 32/64, OSX, *nix

Test Environment Lazarus & FPC Trunk on Windows and OSX (Cocoa Mainly on OSX). Testing also Crosscompile windows to OSX.. 
Any posts made from 2015 will be based on Lazarus Trunk.

Thaddy

  • Hero Member
  • *****
  • Posts: 9187
Re: Midi Packet Size is different between 32carbon and 64 bit cocoa
« Reply #10 on: June 20, 2019, 12:16:32 pm »
Josh, read my explanation above: $packrecords 4 is wrong. You MUST use 1, which is the default. Think!!!!!!
(note I am an avid audio and midi programmer)
Record packing does not influence alignment of the record itself. <sigh>
« Last Edit: June 20, 2019, 12:22:16 pm by Thaddy »
also related to equus asinus.

josh

  • Hero Member
  • *****
  • Posts: 754
Re: Midi Packet Size is different between 32carbon and 64 bit cocoa
« Reply #11 on: June 20, 2019, 12:40:14 pm »
Hi Thaddy,

With carbon, the midipacket in macosall;midiservices is a size of 268.
So for it to work with the same in cocoa and carbon the midipacket size must be the same ie 268.

packrecords 1 create a size of 266  which is wrong
packrecords 4 creates a size of 268
packrecords 8 creates a size of 272
without packrecords a size of 272

If I use Align 1; i still get 266

I see similar problem with midipacketlist type as well..

If the size is not the same in carbon and cocoa should it be placed in bugtracker; as the issue is with coremidi implementation.


Development Installation Lazarus 1.3, FPC 2.7.1,Windows 7/8 32/64, OSX, *nix

Test Environment Lazarus & FPC Trunk on Windows and OSX (Cocoa Mainly on OSX). Testing also Crosscompile windows to OSX.. 
Any posts made from 2015 will be based on Lazarus Trunk.

rvk

  • Hero Member
  • *****
  • Posts: 3842
Re: Midi Packet Size is different between 32carbon and 64 bit cocoa
« Reply #12 on: June 20, 2019, 01:03:33 pm »
According to this the MIDIPacket and MIDIPacketList should have the 4 byte alignment.

Quote
Setting {$Align 1} for the whole file was not the correct solution. Instead only the two records (MIDIPacket and MIDIPacketList) should have 4 byte alignments instead of 8 byte alignment.
https://stackoverflow.com/a/37748973/1037511

Quote
Note the #pragma pack(push, 4) above the definition of MIDIPacket in the CoreMIDI.h header file.

Thaddy

  • Hero Member
  • *****
  • Posts: 9187
Re: Midi Packet Size is different between 32carbon and 64 bit cocoa
« Reply #13 on: June 20, 2019, 03:36:03 pm »
I am sorry but that is the wrong answer. The alignment is 1, not 4. I am writing music software for the past 20 years. There's no padding.
In Freepascal the record in-memory will be properly aligned. The stored records are packed to 1. The reason that 4 works too is because that is the minimum size of the record header members.
See also: https://en.wikipedia.org/wiki/MIDI#Technical_specifications

Just to be sure: the midi format is completely platform independent.

That "approved"  stackoverflow entry is simply wrong! You can test that by simply opening a midi file as a stream.
Which, btw, is its intended use.

(some people even stole my code - the reverb - without proper representation of its origins: Jezar's Freeverb 3, translated and optimized by me and stolen by Christian!, that's not midi, but... he still did not correct that even after asking for it politely. That's OK TobyBear and me know who wrote it...Stays freeware anyway... But I still hold a grudge because of misrepresentation, so Christian if you read this, put it right after all those years)
« Last Edit: June 20, 2019, 04:08:06 pm by Thaddy »
also related to equus asinus.

skalogryz

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 2283
    • havefunsoft.com
Re: Midi Packet Size is different between 32carbon and 64 bit cocoa
« Reply #14 on: June 20, 2019, 04:29:57 pm »
Something odd is going on, on my end.

If I create a midipacket type I my application unit; with pack record 4, then the structure size is correct at 268.

If I use unit MACOSALL; which uses MIDISERVICES definition, then the size is incorrect.
If I modify the MIDIServies definition it still is not correct.
...
So question now is how can I modify midiservices.
It's not wrong or odd, but expected. If you change the sources of FPC packages (and in this case you do)
you need to recompile AND reinstall the modified packages.
(if I remember correctly, it should as easy as entering fpcsource/packages/unvint and running a command:
make clean all install
BUT depending on where your sources are (need of root access?) it might be a bit trickier.
"install" is likely to require you the root access)

Whenever you run a compiler it doesn't check the package source, instead it goes straight to recompiled binaries.
(Btw, Lazarus can recompile a "Lazarus package" automatically, but it cannot recompile "FPC packages" automatically. And both MacOSAll and MIDIServices are FPC packages)
...
and here's the project management problem.
Even if you provide the patch (and I think you should) to correct MIDIServices declaration, you will not get the fixed version until the next official FPC release (which will come with the correctly compiled header).
« Last Edit: June 20, 2019, 04:34:34 pm by skalogryz »
Patron Cocoa Widgetset development https://www.patreon.com/skalogryz