Is it any way to programatically register a default application for a custom file-ending?
You do that in your app's Info.plist file. Since Lazarus does not provide any way to edit that in the IDE, you'll need to edit the Info.plist with a text editor. See any app's Info.plist to see how they associate a file type with an app action (for example, see how Lazarus does it). See also Apple's docs on the subject.
You may also find this page helpful with other Mac related questions:
https://macpgmr.github.io/MacXPlatform/LazXProj.html