Detecting if a file is readable

[ChrisR]

On Windows and Linux (and older versions of MacOS), the LazFileUtils  FileIsReadable function was sufficient to determine if a file has read permissions set. However, this returns the global permissions for this file. With recent/upcoming versions of MacOS, applications by default are supposed to work in a sandbox with limited entitlements. They are only supposed to be able to read a file if the user has explicitly associated it with that application, for example by dragging and dropping or by using a file open dialog.

Consider an application that wants to read the file ~/Desktop/notes.txt. This files global permissions mean that this file CAN be read by programs. However, our specific program is not entitled to read the file. Unfortunately, FileIsReadable simply reports that the file is readable globally. Yet when the application actually attempts to read the file it has an error.

Does anyone have a trick they use to see the permissions from the perspective of the invoking application?

I provide a sample program to demonstrate the issue, and my ugly kludge
  https://bugs.freepascal.org/view.php?id=37403
But I hope someone has a Moree elegant solution.

[trev]

There's been several threads in the Apple Dev Forums over the last couple of years asking for an API to determine this - Eskimo usually suggests lodging an enhancement request which many have done to no avail.

The other related thread was how to ask for full disk access rather than simply failing. The answer to that one has been found: https://github.com/gnachman/iTerm2/pull/382

As one Apple Dev Forum user suggested - "the writing is on the wall, get used to writing apps for iOS" which was rather prophetic as it turns out.

[ChrisR]

Thanks for your insight. Do you think that the LazFileUtils FileIsReadable function should be modified so that on MacOS it uses my ugly hack? At the moment, the function can return a misleading value. Your experience suggests there is not an official API way to detect if an application actually is able to read a file.

I had really expected the Apple API function isReadableFileAtPath
  https://developer.apple.com/documentation/foundation/nsfilemanager/1418292-isreadablefileatpath

would provide the right answer, but as the demo project in my bug report demonstrates, it does not give the desired information. The only true way to know if a program has permission to read a file appears to be to read a byte from the file.

[trev]

Do you think that the LazFileUtils FileIsReadable function should be modified so that on MacOS it uses my ugly hack? At the moment, the function can return a misleading value.

If that code can return a misleading value, probably not

[ChrisR]

The Apple API reports a misleading value. However, the ugly hack in my bug report DOES accurately report the true value regarding whether the application can read a file or not. It does this by physically reading the first byte of the file and using try...except to detect a failure. If this hack replaced LazFileUtils FileIsReadable() for MacOS, it would support the write-once compile anywhere nature of Lazarus, with projects that use FileIsReadable accurately detecting that a file could not be read.

Here is the ugly hack:


function IsReadable(fnm: string): boolean;
var
  f: file;
  b: byte;
begin
  result := false;
  if not fileexists(fnm) then exit;
  if FileUtil.FileSize(fnm) < 1 then exit;
  AssignFile(f, fnm);
  {$I+}
  try
    FileMode := fmOpenRead;  //Set file access to read only
    Reset(f, 1);
    if ioresult <> 0 then
       exit;
    b := 0;
    BlockRead(f, b, sizeof(b));
    CloseFile(f);
    result := true;
  except
    result := false;
  end;
  if result then exit;
  writeln('Unable to read file (not in sandbox?): '+fnm);
end;

[ChrisR]

Here is my suggested change for unixlazfileutils.inc

Code: Pascal  [Select][+][-]

1. {$IFDEF Darwin}
2. function FileIsReadable(const AFilename: string): boolean;
3. var
4.   f: file;
5.   b: byte;
6. begin
7.   Result := BaseUnix.FpAccess(AFilename, BaseUnix.R_OK) = 0;
8.   if not Result then exit; //globally not readable
9.   if FileSizeUtf8(AFilename) < 1 then exit(false);
10.   AssignFile(f, AFilename);
11.   {$I+}
12.   try
13.     FileMode := fmOpenRead;  //Set file access to read only
14.     Reset(f, 1);
15.     if ioresult <> 0 then
16.        exit;
17.     b := 0;
18.     BlockRead(f, b, sizeof(b));
19.     CloseFile(f);
20.     result := true;
21.   except
22.     result := false;
23.   end;
24. end;
25. {$ELSE}
26. function FileIsReadable(const AFilename: string): boolean;
27. begin
28.   Result := BaseUnix.FpAccess(AFilename, BaseUnix.R_OK) = 0;
29. end;
30. {$ENDIF}

[Jonas Maebe]

Maybe it's simply a bug in the OS? It's a beta after all. Or does it work the same in a sandboxed application on 10.14/10.15?

[ChrisR]

10.14 was less strict. I do not have access to 10.15. I think this reflects new enforcement of entitlements. MacOS seems to treat an application as more suspicious if you download it from the web than if you compile it locally. While this is reasonable, it makes it harder to develop robust methods of handling the security measures gracefully, as the same app behaves differently if it is compiled locally, downloaded but run from the terminal or downloaded and run by clicking the icon in the finder.

[Jonas Maebe]

Still, there's a difference between being more strict and file access APIs being less strict than actually accessing the files.

[trev]

The "full disk access" issues started to appear with the betas of macOS 10.14 Mojave in 2018 and  increased with Catalina and now Big Sur (Betas).

The additional Catalina "privacy protections" are detailed in the WWDC2019 video "Advances in macOS Security" (https://developer.apple.com/videos/play/wwdc2019/701/) at the 21 minute mark.

I've not come across any summary of changes for Big Sur yet - any references are pretty scattered.

[Jonas Maebe]

Again: no matter how restrictive or liberal the access policies are, the OS functions to verify access should take this into account (otherwise, what is the point of having those functions?). Please file radars/submit feedback via the feedback application, or otherwise these things may never get fixed.

[trev]

No dispute there Jonas, but it does seem Apple has a different philosophy - a note by Apple on the isReadableFileAtPath method reads in part:

It's far better to attempt an operation (such as loading a file or creating a directory), check for errors, and handle those errors gracefully than it is to try to figure out ahead of time whether the operation will succeed.

which, as you say, makes this method pointless today with the additional security and privacy mechanisms implemented since it was introduced in macOS 10.0.

Further digging on this method reveals that it is "essentially" a wrapper around the BSD access system call which explains its failure modes that ChrisR experienced.

[Jonas Maebe]

No dispute there Jonas, but it does seem Apple has a different philosophy - a note by Apple on the isReadableFileAtPath method reads in part:

It's far better to attempt an operation (such as loading a file or creating a directory), check for errors, and handle those errors gracefully than it is to try to figure out ahead of time whether the operation will succeed.

That's because between the time of testing access and performing the access, the situation might have changed. E.g. the user may have made a file file read-only, or the volume that contained the file may have been unmounted. It's not because those routines are supposed to give different results from the actual access.

which, as you say, makes this method pointless today with the additional security and privacy mechanisms implemented since it was introduced in macOS 10.0.

Even on Mac OS X 10.0 all of the above could already happen.

Further digging on this method reveals that it is "essentially" a wrapper around the BSD access system call which explains its failure modes that ChrisR experienced.

The BSD "open" and "read" APIs do take into account sandboxing (otherwise they would present truck-sized holes in the entire sandbox concept). The access man page does say you should only use it for UI elements or optimisation purposes because by the time the actual access happens the situation may have changed (as mentioned above).

[ChrisR]

Pragmatically, I advocate changing the Lazarus fileutils function FileIsReadable() to report if the file is readable by the current application, not merely that the file system says it is readable by the user. I use this call not only prior to reading a file, but also when validating the most-recently-used menu items. In this case, I do not want to display a file in the menu if I know the application will be unable to load it. Since a MacOS applications ability to read a file depends on whether it is launched from the terminal or the finder, whether the operating system thinks the application has been updated, etc. the current method is unreliable and operates differently from other operating systems. My suggestion makes the FileIsReadable behave similar to how it behaves with other operating systems.

[Jonas Maebe]

I would advocate reporting a bug. This is a beta OS. It's like when Lazarus 2.0.10 worked around a bug in FPC 3.2.0RC1, which then made in incompatible with the final FPC 3.2.0 where the bug was fixed.

Opening all files and reading from them may work around this issue, but it may not be necessary in the first place and it's guaranteed to be much slower. It may make programs unusably slow if they try to check permissions for a couple of thousand files in a folder (like pictures or so) for no good reason.